Parinfer
building table of contents...
Let's simplify the way we write Lisp...
Parinfer is a proof-of-concept editor mode for Lisp programming languages. It simplifies the way we write Lisp by auto-adjusting parens when indentation changes and vice versa. The hope is to make basic Lisp-editing easier for newcomers and experts alike, while still allowing existing plugins like Paredit to satisfy the need for more advanced operations.
Lisp is often dismissed as an undesirable programming syntax due to excessive parentheses. Those who have adopted Lisp have long recognized its amazing strengths, but there is still a widely held uncertainty among newcomers about how or even why we must manage so many parens. As a result, Lisp's unique power remains invisible to most under this guise of difficulty.
Newcomers aren't satisfied with the current tools designed for editing Lisp. Expert-level editors (Emacs, Vim) and advanced hotkeys (Paredit) are powerful but steepen the learning curve. And alternative syntaxes (Lisps without parens) have faltered since they sacrifice some of Lisp's power that seasoned users aren't willing to part with.
Parinfer is a new system that tries instead to fix this problem at its source. We formally define the relationship between Parens and Indentation. With it, we create an intuitive editor mode to make paren management fun and easy without sacrificing power. We demonstrate this through interactive proof-of-concept demos here, providing capabilities for:
NOTE: When I say "parens" (parentheses), I also mean [square] or {curly} brackets. Some Lisps (e.g. Racket, Clojure) use these extra delimiters to help visually separate certain constructs.
Most programming languages have several syntax rules. Lisp has one: everything is a list. The first element is a function name, and the rest are its arguments. Thus, the language is simply a collection of compile- and run-time functions, trivially extensible.
But parentheses in Lisp are infamous for bunching together at the end of long expressions. This indentation convention can be jarring at first if you are used to curly braces in other languages being on their own lines:
The idea behind this convention is to make every line inform with content rather than just parens. Readability is helped by employing a Python-like indentation style. This achieves a sort of balance— Indentation allows you to skim while the parens allow you to inspect:
Though both perspectives are visible at once, we must focus on one at a time. A LEGO analogy helps here. Imagine each list in the previous example as a LEGO block stacked over its parent. Checking the sides to see the layers below is like checking the parens at the end of a line.
This is a physical analog to the way we read Lisp code. Now let's look at the space of tooling solutions that we use for writing Lisp code and specifically how Parinfer can help.
In the previous section, we saw how Parens and Indentation provide two perspectives for reading Lisp code. Unfortunately, this incurs some redundancy when writing Lisp since we must edit both for every change, ensuring that one will correctly imply the other.
There are existing tools to help with this. To best represent how Parinfer compares to them, we will represent this complex space of tooling in a way that can be visually compared below.
The menial and default way to edit Lisp is to manually ensure our parens are balanced after inserting or removing them. Then, we adjust the indentation to match it in kind. This back-and-forth happens for most editing tasks.
Existing tools automate some of these editing tasks. For example, Paredit forces you to transform or add parens in a balanced way through special hotkeys. And Auto-indent allows you to auto-correct indentation of selected lines when desired. This automates the tasks, but the back-and-forth actions are still manually triggered.
Parinfer is a new tool to combine and simplify this type of automation by naturally keeping Parens and Indentation in lockstep. It formally infers changes to one based on the other. The back-and-forth actions have been reduced with special modes, which we will explore next.
As we saw with the previous visualization, Parinfer will infer some changes to keep Parens and Indentation inline with one another. To keep this inference simple and predictable, we have the user explicitly choose the degree of freedom that they want full control of, while relinquishing some control of the other to Parinfer.
Thus, Parinfer consists of two modes:
Some noteworthy unintended use-cases which we will explore later:
The foundation of Parinfer relies on a few somewhat formalized definitions and properties:
We wish to define properties that each Parinfer mode should exhibit.
With the following definitions:
The following should be true:
The actual operations performed by the modes rely on a formal definition of what it means for Lisp code to be "correctly formatted". We establish an invariant— something that must be true for every line of code. From that, Parinfer corrects indentation or parens simply by choosing correct values for $i_n$ or $p_n$ (defined later) to satisfy this invariant:
The following is a concise reference (not a guide) to establishing this invariant that Parinfer's modes rely on.
We wish to define necessary conditions for determining if a given file of Lisp code is "correctly formatted".
We start with a clarification that we only consider non-empty lines (i.e. lines that have at least one non-whitespace, non-comment token). Thus, all following references to line number $n$ will refer to the $n$th non-empty line.
To proceed, we define the following:
Next, we define a function which determines if a line's indentation is valid:
where:
Finally, we define that a file of Lisp code is "correctly formatted" if:
These rules are necessary and sufficient for determining when indentation is what we consider "unambiguous", but they are not sufficient in determining if code is "pretty". For example:
Formal descriptions of the actual operations performed by the modes are pending, but informal ones follow in their respective sections below.
Indent Mode gives you full control of indentation, while Parinfer corrects or inserts close-parens where appropriate. Specifically, it only touches the groups of close-parens at the end of each line. As a visual cue, we slightly dim these parens to signify their inferred nature.
You can select multiple lines and adjust their indentation the standard way using the controls below. If you are familiar with Paredit, these operations are roughly equivalent to those listed.
| Controls | Description | Paredit equivalent |
|---|---|---|
| Tab | indent line(s) | slurp line(s) down |
| Shift + Tab | dedent line(s) | barf line(s) down |
We perform the following steps to rearrange close-parens based on indentation.
We will refer to these later as rules #1, #2, #3 and #4.
This is the gist of what's happening. There are more steps performed, but we will just explore their effects in the next section.
You should be aware that the steps in the previous section have a side effect on what you type. Interestingly, these effects translate into four of the main Paredit operations.
| Cause | Effect | Description |
|---|---|---|
| Insert ( | Wrap |
inserts a matching ) as far as it can
i.e. "wraps" all possible elements to the right of your cursor
|
| Insert ) | Barf |
removes the original ) when inserted inside a matching pair
i.e. the current list "barfs" out all elements to the right of your cursor
|
| Delete ( | Splice |
removes the matching )
i.e. "splices" the current list into its parent (or simply "unwraps" it)
|
| Delete ) | Slurp |
inserts another ) as far as it can
i.e. the current list "slurps" all elements to the right of your cursor
|
We illustrate these operations in the following examples.
As a courtesy, Indent Mode will not move your parens until you are done typing in front of them. Just move your cursor away when you're done. A helpful analogy might be to think of your cursor as a paperweight that keeps your parens from blowing away.
Parinfer cannot infer anything about quote positions like it can with parens. So it doesn't try to do anything special with them, other than abandon processing if imbalanced quotes are detected.
Paren Mode gives you full control of parens, while Parinfer corrects indentation. You can still adjust indentation, but you won't be able indent/dedent past certain boundaries set by parens on previous lines. As a courtesy, this mode also maintains relative indentation of child elements when their parent expressions shift.
Here are some things that cannot be done in Indent Mode:
Paren Mode performs the following steps:
If there are paren imbalances in Paren Mode, the code is not processed, and you are prevented from switching to Indent Mode. This safely quarantines the imbalances that could be misinterpreted in Indent Mode. Thus, Paren Mode gives you an environment to fix them, after which the code is automatically formatted and ready for Indent Mode should you choose to switch.
We must take parens literally when opening an existing file. Incidentally, Paren Mode is perfect for this job, so we preprocess existing files with it before allowing the switch to Indent Mode.
Notice that this process is NOT an invasive pretty-printer. Newline characters are never added or removed. It preserves as much as it can of the original code, only moving close-parens and changing indentation.
Parinfer should remain in Paren Mode if the file cannot be processed, due to the reasons stated in the previous section.
As a courtesy, Paren Mode will not move your parens while your cursor is behind them. Just move your cursor away when you're done. A helpful analogy might be to think of your cursor as a paperweight that keeps your parens from blowing away.
Inferring parentheses based on indentation seems to lead to simpler editing mechanics for Lisp code. It leads to a system that keeps our code formatted well. And it allows us to use paredit-like features without hotkeys.
I think the biggest win is its potential to quell fear of managing end-of-line parens by enforcing a direct driving relationship with indentation.
And just like Paredit it maintains paren-balanced code.
The rules for what happens when inserting/deleting parens must be learned. Also, the case necessitating a "Paren Mode" comes at the cost of forcing the user to understand when and how to switch editing modes.
Also, the preprocessor step performed when opening files will cause more formatting-related changes in your commit history when collaborating with others not using Parinfer.
Regardless of how we choose to edit our Lisp code, there seems to always be a balancing act between maintaining the simplicity of how we interact with the editor and accepting some editor complexity to gain automation over these powerful but numerous parens.
Building the interactive examples for this page has allowed me to explore how well Parinfer can play this balancing act, but only in a demo environment. The real test will come once it becomes available to major editors. See editor plugins for progress.
The text formatting code is intended to be editor-agnostic. It is implemented in straightforward, imperative JavaScript, optimized for speed and designed to be easy to port to other languages. There is a test suite which is also designed to be easy to port with all test cases represented in JSON files.
The editor demos on this site are created in CodeMirror with hooks to apply our formatters and update cursor position. Source code for both the library and site are available on github:
Parinfer is still in early development. Several people have started integrating it into code editors at various stages of development.
Works in Progress:
Parinfer is available for some REPL environments as well:
$, $ [], $
{} and ,.
