These Months in explaine.rs

2020-12-14

TL;DR: we now explain more.

explaine.rs is a playground to explore and learn the syntax of the Rust programming language. Our initial post ended with:

[...] I suspect there's probably a better way of doing this. rustc or rust-analyzer lower the AST to some intermediate representation that might be more amenable to be inspected. However [...] that sounds like a lot of work [...]

Well, that work is already done 😅. I had a pretty long post prepared giving details of how I did it, but it felt like too much. The short version is:

• We wrap syn nodes and do some unsafe shenanigans (!) to build a "syntax nodes pool".
• We pre-compute an ordered [(id, start, end)] list that allows for efficient "hitbox" search.
• We build and visit the ancestors chain as before.

Another unsatisfactory aspect of the initial implementation was the "exploration phase", where we pre-compute all possible hitboxes for efficient highlight-on-hover. I went over several versions and optimizations of the algorithm. In the end, the solution is simple and blunt: instead of a single Wasm worker we spawn two, and dedicate one to the "exploration loop". This consumes more memory but allows the computation to finish very quickly by not interrupting the loop trying to attend other user requests.^a

But the really important work is the amount of hints that we currently deliver.

The actual explaining

At the end of the day, what I'd like for explaine.rs is to be useful. Rust syntax is a bit heavyweight and has its own share of fun corner cases and less-known bits. There is plenty we can do, however, within the limits of a purely syntax-based analysis. Here is a short tour of examples that I fancy:

Generics

Deciphering generics was high on my todo list, only recently we've started showing hints about them.

fn foo<T>(x: T) {}

[link] This produces:

This T refers to the type parameter in the declaration of the foo function.

We also report lifetime and const parameters, though there are still plenty of cases to cover.

Comments

Comments are a funny one, because they're (mostly) ignored by syn, so we have to tweak our wrapping to fake a "syn comment node".

fn main() {
  /* Hello */
}

[link] This outputs A block comment. (Still a WIP when you use weird doc comments like /** foo */).

Empty Type

A nice corner case when defining an enum.

enum Void {}

[link] This outputs:

[...] This is an empty enum without variants, which constitutes an empty type which cannot be instantiated.

Legacy Trait Objects

Sometimes, we can recognize a trait object being used without dyn.

fn foo(x: Foo + Send) {}

[link] This outputs:

[...] A trait object is usually marked with the dyn keyword.

Catch-all wildcard

Using the wildcard pattern _ as final catch-all is a usual pattern.

match x {
  1 => {},
  _ => {}
}

[link] This outputs:

The wildcard pattern _ [...] In this case it acts like a final "catchall" in this match expression.

And more...

There are many more minor contextual details that I've enjoyed adding (some of them were already mentioned in my previous post):

• Writing the receiver of a method explicitly: mut self: &Self [link]
• Complicated numeric literals like 0xA_B_C [link]
• Trait method vs. associated function, with/without default implementation [link]
• Fn-like generic arguments [link]
• Trailing commas in tuples [link]
• Unit struct vs. "normal" structs [link]
• mut bindings of several classes [link]
• break/continue with labels and values [link]
• A neat detail: we can fix your code if you write a bunch of statements (see the "Wrap" button) [link]

The Future

$ grep -R TODO --include "*.rs" | wc -l
29

Fun times.

_rvidal   🐦 · gh

^a I don't feel like getting into multi-threaded Wasm modules just yet, but that would be the cool/right solution: sharing memory, i.e. the analysis session, between the two workers.