Learning Rust by Project

Recently, I decided I wanted to learn Rust. I’d checked it out briefly at some earlier point a few years ago, trying to wrap my head around the borrow checker by going through the Rust tutorial. I didn’t continue however, as I didn’t have a project to apply it to at the time.

Luckily, this time was different.

The Marked-Space Project #

I already had an idea to write an improved version of an existing markdown to Confluence tool Mark. We used it at one of my previous clients to render our documentation in a documentation-as-code approach, but it has a number of drawbacks:

• Moving pages was hard: when moving a page to a different parent, you need to first delete the old page and then create the new one totally from scratch. It’d be nice if Confluence’s move functionality was used instead: redirects, comments and reactions are preserved, etc.
• On disk structure didn’t reflect the in space structure: instead parenting was done with internal comment directives. We had to manually ensure all of these we up to date as we reorganised the documentation. It also meant cross linking was a pain; you had to reference titles, and these aren’t easily navigated in your editor (eg, by ctrl-clicking in VsCode). They also couldn’t be checked automatically, frequently resulting in broken links if the page was renamed.
• Images had to be explicitly mentioned as attachments: instead of being automatically added/removed as necessary. It would be much more convenient to just specify the link like as normal ![Image Label](image.png) and have the tool automatically add the image as an attachment so it can be displayed properly by confluence.
• Line breaks are explicitly parsed: meaning you couldn’t use hard breaks in your markdown paragraphs without it being replaced by <br/>. With this approach nice looking long paragraphs in Confluence requires long lines in markdown - a problem if you’re editing on a wide screen, let me tell you.

There are a number Good Things that would be nice to keep:

• Support for Confluence native structures and macros
  • links
  • images
  • attachments
  • tags
  • table of contents
  • code blocks
• The first heading is used as the page page title

As I write this, I did manage to solve all of the drawbacks and it’s just a matter of implementing some more Confluence specific features - and maybe some templating, who knows?

What Made this A Good Learning Project #

Primarily what made this a good project to learn Rust with was its relatively well defined scope. I could basically write down all the features/use cases that I needed to implement and work through it methodically without necessarily having to dive into big upfront design.

Functionally, I got to see a lot of the best parts (IMHO) of Rust due to the fact that the project was a CLI without the need for asynchronous behaviour, while still involving calling a REST interface with regular responses that could be easily serialized (mostly).

The project itself was completely reliant on the foundation provided by the comrak application/library. Comrak is a complete CommonMark parser with an existing html generator. Not only was it easy to work with, it was also a great way to learn Rust by example.

Additionally, I got to use and experience these great libraries:

• serde - the go-to serialization and deserialization library for Rust. I literally laughed with joy at how easy this made deserializing responses from the Confluence API.
• reqwest - Rust’s answer to Python’s requests module; super easy HTTP library. I used the blocking mode to keep things simple.
• clap - a nice CLI interface generator. I didn’t fully explore this one yet, but I like its declarative syntax.
• anyhow and thiserror - the way to do error handling in Rust without tons of boilerplate. I admit, I didn’t really handle errors until later in the project’s lifetime: I was basically assuming things worked with .unwrap which asserts if there were errors. However, I was glad these libraries were there when I did finally get around to doing things properly.
• assert_fs was a convenient library to run the filesystem tests with and ensure everything was cleaned up properly afterwards.

The Lessons #

I am someone who grew up on C/C++, but with large experience in both Python and C#. This probably made it easier for me to pick up Rust in some ways than say, someone with only a background in Javascript. But it also made certain things a bit more jarring as I had to unlearn certain patterns.

For example, the references of C++ are a lot like Rust’s borrows (and are even denoted by an ampersand &): both allow pass by reference and won’t copy the original object/struct/data. However, Rust requires them to be explicit:

struct MyStruct { x: i32 };
fn foo(my_struct: &MyStruct) {}
// assume already constructed MyStruct
foo(&my_struct); // need to explictly show the borrow with &

Whereas C++ passing by reference or passing by value is transparent at the call site:

struct MyStruct { int x };
void foo(MyStruct& my_struct) {}
// assume already constructed MyStruct
foo(my_struct); // can't tell at call site whether this is pass by value or
    // reference without consulting the definition.

In fact, from the point of view of C++, making a call like foo(&my_struct) felt wrong because this would pass a pointer to my_struct - and in C++ you should be preferring references to pointers where-ever possible.

All this talk of references and borrows and pointers leads me to the first of the 3 major lessons I learnt. Unsurprisingly, all the lessons follow themes that Rust is already known for, but it was good to learn them in my own context.

Lesson 1: The Borrow Checker #

I think this is the first lesson everyone learns once they start writing Rust. Not least because many common patterns you might use in other languages (using pointers) will just not work in Rust - at least, not without significant modification.

I heard the borrow checker described (most likely in one of the “Decrusting Rust” videos by Jon Gjengset - a great resource, btw) like so:

The goal of the Borrow Checker was not to allow all safe Rust code, but to deny any potentially unsafe code.

Frequently, although you know by your human logic and reasoning that the code is actually safe, you may struggle to jump through the right hoops to get Rust to see it the same way.

You struggle and struggle and may even dabble in “Rust lifetimes” - a language feature that allows you to annotate your structures with their intended life times. This is still an area of the language I am not super confident in deploying and eventually you just start writing your code to avoid such pitfalls. After the initial adjustment to this style, I often experienced a feeling of Zen when I just “got it”.

It actually felt kind of refreshing to have to worry about ownership and lifetimes again after being in GC’d languages for so long. In my old C++ days, I was all for clear ownership between objects, preferring const Type& (non-mutable borrows) for most things, std::auto_ptr to be clear about transfer of ownership (non copying transfer of ownership: pre move semantics days) and so on.

This leads us to the next thing I learnt: the confidence of type systems.

Lesson 2: Confidence and Type Systems #

One thing I relearnt as I wrote this tool - something that ordinarily I would have written in Python - is that there’s something confidence building when the compiler is able to check your program for type correctness before you even write your first test.

Now, you can get the same confidence if you’re diligent with your type annotations in Python (or enforce them with something like mypy), but when I’m trying to move quick - as I was in this case, trying to determine if ideas like checking internal links and translating them to Confluence links would work - I tend to forget them and rely on tests to ensure I’m not breaking things.

When I was forced to be type-clear by the nature of the language, it didn’t really add that much overhead, but I also found myself needing to test much less as the typing of the system made a lot of tests I would have written unnecessary - instead of testing the structure of my code, I just tested the behaviour, if that makes sense.

There were a number of instances of type wrangling however; where you had to convert, borrow, beg and steal into the right type (only two of those being supported by the language directly). This is particularly true with string slices (str), strings, paths and utf8 byte arrays. Such is the tradeoff you make with type systems and encourages you to be consistent with types just to avoid long conversion chains.

It was a very interesting feeling after having been in type agnostic land for such a long time. Even more so as I came back in a time when debates about major frameworks giving away TypeScript are happening… but those are thoughts for another time.

Lesson 3: Errors over Exceptions #

As I alluded to earlier, this is a part of Rust I came to relatively late; probably because I didn’t follow a tutorial.

Rust’s error handling is built around the Result<T, E> enum, where T is the “Success/Ok” type and E is the error type. On its own it can be a little irritating to deal with and I was frequently writing constructs akin to the following:

let result = match some_op_that_returns_result() {
 Ok(result) => result,
 Err(err) => return Err(err) // return early
}

This doesn’t really achieve that much for all those keystrokes, and thankfully the Rust maintainers realised it.

It took me a while to realise it myself, but you can replace this common construction with a single question mark (?) operator. The ? operator basically checks if the result is an error, and if it is returns from the current function early, or otherwise unwraps the ok result.

Ergonomically this is great, but I find myself wondering if it doesn’t somewhat defeat the generally touted advantage of “results are explicit and exceptions are not”. Consider the following actual code:

let existing_page: responses::PageSingle = confluence_client
            .get_page_by_id(homepage_id)?
            .error_for_status()?
            .json()?;

1. It can be quite hard to spot exactly where the returns (or throw points 😉) are. Admittedly, formatting can help.
2. There are a number of different errors that can be returned here; they either need to be translated into the same error type as the calling function through implementing From<> traits for each error type, or by using a boxed std::error::Error.

For the second point I ended up using a combination of anyhow and thiserror, but it’s still a tension between ease of use and a cohesive error object. It does work, and it works in a much more explicit way than C++ exceptions, for example.

Bonus One Line Lessons #

A couple of brief points:

• The rich Enums in Rust are like how C/C++ Unions were meant to be. Really interesting tool for compile type modelling of variant types.
• There’s no partial specialisation like in C++, which was a bit jarring when approaching writing generic code.

Conclusion #

I took a lot of energy from this project and it really reminded me of the joys of programming. Although I have had ample coding time in the past years, it hasn’t been my primary focus - that was usually more along architectural or managerial lines. The focus of a single project, of being able to really get deep and alter my method of solutions to suit a new language has been invigorating.

I’m looking forward to being able to use Rust for more projects in the future.

Note: I’ll probably open source the project some time in the near future. Stay tuned for an announcement via the same channels!