Up Learn

Up Learn is a learning effectiveness platform I’ve been partnering with since March 2022. Together we shipped some pretty exciting projects! This is a highlight reel from my time working with Up Learn.

Quiz Migration

Imagine receiving the following prompt:

Here’s a feature actively used by ~60.000 users. I want you to migrate it to Elixir and React without downtime. It’s the centerpiece of learning features and the platform basically doesn’t exist without it. There are circa ~175000 quizzes in the database and they’re encoded using a homebrew DSL for which half the documentation is the actual code. Speaking of which - the current code is a single massive pile of legacy PHP and we have to keep everything backwards-compatible at all times.

How would you approach this?

This project was likely our boldest achievement and we learned a lot from it. Many aspects of quizzes made them a complex feature to implement. There are 10 different question mechanisms (multiple choice, fill-in-the-blanks, math formulas, etc..), 4 types of host feature (Progress Quizzes, Pre-video Quizzes, In-video Quizzes, Subsection Quizzes) and they span every learning topic in the platform.

Running this on a legacy codebase was code-wise the single biggest block to a more effective product development cycle. We immediately followed-up the release with great UX improvements infeasible in the previous platform: a “tour” element to display and navigate feedback when a student gets a response wrong, improvements towards marking accuracy, juicy feedback via animations and pretty colors, many bugfixes such as misplaced explanations, etc.

It was also an opportunity to explore many interesting problems:

How to even develop and ship something this big?

Break this work down into small (say ~2w) releases. Have both new and old versions running simultaneously as you build the new one. Set up a switching mechanism: we chose pairing feature flags with a “fallback” logic to redirect the user to the old version accessing an unmigrated quiz. Focus on getting to UI first to get user feedback ASAP. Track usage of the new version via a reliable monitoring tool (eg. Cloudflare). Once fully deprecated, remove switches and finish any cleanup work.

Where to begin?

I proposed and ran some data analysis to find out the most populous quiz groups broken down by both mechanisms and host feature and hit them first. That led to a migration roadmap starting off with multiple-choice-only Progress Quizzes. After around 4 releases we had roughly half the entire content live.

How to make sure everything still works?

Very tricky question. We dumped a large set of encoded quizzes into a file and set up a script (akin to a mix task) to mark them under both systems. We’d study the failures to determine the causes, chose a few case and wrote ExUnit cases for them, ensuring they won’t break again. We set up a margin of around 10% inaccuracy - it may sound quite high until we consider the inaccuracy of the previous system. Part of the discrepancy is actually a gain in overall marking quality.

How to mark an arbitrary math formula?

We did some research as part of a hackathon project and determined that our only viable solution was using a javascript library called CortexJS. It’s remarkably accurate and also has a very straightforward API. Integrating it with Elixir was another interesting challenge: we decided to compile the binary via bun and invoke it using MuonTrap.

How to negotiate scope around a large migration?

There was still the issue of catering to stakeholder expectations on the product, meaning we didn’t have the option of a full-stop to focus on migrating. We agreed to ship small improvements as we went, prioritising purely UI changes and guaranteeing they wouldn’t affect business logic. Anything touching business logic (eg. the DSL) is a no-go, as it could compromise backwards compatibility.

Summaries

Up Learn courses are organized in subsections - thematically cohesive sets of learning content, such as video lessons, article lessons and quizzes. This is a great, tailored experience for first-time learning, but not satisfying for revising content - a very common use case for Up Learn students.

Subsection Summaries were born after multiple rounds of user research, which validated students’ wish for a quicker way to revise an entire subsection - as opposed to revisiting each lesson.

Summaries are special lessons with a video and an article exposing main concepts of an entire subsection, and also a table of links to each relevant piece of content. As a dev I built the page and added videos to it. As a lead, the team built the article and link table features.

Section and Subsection Overview Pages

The first major step towards migrating learning content since I joined the team happened with the Section and Subsection pages. The experience for navigating content was subpar - very akin to the prior PHP stack.

This redesign made content easier to access and progress easier to track, as well as improving use of screen space, layout responsiveness for mobile (massive improvement) and centralized view of non-content-related actions: begin a review flow, check the summary for that subsection, etc. It’s had great reception by students and continues to be iterated moving forwards.

Hackathons

I wrote an entire post on Up Learn’s blog about this if you wanna learn more

Every two months, Up Learn sets up a hackathon. That’s two days where the entire engineering team can organize, pick and build ad-hoc projects of their own volition. Often these lead to actual improvements on the platform. Getting your hands dirty with different technologies also provides a different angle on engineering decisions - hackathons therefore tend to bring insights leading to improvements in engineering culture.

LiveView Admin

Out of curiosity and excitement for the Phoenix LiveView, we decided to rebuild our custom back-office tool using LiveView. What you see above is a glimpse of our editor for Quizzes (oh yes). This editor was built using LiveView and Monaco Editor. We also had to add CK Editor for editing Rich Text content later.

The stack proved massively more productive, even if in the end we ditched that in favor of a different approach in the end.

Up Learn Anki

Probably our most ambitious hackathons - I organized a project and led 12 people to collaborate in building a flashcard app based on AI-generated content, using Up Learn’s content as a source of knowledge. We split in 4 teams and at three days had a working (react-)native app serving multiple choice quizzes (including marking data) from a GraphQL Absinthe / Phoenix / OpenAI / Postgres backend. This one was super fun and super rewarding!

Ash Backend

Out of curiosity and excitement for the Ash Framework, we built a small api using Ash to deliver summary content. Ash is extremely productive, especially at when bootstrapping a project. It took 3 hours to get from 0 ash knowledge to having a GraphQL Absinthe API delivering data from a postgres table. At the end of the first day, we had most content migrated and available via the API.

On the second day we basically built a page to display the content and improved code quality in general.

For more projects, read the linked blog post.