The Idea Vending Machine Of The Mind
Digital Illustration

The Idea Vending Machine

Learnings from One Year of Building an Open Source Elixir Application

If you're ever in Japan, one of the many things that might stand out are the multitude of glowing candy colored mechanical cubes lining the streets. These machines are often filled with an array of surprisingly tasty snacks and beverages. It's hard to resist trading the extra yen in your pocket for one of these delectable treats.

I couldn't help but relate the idea of working on a project to the concept of these vending machines. In a similar vein, you end up trading time and effort to bring something into reality. And beyond the seductive temptation of these potential project ideas calling to you from behind the glass window, it's difficult to prove its merit without first investing time and work.

I traded my free time last year to learn about and build an Elixir application. The result, Elyxel, is a basic community tool much like Hacker News et al. designed to be a space to gather and share stories1. My goals were simple—finish and learn new skills along the way. The following is a detailed account of the things I learned along the way.

Bamboo Forest Vending Machine
Quiet Alley Vending Machine
Harajuku Vending Machine
Coffee
Tokyo Spring 2017

Additionally, Elyxel will be open sourced in the hope that it might be beneficial to anyone trying to learn from what I did. This piece is geared towards someone who is familiar with the fundamentals of programming and general web development.

Avoiding Design Limbo

I wanted to start with a fairly unoriginal simple concept that was familiar to me. At the time, solving the well defined problem of community software was particularly appealing. I was getting fatigued from the constant negative rhetoric that seemed to dominate the conversation on forums I frequent. That coupled with a desire to learn new technology was all the kindling I needed to get started.

The actual design of the site took a minimal amount of time. I opted for a simple relatively sparse interface. On past projects I would spend a lot of time up front on the design phase of the project. It's easy (and honestly quite fun) to get lost in the design process.

Elyxel App 1.0
Final 1.0 design live

Elixir & Phoenix

To gain better leverage, I decided to use this spare time as an opportunity to learn a new programming language and framework. The sheer amount of ways to build a modern web application was staggering. But Elixir and Phoenix came out on top after some research—both were fast, being actively developed and most importantly easy to get started with.

There's so many different ways to accomplish the same thing, and to be honest it was quite difficult cutting through the noise. I opted to try and remove as many elements as possible, with one of the major ones avoiding using a front-end javascript framework. Static pages were plenty fast for my use case. A vital part of the process is whittling down the problem to its core.

After settling on my choice, I knew I needed a server to host the finished application. Instead of using an automated service like Heroku, I set about learning how to provision and setup a small virtual private server (VPS). Now having gone through the process, I've gained a greater appreciation and understanding of the setup infrastructure. I highly recommend doing it at least once.

If you're interested in doing the same, I've listed the major steps of the process below2. The entire setup process took about a month of learning and understanding best practices. I had to cut some of these explorations short as each thread I pulled on unearthed a hundred more. There's a staggering amount of low level tech built by brilliant folks we rely on everyday.

  1. Installed Ubuntu
  2. Learned about Ubuntu's file system layout
  3. Setup password less login via SSH
  4. Installed NGINX
  5. Setup reverse proxy to point towards Phoenix application
  6. Enabled gzip support to improve performance.
  7. Installed htop for monitoring
  8. Pointed the elyxel.com domain to the server
  9. Setup SSL via Let's Encrypt
  10. Secure NGINX on Ubuntu

Diving In To The Deep End

There's quite a few ways to approach learning within a new domain. What works best for me is starting out by reading through some of the foundational principles. Once I have a rudimentary understanding, I try to quickly apply what I just learned to solve a related problem. If I don't take this next step I usually don't end up making enough meaningful connections in my brain to remember any of it.

Building the application consisted of two major learning steps. Learn enough of Elixir, which meant wrapping my head around functional programming and the syntax. Secondly, learn Phoenix, which was similar enough to Rails that it didn't require as much effort.

There was an upfront cost of having to learn Elixir before I could build anything. While the syntax was familiar there were a few new concepts to grapple with due to Elixir being a functional programming language.

Once I had the foundation, learning the the Phoenix framework was fairly smooth. The documentation on the Phoenix project web site served as the backbone of the steps I took to write up the application. While it was a good overview some of the challenges I faced required looking up supplementary material. Luckily, any gaps in knowledge were then filled with shorter guides and pieces written by intrepid early adopters. I've included links to some of the most useful ones below.

After gaining some familiarity with the chosen tool set, I knew I needed to breakdown the project into smaller milestones—teasing out critical features needed for version 1.0. This is the whittled down list I came up with:

  • Written draft of the problem
  • Sketch & wireframes
  • Server provisioning
  • Landing page
  • Signup flow
  • Invite Flow
  • Application functionality
    • Top page
    • Recent page
    • Submit page
    • Comments
    • Profiles
    • Voting mechanism
    • Sentiment analysis
    • Curated stories
    • Content fire hose

Once I had an outline, it was a matter of building things piece by piece. The main challenge that kept coming up was the lack of robust best practices for features I was trying to build. In hindsight this was actually favorable because it ended up being a rewarding challenge to figure out things on my own when I did get stuck. I wouldn't recommend this process if you're under a deadline, but if the goal is learning then it definitely is valuable.

Another learning constraint was picking simpler libraries to utilize. The goal was to be able to understand the tools I was using and avoid cruft. We've all heard the horror stories of including some mega complex library to achieve a relatively simple goal.

Code Highlights

To avoid getting overly prescriptive, I'm only going to describe a few of the interesting challenges of the project3.

The login system was particularly tricky even though I used a simple library called openmaize4. It was great but through out the build I found myself getting increasingly paranoid about missing some big security feature and leaving myself exposed to an unforeseen vulnerability.

For less trickier parts I ported over some of the code from Lobster, another Rails based community site. Their code base seemed well built and accessible. One example was the particularly clever bit of code creating human readable timestamps below.

      defmodule Elyxel.Time do
        @moduledoc """
        Time Helper
        """

        epoch = {{1970, 1, 1}, {0, 0, 0}}
        @epoch :calendar.datetime_to_gregorian_seconds(epoch)

        def elapsed(time) do
          now = :os.system_time(:seconds)
          past_time =
            time
            |> Ecto.DateTime.to_erl
            |> :calendar.datetime_to_gregorian_seconds
            |> -(@epoch)

          now - past_time
        end

        def simple_time(time) do

          seconds = elapsed(time)

          cond do
            seconds <= 60 ->
              "#{seconds}s"
            seconds < (60 * 60) ->
              "#{ round(seconds / 60.0) }m"
            seconds < (60 * 60 * 48) ->
              "#{ round(seconds / 60.0 / 60.0) }h"
            seconds < (60 * 60 * 24 * 30) ->
              "#{ round(seconds / 60.0 / 60.0 / 24.0) }d"
            seconds < (60 * 60 * 24 * 365) ->
              "#{ round(seconds / 60.0 / 60.0 / 24.0 / 30.0) }mo"
            true ->
              "#{ round(seconds / 60.0 / 60.0 / 24.0 / 365.0) }y"
          end
        end

      end
      

Most implementations of rating I found followed a similar pattern. I opted for the version below. It will be interesting to watch how this evolves as the community grows.

      defmodule Elyxel.Rating do
        @moduledoc """
        Rating Helper
        """
        import Elyxel.Time

        def calculate_rating(pluses, comments, time) do
          comment_weight = 0.2  # Comments carry a little weight
          gravity = 1.5         # Rating decreases much faster for older items if gravity is increased
          amplifier = 10000     # Surfaces buried significant digits

          round(((pluses + (comments * comment_weight) - 1) / (:math.pow(((elapsed(time)/60/60) + 2), gravity))) * amplifier)
        end
      end
      

I resisted the temptation to use a library for pagination. Here is the simple solution I cobbled together.

      defmodule Elyxel.Pagination do
        @moduledoc """
        Generic ecto pagination helper
        """
        import Ecto.Query
        alias Elyxel.Repo

        def page(query, page: page, per_page: per_page) do
          scrub_page = page |> scrub
          count = per_page + 1
          result = query
                    |> limit(^count)
                    |> offset(^(scrub_page*per_page))
                    |> Repo.all
          %{ has_next?: (length(result) == count),
             has_prev?: scrub_page > 0,
             current_page: scrub_page,
             list: Enum.slice(result, 0, count-1) }
        end

        defp scrub(page) do
          cond do
            is_integer(page) && (page >= 0) ->
              page
            is_binary(page) ->
              case Integer.parse(page) do
                {page, _} ->
                  if (page < 0) do
                    0
                  else
                    page
                  end
                :error -> 0
              end
            true ->
              0
          end
        end
      end

      

Performance

Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something.

I achieved this by keeping within certain design constraints from the start. First was using a system font stack. In recent years, most operating systems come with a robust set of defaults. Not including custom typography saves the cost of sending it over the wire and reducing browser render time.

      /* This typographic stack should work well across most platforms */

      --system-fonts: -apple-system, BlinkMacSystemFont, "Segoe UI",
                      Roboto, Oxygen, Ubuntu, Cantarell, "Open Sans",
                      "Helvetica Neue", sans-serif;
      

Furthermore, all assets were vector (SVG) and animations created with CSS. I didn't use a javascript framework because it didn't feel necessary for the task. Coupled with the speed of Phoenix and Elixir these small compromises improved speed in aggregate.

Fatigue and Completion

One thing that stuck with me throughout this project was how challenging it was to work a full work day and spend a few extra hours at night chipping away at Elyxel6. Particularly because I was doing really interesting rewarding work as well. There were so many times where I was too exhausted to come home and work on Elyxel even though I was super excited to. I always admire folks who are able to do both consistently well.

Dans ses écrits, un sage Italien Dit que le mieux est l'ennemi du bien.

To keep moving along I had to adapt the mantra, popularized by Voltaire, of not letting perfection being the enemy of good. There is endless amount of ways to iterate and improve something but I had to call whatever I was working on as done for the sake of progress even though I knew with more time I could improve it significantly. I've gotten better at it but it's still an uncomfortable decision most of the time.

The good news is I ended up finishing version 1.0 in January but have since stopped working on it. This is partially due to a massive project at work taking up head space and generally wanting to take a break.

What's next?

Truthfully, I'm not sure. Now that it's out there I'm not quite sure what to do with it. I learned a ton from building Elyxel out that has helped me grow various skills that directly apply to my career. Ideally with a little more work it becomes a small growing community. If you’ve read through all of this you’re the kind of person I made Elyxel for. If you’d like to join please don’t hesitate to send me an e-mail.

At the end of the day, I find the real delight comes from the moment you experience the vending machine whirring and clicking to life as it dispenses your reward with a satisfying thud.

Pointers

Here is a supplemental collection of links that I have read through and have helped throughout the process of building this app:

Dev Ops

Elixir

Phoenix

Thanks to Rich, Jamie, and Jake for reading and providing invaluable advice.

If you liked this story, you might enjoy my article on prototyping an ambient notification cube. And if you think I've missed anything, please let me know.


  1. Since this was a learning exercise, it took more time than was truly necessary. I am fairly certain a more competent programmer could have done the same in a few month or less.↩︎︎

  2. Guides for building simple system infrastructure are pretty easy to find. In fact most of the basics can be found on in one place—Digital Ocean.↩︎︎

  3. After all, it's hard to get more detailed than providing the full source code.↩︎︎

  4. The risk of working on the cutting edge—It looks like future development has ceased on this library with the author working on a newer project.↩︎︎

  5. These estimates are derived from an average testing done using Chrome 59 on a maxed out 2016 MacBook Pro with a 120 Mbps connection.↩︎︎

  6. If you're really digging through my commit history you would notice there were periods of time where no progress was made. This was either when we were moving homes or work was really overwhelming.↩︎︎