The Idea Vending Machine
Learnings from One Year of Building an Open Source Elixir Application
If you're ever in Japan, one of the many things that might stand out are the multitude of glowing candy colored mechanical cubes lining the streets. These machines are often filled with an array of surprisingly tasty snacks and beverages. It's hard to resist trading the extra yen in your pocket for one of these delectable treats.
I couldn't help but relate the idea of working on a project to the concept of these vending machines. In a similar vein, you end up trading time and effort to bring something into reality. And beyond the seductive temptation of these potential project ideas calling to you from behind the glass window, it's difficult to prove its merit without first investing time and work.
I traded my free time last year to learn about and build an Elixir application. The result, Elyxel, is a basic community tool much like Hacker News et al. designed to be a space to gather and share stories1. My goals were simple—finish and learn new skills along the way. The following is a detailed account of the things I learned along the way.
Additionally, Elyxel will be open sourced in the hope that it might be beneficial to anyone trying to learn from what I did. This piece is geared towards someone who is familiar with the fundamentals of programming and general web development.
Avoiding Design Limbo
I wanted to start with a fairly unoriginal simple concept that was familiar to me. At the time, solving the well defined problem of community software was particularly appealing. I was getting fatigued from the constant negative rhetoric that seemed to dominate the conversation on forums I frequent. That coupled with a desire to learn new technology was all the kindling I needed to get started.
The actual design of the site took a minimal amount of time. I opted for a simple relatively sparse interface. On past projects I would spend a lot of time up front on the design phase of the project. It's easy (and honestly quite fun) to get lost in the design process.
Elixir & Phoenix
To gain better leverage, I decided to use this spare time as an opportunity to learn a new programming language and framework. The sheer amount of ways to build a modern web application was staggering. But Elixir and Phoenix came out on top after some research—both were fast, being actively developed and most importantly easy to get started with.
There's so many different ways to accomplish the same thing, and to be honest it was quite difficult cutting through the noise. I opted to try and remove as many elements as possible, with one of the major ones avoiding using a front-end javascript framework. Static pages were plenty fast for my use case. A vital part of the process is whittling down the problem to its core.
After settling on my choice, I knew I needed a server to host the finished application. Instead of using an automated service like Heroku, I set about learning how to provision and setup a small virtual private server (VPS). Now having gone through the process, I've gained a greater appreciation and understanding of the setup infrastructure. I highly recommend doing it at least once.
If you're interested in doing the same, I've listed the major steps of the process below2. The entire setup process took about a month of learning and understanding best practices. I had to cut some of these explorations short as each thread I pulled on unearthed a hundred more. There's a staggering amount of low level tech built by brilliant folks we rely on everyday.
- Installed Ubuntu
- Learned about Ubuntu's file system layout
- Setup password less login via SSH
- Installed NGINX
- Setup reverse proxy to point towards Phoenix application
- Enabled gzip support to improve performance.
- Installed htop for monitoring
- Pointed the elyxel.com domain to the server
- Setup SSL via Let's Encrypt
- Secure NGINX on Ubuntu
Diving In To The Deep End
There's quite a few ways to approach learning within a new domain. What works best for me is starting out by reading through some of the foundational principles. Once I have a rudimentary understanding, I try to quickly apply what I just learned to solve a related problem. If I don't take this next step I usually don't end up making enough meaningful connections in my brain to remember any of it.
Building the application consisted of two major learning steps. Learn enough of Elixir, which meant wrapping my head around functional programming and the syntax. Secondly, learn Phoenix, which was similar enough to Rails that it didn't require as much effort.
There was an upfront cost of having to learn Elixir before I could build anything. While the syntax was familiar there were a few new concepts to grapple with due to Elixir being a functional programming language.
Once I had the foundation, learning the the Phoenix framework was fairly smooth. The documentation on the Phoenix project web site served as the backbone of the steps I took to write up the application. While it was a good overview some of the challenges I faced required looking up supplementary material. Luckily, any gaps in knowledge were then filled with shorter guides and pieces written by intrepid early adopters. I've included links to some of the most useful ones below.
After gaining some familiarity with the chosen tool set, I knew I needed to breakdown the project into smaller milestones—teasing out critical features needed for version 1.0. This is the whittled down list I came up with:
- Written draft of the problem
- Sketch & wireframes
- Server provisioning
- Landing page
- Signup flow
- Invite Flow
-
Application functionality
- Top page
- Recent page
- Submit page
- Comments
- Profiles
- Voting mechanism
- Sentiment analysis
- Curated stories
- Content fire hose
Once I had an outline, it was a matter of building things piece by piece. The main challenge that kept coming up was the lack of robust best practices for features I was trying to build. In hindsight this was actually favorable because it ended up being a rewarding challenge to figure out things on my own when I did get stuck. I wouldn't recommend this process if you're under a deadline, but if the goal is learning then it definitely is valuable.
Another learning constraint was picking simpler libraries to utilize. The goal was to be able to understand the tools I was using and avoid cruft. We've all heard the horror stories of including some mega complex library to achieve a relatively simple goal.
Code Highlights
To avoid getting overly prescriptive, I'm only going to describe a few of the interesting challenges of the project3.
The login system was particularly tricky even though I used a simple library called openmaize4. It was great but through out the build I found myself getting increasingly paranoid about missing some big security feature and leaving myself exposed to an unforeseen vulnerability.
For less trickier parts I ported over some of the code from Lobster, another Rails based community site. Their code base seemed well built and accessible. One example was the particularly clever bit of code creating human readable timestamps below.
Most implementations of rating I found followed a similar pattern. I opted for the version below. It will be interesting to watch how this evolves as the community grows.
I resisted the temptation to use a library for pagination. Here is the simple solution I cobbled together.
Performance
Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something.
I achieved this by keeping within certain design constraints from the start. First was using a system font stack. In recent years, most operating systems come with a robust set of defaults. Not including custom typography saves the cost of sending it over the wire and reducing browser render time.
Furthermore, all assets were vector (SVG) and animations created with CSS. I didn't use a javascript framework because it didn't feel necessary for the task. Coupled with the speed of Phoenix and Elixir these small compromises improved speed in aggregate.
Fatigue and Completion
One thing that stuck with me throughout this project was how challenging it was to work a full work day and spend a few extra hours at night chipping away at Elyxel6. Particularly because I was doing really interesting rewarding work as well. There were so many times where I was too exhausted to come home and work on Elyxel even though I was super excited to. I always admire folks who are able to do both consistently well.
Dans ses écrits, un sage Italien Dit que le mieux est l'ennemi du bien.
To keep moving along I had to adapt the mantra, popularized by Voltaire, of not letting perfection being the enemy of good. There is endless amount of ways to iterate and improve something but I had to call whatever I was working on as done for the sake of progress even though I knew with more time I could improve it significantly. I've gotten better at it but it's still an uncomfortable decision most of the time.
The good news is I ended up finishing version 1.0 in January but have since stopped working on it. This is partially due to a massive project at work taking up head space and generally wanting to take a break.
What's next?
Truthfully, I'm not sure. Now that it's out there I'm not quite sure what to do with it. I learned a ton from building Elyxel out that has helped me grow various skills that directly apply to my career. Ideally with a little more work it becomes a small growing community. If you’ve read through all of this you’re the kind of person I made Elyxel for. If you’d like to join please don’t hesitate to send me an e-mail.
At the end of the day, I find the real delight comes from the moment you experience the vending machine whirring and clicking to life as it dispenses your reward with a satisfying thud.
Pointers
Here is a supplemental collection of links that I have read through and have helped throughout the process of building this app:
Dev Ops
- Apache vs Nginx Practical Considerations
- Server Side TLS
- Primer for Securing Ubuntu
- AWS in Plain English
- Securing NGINX SSL
- A tmux Primer
- Postgres Quick Start for People Who Know MYSQL
- Evaluate Performance with Chrome Dev Tools
Elixir
- Comparing Date Time in Elixir with Ecto
- Elixir Bridge Documentation
- Unix Timestamps in Elixir
- My First Week Learning Elixir
- Ask HN—Are you using Elixir or Phoenix in Production
- Pinterest Engineering—New Open Source Tools for the Elixir Community
- Book—Programming Elixir
- The Elixir of Concurrency
- Elixirstatus—Latest Elixir News
- Deploying Phoenix
- Official Erlang & Elixir Syntax: A Crash Course
- Video—From a Ruby on Rails Monolith to Elixir and Elm Microservices
- Postgres Indexing with Ecto
- How We Deploy Elixir Apps
- Stack Overflow—Check if a URL is Valid in Elixir
- Stack Overflow—Specific Fields When Fetching Data Using Ecto
Phoenix
- Deploying Phoenix with a Bash Script
- Phoenix Deployment
- Elixir and Phoenix the Future of Web APIs and Apps
- Complementary Frontend Tools for Single Page Applications
- Rate Limiting a Phoenix API
- Pagination with Elixir and Ecto
- Phoenix React Redux Example
- Video—Rails to Phoenix by Brian Cardarella
- Adding Field to an Ecto Model in Phoenix
- Book—Programming Phoenix
- Elm Phoenix Socket
- Phoenix React Love Story
- Comparing Rails and Phoenix
- Deploying Phoenix with Git
- Continuous Deployment with Phoenix
- Phoenix Tips and Tricks
- Best Of Rails in Phoenix
- Stack Overflow—Adding Current Users Information to a Post
Thanks to Rich, Jamie, and Jake for reading and providing invaluable advice.
If you liked this story, you might enjoy my article on prototyping an ambient notification cube. And if you think I've missed anything, please let me know.
-
Since this was a learning exercise, it took more time than was truly necessary. I am fairly certain a more competent programmer could have done the same in a few month or less.↩︎︎
-
Guides for building simple system infrastructure are pretty easy to find. In fact most of the basics can be found on in one place—Digital Ocean.↩︎︎
-
After all, it's hard to get more detailed than providing the full source code.↩︎︎
-
The risk of working on the cutting edge—It looks like future development has ceased on this library with the author working on a newer project.↩︎︎
-
These estimates are derived from an average testing done using Chrome 59 on a maxed out 2016 MacBook Pro with a 120 Mbps connection.↩︎︎
-
If you're really digging through my commit history you would notice there were periods of time where no progress was made. This was either when we were moving homes or work was really overwhelming.↩︎︎