Nominate is the product of some creative thinking with a former classmate Raga (Srinivasan Raghavendra). We were inspired by the virality of the ALS ice bucket challenge and wanted to create a native application for sharing video challenges among friends. We knew that the sharing aspect of the application was most important so we didn't compromise when it came to engineering a solution for frictionless app invites as well as sign-up.

You can check out the nominate website here: http://thenominateapp.appspot.com/web/index.html

** Source code for this project is private but I can supply read access to any interested parties per request.

Background

Nominate was truly an example of fail early and often. The original backend was started by Raga (written in Java for AppEngine), to which I applied my own OCD code refactoring to take hold of the backend while he focused on the application and video processing instance (Google Compute Engine). As I learned AppEngine, I discovered that Python was clearly the better runtime. With Guido van Rossum at Google (at the time), it was evident that Python was going to be well supported. Not to mention that the verbose nature of Java made it tedious to do trivial tasks with AppEngine. With about 40% of the backend completed and much to Raga's chagrin, I migrated to a Python AppEngine backend. I am so glad I did.

Technologies

Me(Nate)

• Python
• AppEngine - WebApp2 Framework
• NDB

Raga

• Java/Android

Goals/Outcomes

RESTful Design

This was my first project working with a RESTful backend. It was a bit of an adjustment to move away from RPC-like endpoints to CRUD actions performed on resources. Ultimately, I loved the abstraction and the project benefited from the organization.

TDD

I was also relatively new to the test driven development. I had always responded to questions about it as "sure sure I know TDD" but I had never enforced the discipline of actually writing tests first. I'll admit that most of Nominate was written with development code written first or intermittently with the tests, but for the few endpoints that I respected strict TDD, it was refreshing.

Minimum Viable Product

MVP is a buzzword, but it's a good one. Ideas that take too long to become perfect never get done because perfect doesn't exist. This conflicts with some of my perfectionist tendencies, so a concrete goal of this project was to push forward from refactoring towards feature development. The most effective way of doing this was using appropriate task management, Trello, and marking some features as "nice-to-have". By not evicting these tasks from the realm of possibility, it allowed me to prioritize concrete features and then put in some extra hours on the "nice-to-have" refactoring to make the code as clean and neat as possible.

Key Challenges

NoSQL

Most all database experience I had was with relational storage. Understanding AppEngine's BigTable methodology was a bit tricky as I found that I had to de-normalize problems which seemed counter-intuitive. I spent a lot of time considering the tradeoffs of various indexes and modeling the data most efficiently.

One of the most challenging models was storing the Nomination entries themselves. Since nominations could be forwarded continually, a single thread of nominations formed a 3-ary tree. Flattening the tree in a way that allowed for retrieval by the nominee as well as nominator was a task that seemed to have competing optimizations. Ultimately, we struck a balance and found a solution that adequately prevented contention.

Nomination Process - Facebook

Unfortunately, the Facebook API was changing to 2.0. This meant that you could no longer get a stable Facebook User ID from a user's friend unless they were a user of your app. This meant that if someone nominated a friend that was not a user of our app yet, we had no way of uniquely identifying that friend so they could retrieve the nomination once they signed up. This was a high priority since we absolutely had to have the best sharing/inviting experience possible to support the viral nature of the challenges. Our solution used a multistep approach for retrieving past Nominations for new users.

1. String match on profile picture url (this url was not perfectly stable but it seemed to last for a few days or so).
2. String match on user's first & last name (obviously this could not be our first criteria since many users may have the same first and last name).
3. Facial recognition matching of profile picture (We didn't implement this step as we never found the need to, but we had it ready if necessary).

Friends Endpoint

This challenge was by far the most difficult in designing the Nominate backend. The task was straightforward -- fetch n number of nominations for a given user where the user is not involved in the nomination but where the nominations are related to the user by Facebook friendship. A second requirement was that the friends were organized by priority of friendship within the app (i.e. most interaction in likes/comments/nominations). The icing on the cake was that the endpoint needed to support pagination, so after retrieving the first, say 10 entries, a marker need to be made to pick up at 11 if the endpoint supplied a cursor key. While many other endpoints had cursors, the complexity of the /friends endpoint made this cursor even more interesting.

Ultimately, the best solution was probably to execute some map-reduce task to curate lists of friend nominations at some set frequency. Given that we didn't have the resources to run a job as frequently as we wanted, we had to get creative with on-the-fly processing.

Result: We ended up pre-computing a model that represented friend interactions. Each time a user liked/commented/nominated a friend, their interaction score increased. The idea here was to provide a quick query to fetch n number of "best friends" for a given user from which to select nominations. The /friends endpoint then used this list to start fetching nominations for each of the friend. The troubling part was that each nomination had to be checked against the querying user id for involvement (remember that the user can't be involved in the /friend nomination displayed). I had to adapt a bitstring class and write a custom bloom filter model for AppEngine since it didn't (doesn't?) support native c code libraries and apparently every bloom filter implementation used native code to some degree. Ultimately the bloom filter worked very efficiently and the friends endpoint worked moderately well. The worst part of the resulting implementation was that we were retrieving a list of "best friends nominations" and then weeding out all nominations where the user was involved. Intuitively, a user would be most involved with those nominations, essentially giving us a higher probability of having to weed out the entries we fetched. But sometimes you have to just get something to work, and it did.