Background

I was co-creator for Handshake as a final project for the graduate course Advanced Programming Tools taught by Adnan Aziz. We looked at how communication channels existed currently and were disappointed by how static they had to be. Giving out your cell phone or email was nearly a permanent channel in which you had to trust the recipient.

The general idea behind Handshake was to present users with temporally relevant communication channels. If a channel was no longer required, it could expire or be explicitly closed. The key was to not disrupt how users naturally engaged in communication (i.e. text, email) since we were not interested in becoming just another chat application. Uses for Handshake might include delivery personnel, craigslist transactions, or course teaching assistants.

The result was an application where users could create routes with friendly names of [adjective][noun] such as Green Leaf, tune the availability of the route such as Fridays 2-5pm, and attach native communication mechanisms such as text and email. Anyone that had access to the route identifier could communicate with the user by means of animal screen names such as @duck and @zebra. If the route was open, the messages would be pushed to the user via the app, email, and text if indicated. Effectively, the native communication details are hidden and routes can be closed or opened at any time, revoking or granting communication permissions.

You can check out our presentation slide deck and a demonstration video.

Technologies

Me(Nate)

• Python
• AppEngine - WebApp2 Framework
• NDB
• Twilio API
• Mox Mocking (Unit Testing)
• Syrup MicroFramework

https://github.com/nathanielwendt/Handshake

Tomasz

• Java/Android

Goals/Outcomes

Syrup Effectiveness

This was the first project that integrated my microframework Syrup at the onset. A primary goal of Syrup is to pair input/output expectations from the backend tightly with the API docs to better facilitate team development across client and backend. Handshake was a great opportunity to evaluate Syrup's effectiveness in a real-world project. Ultimately, the time saved from direct contract validation and tight coupling of documentation was invaluable. There were few cases of "I expected X and you gave me Y" between the client developer and myself as the backend developer. Syrup greatly help the efficiency of team development for Handshake.

Scalability

We really wanted to make Handshake ready for a real deployment of many users. While this certainly wasn't required for the course project, it was a great exercise and designing the data models and request handlers to best support concurrent actions and try to reduce contention as much as possible. More details of my experiences in designing and testing for scalability are discussed in the challenges section.

Testing Coverage

This was now my third project using AppEngine as the backend and I really wanted to employ TDD where possible and create a test suite that covered the codebase thoroughly. The backend consisted of ~4000 lines of code and ~1800 lines of tests that covered a high percentage of the backend source. This project is probably the most thoroughly tested project I have completed for personal use.

Key Challenges

Generating Unique Route IDs

In line with our scalability goal, I wanted to make sure that route name generation was unique and could support many concurrent requests. Since route names are created from our dictionary of words in the format [adjective][noun], and given we used 500 adjectives and 1,300 nouns there were 650,000 total combinations available. Naturally, we could always expand the list of candidate nouns and adjectives.

I needed to design the backend such that many concurrent requests would still generate unique route names. I accomplished this with the following steps:

1) Naming Model
I created a database model that stored a long list of adjectives or nouns and modeled them in logical bins (e.g. 100 bins). An entry was retrieved by using a timestamp, indexed to a specific bin, and then a random entry was selected from the bin.

*Note this model could have been sharded to better control contention.

2) Naming Generator
The Naming Generator first retrieved a random entry from the adjective and noun naming models. Then, in a separate function a route creation was attempted, if it failed due to a duplicate entry existing, it simply retried until successful. Note that the separate function was marked as transactional, meaning that only a single entry could write within the transaction at a time. It was important to minimize the amount of code within the transaction to limit other attempts from failing. With the transactional protection, two separate requests couldn't generate identical route names and return before knowing of each others existence.

In order to also support our project goal of strong test coverage, I designed tests that flexed the concurrent support of the backend. I did so using the following steps:

1) Mock the dictionary
I wanted to reduce the sample space of the dictionary of adjectives and nouns so I mocked the database model to only include 200 items so many requests would cause collisions.

2) Set eventual consistency to 0
AppEngine supports a testing feature where you can model the database such that eventual consistency is nonexistent. The purpose of doing this is to support repeatability of test results. I didn't want the backend to depend on eventual consistency timing working out to pass the test.

3) Spawn many threads
I spawned several threads to access the Naming Generator each independently. Each thread reported its resulting unique route name and if any of the entries were duplicates, the test failed. My goal was to make the number of total requests close to the state space of the dictionary so collisions would likely happen. I was able to pass my concurrency test and to make sure it was working correctly, I removed the @ndb.transactional decorator from the NamingGenerator and the test would fail, indicating duplicate entries were being generated as expected.

Maintaining Strong Consistency

In modeling the routes and route members (recall that many people can join a route to communicate) it was really important to maintain a consistent view of the entities. This was accomplished using AppEngine's entity group model for relationships and using ancestor queries to resolve members of a given route.