Project - Syrup
Syrup is a microframework for developing webapp backends as well as a methodology for validating inputs/outputs and for documentation. Syrup also includes a utility, the Syrup Bottler, that scrapes the backend code and automatically creates API documentation from the source code.
Background
Syrup is a project born from my experience developing several Python AppEngine backends from scratch. I noticed that there were often conflicts between clients of the backend API and the API developer (me). Examples include "I expected X response format and you gave me Y". I decided that a tight coupling between the API documentation and the actual backend code would help with this problem as well as encourage good documentation principles in backend design.
Developers don't need to adapt their codebase significantly to integrate Syrup. They simply define an input contract, output contract, and include documentation in their code in accordance with Syrup documentation specifications. From these documentation specifications, the Syrup Bottler can automatically generate API docs for clients of the API. Syrup's bottling process is visually depicted in the following image:
Technologies
- Python
- AppEngine - WebApp2 Framework
- NDB
- Formencode, OAuthLib
Goals/Outcomes
Better Client-Dev Communication
A large goal for this project was to better facilitate communication between the API designer and clients of the API, especially in development cycles where they are evolving in tandem. In my experience, small changes to expectations in either input or output from a backend caused large problems for the client. Syrup served as a communication medium between the parties since the generated API documentation is tightly coupled with the source code. API developers can then focus on writing code and the documentation will reflect the changes in the code while still maintaining API abstraction.
Flexible & Extensible
I wanted Syrup to be useful for various developers across various types of projects. In order to do this I often had to think of the general case when developing a feature or aspect of Syrup. Where applicable, I provided customization or generalization to support extension of Syrup as well as configurability in hopes of making Syrup more useful. One concrete example of this extensibility is in the input validators (url/num/geo/email/etc). Clients of Syrup can extend the input filters to validate input according to their own specifications. Syrups continued development will hopefully continue with this goal in mind.
Key Challenges
Since this microframework evaluates input and output JSON values, there are many instances of recursively following nested structure in JSON arrays and objects. This coupled with the special notation created for depicting fields as optional or required, created a challenging problem of evaluating the nested nature of the JSON correctly. While not incredibly challenging from a conceptual standpoint, there were plenty of edge cases to trip up on. Hence, unit tests were used liberally to ensure the result was correct.