Building and compiling code can take a huge hit on our time and resources. If you have dockerized your application, you may have noticed how much of a time-saver Docker cache is. Lengthy build commands can be cached and not have to be run at all! This works great when you’re building on a single host; however, once you start to scale up your Docker hosts, you start to lose that caching goodness.
Keep reading
We use a number of external services to handle data retrieval, monitoring, and reporting. The importance of each service depends on its role in our sandboxes product. For instance, it’s generally fine if we miss a few reports sent to Datadog but it is not okay if we are unable to reach the GitHub API. In this post, we’ll discuss the issues we’ve faced when interfacing with external services and dive into how we use Varnish Cache to handle them.
Keep reading
I see Runnable as a very versatile, and therefore powerful, tool. It enables teams to deploy and run code on every commit. The concept sounds simple, but it lights up a vast number of not-so-obvious use cases.
For example, here’s how Runnable can be added to an existing revision workflow for static sites / blogs to improve how updates can be made.
Our problem started out benign enough: we were dealing with third party endpoints that started dropping requests and we could no longer count on to be reliable. The immediate solution was to write retry logic in-place within our API server that instigated these calls, adding a layer of protection around the library that did not have its own retry logic. However, when put under load, this retry logic started becoming a bottleneck, holding onto unnecessary resources and creating race conditions we had never before seen. Debugging it started becoming nightmare-esque and maintaining it became nigh impossible.
Keep reading
How much do you log (in your application)? Do you use tools like Bunyan to make your logs easier to read? Do you Stringify everything and print it straight to console.log()? If so, logging may be the reason why you’re having trouble reducing your app’s average route response time to under 200ms.
When we first started implementing a unified logging system throughout our API, we didn’t anticipate how something as simple as logging to console could possibly slow down our system. Our data models were in their infancy; they barely had actual data on them, so it became common to just Stringify the whole object into the log. It sounded like such a great idea! Investigating bugs was easier since we had a record of the entire data object during each step of our flow.
But then things started to change…
We had just performed an overhaul of how we handled logging across our services at Runnable. It appeared to work as intended — log files were being rotated and sent over to Loggly. I later learned we had a bug that was causing logs to be improperly formatted.
“Logs aren’t formatting correctly for me.” My fellow engineer showed me what looked like JSON, which is what our various services produce as log output.
“Looks OK to me,” I offered trepidatiously.
“You would think so, but when I pipe it into Bunyan, nothing happens.”
Sure enough, Bunyan wasn’t re-formatting the JSON as expected.
Sockets can be hard. Scaling out websockets can be even harder. At Runnable, we make use of websockets heavily — for notifying users that their containers are running, for implementing deployment messages, and for powering our terminals in the browser. Terminals are a tricky beast to tackle. I learned that a few weeks ago, when our users’ terminals were getting lost and reset based on uncontrollable network issues.
Keep reading
Over the last months, we‘ve been converting our code from using callbacks to using promises. In our coding style, we‘ve found promises to be a cleaner way to organize code and a better way to deal with error handling. As we‘ve done more and more of this, we‘ve gotten better at identifying effective patterns for using promises and the best ways to migrate to them. We‘ve also found Bluebird to be the best promise library out there. Bluebird not only provides solid performance, but it also provides wonderful abstractions over promises.
In this article, I‘ll show you some of the more useful methods in Bluebird and how we use these here at Runnable. Some of these are taken directly from our codebase in order to help out anyone looking to start migrating to promises or just improve and clean up your current implementations.