Adrian Pike

How we used Rails & Backbone.js to build a lightning-fast webapp.

Sun, 24 Jun 2012 16:20:00 -0700

Whenever I build a greenfield webapp, there’s a bunch of things that I take into consideration when deciding what tools to use. When we first sat down & started planning out Stride, I immediately started thinking through how I was going to put it together, what tradeoffs we were going to have, and what kinds of tools would fit best.

There were four key things we wanted to solve with our choice of tech stack & architecture:

We wanted it to be fast and responsive to the user. Really fast.
We wanted it to be maintainable moving forward, and make our lives easier rather than harder.
If Stride wound up being a massive success, we wanted to be able to hire against our tech stack. A specific stack or framework affects hiring in two big ways - obviously it’s nice when the talent pool for a given skillset is large, but good folks can learn new things easily. What your tool choice does affect, however, is who’s interested in working with you. As an extreme example, building something in COBOL will self-select your applicants quite effectively.
This fits into the last one nicely - we wanted to have fun building it.

So with those things in mind, we settled on the following big moving parts:

We’re using Backbone.js to provide structure for our frontend, which is a hefty JavaScript app. Everything happens client-side, and is then persisted to the server asynchronously.
The backend is Rails. We chose Rails over other options for a few reasons - Backbone was relatively new territory for us starting out, and if everything went poorly, we could implement more of the app in the Rails side if need be. We thought about working with a lighter Ruby framework like Sinatra or Padrino, but we decided that at least early on, more power & flexibility was better, and we’d optimize later.
Because so much of the app is client-side, we’re using a messaging queue built with node.js and socket.io to send server-initiated messages to the client. These might include user-facing alerts (think Rails’ flash messages), updates to your dashboard. We’re currently using juggernaut, but we’re looking at migrating to something else soon. I’ve had to hack in SSL certificate chain support, and there’s a few bugs that we’ve found, but the big reason we’re moving away is we want persistence for our messaging layer. The current frontrunner is spire.io - their team has been very open and forthcoming when discussing our needs with them.
All of our calculations and intelligence happens in a work queue. We’re using Resque for this at the moment, but as we move forward and things get more complex, we’ll likely wind up with something different at this level. Specifically, as our metrics calculations are going to get a lot more complex, I’m exploring some of the various BigData offerings. I’ve been exploring Hadoop and Mahout lately, and it’s quite likely that we’ll be leaning on them. Resque is great for simply getting things asynchronous, but our as intelligence calculations are getting more and more complicated, it’s time to move to something more suited for it.
We also have some API endpoints that are built with Sinatra. We hope to expose these soon to developers - if you’re interested in building things with Stride, drop me a line - I’d love to chat.

There were some interesting lessons learned, especially with this being our first big backbone.js app as a team.

Having clients be asynchronous was a huge boon for us under high load - the user received feedback immediately, and the actual persistence was able to take some time without the user having to think about it. This worked well for us because only one user can update a given dashboard - once we have multiple users, we’ll have to work through concurrency resolution.
We fell into some bad JavaScript practices early on, and didn’t build as modular as possible. We expected too much out of Backbone, and didn’t do a good enough job of building good, modular code. As with any tool, Backbone’s just a tool - it’s up to you how you use it.
As you build more complicated user interaction on the client, keeping completely RESTful gets hard. Say, for example when you update Resource A, Resource B might need an update, but it has to be up to the server to decide. If there was a page refresh, that would be fine, but if you’re interacting with a RESTful API, how does the server notify the client whether or not to update? Do you always call for an update? Do you attach extra information to the API response? Do you send an out-of-band message back to the client from the server using some sort of messaging layer? We’re currently going with the extra information route, but we’re planning to move to leveraging our messaging layer in the future.
A framework itself can serve as documentation, but when you move outside that framework, documentation is needed. I don’t need to have documentation describing the standard ActionController actions, for example. I don’t need to tell someone what “def show” is doing. But once I step outside the framework, it needs documentation. We’ve got a few places where we’re doing totally non-standard things with either Rails or Backbone, and we haven’t done a good enough job at explaining them to current or future developers.

So that’s a brief overview of the big moving parts of Stride, and some of the lessons we’ve learned so far. It’s been great fun, and I’m very excited with where we’re going. I hope you’ll join us for the adventure.

If you’ve got any comments or questions, please drop me a line on Twitter, or join the discussion on Hacker News.

Analytics from Hacker News & TechCrunch

Sat, 10 Mar 2012 14:50:00 -0800

So we launched Stride yesterday, and hit both the HN frontpage and TechCrunch within a half hour or so of each other. There was a bunch of learnings on the ops side that I’ll save for another post, but one of the things that I’m really excited about is that we have side by side traffic comparisons between TechCrunch & Hacker News.

Hopefully I’ll get the time to analyze and share more later, including some raw data, but here’s some interesting takeaways I saw:

We had ~4500 uniques driven while we were the TC top post & on the HN frontpage.
Techcrunch drove a tiny bit over twice the uniques of Hacker News.
[Subjective] HN comments were way more insightful, interesting, and helpful.
TechCrunch readers stayed on the site ~1 minute 20 seconds longer than HN readers.
HN’s bounce rate was ~65%, TC’s was ~32%.
HN’s Pages/visit was 2.2, TC’s was 3.4
70% of TC viewers went through to our sign up page, whereas only 30% of HN viewers did.
TC had 3% IE, HN had one dude.
TC had 66% Chrome, 19% Firefox, 8% Safari.
HN had 70% Chrome, 13% Firefox, 12% Safari

Note: I sampled all this data around 10PM yesterday.

Moment.js - javascript date parsing/formatting/manipulating

Thu, 26 Jan 2012 13:12:46 -0800

Moment.js - javascript date parsing/formatting/manipulating

brain.js - neural networks & classifiers in JavaScript.

Tue, 24 Jan 2012 15:03:05 -0800

brain.js - neural networks & classifiers in JavaScript.

Signs you're not building an MVP.

Sun, 22 Jan 2012 15:05:06 -0800

Signs you're not building an MVP.

On Content, Availability, and Piracy

Sat, 21 Jan 2012 12:37:39 -0800

The two things that matter to me as a media consumer are content quality and availability. Price is very rarely a consideration when I make my buying decisions - there’s a threshold of quality for my dollar, but if it meets that, I don’t care if it’s $2 or $20.

From the perspective of Old media companies, piracy is about the money. “They’re stealing from us.”

From the perspective of a media consumer, piracy is about availability. “I can watch the movie I want to watch, when I want to watch it.”

Old media is going to keep squashing piracy, and you know what’s going to happen? People are going to follow the availability. Low-budget entertainment that’s widely available is going to take off, and the movie studios are going to be left tilting at windmills.

Loving these shots from my old college roomie,...

Mon, 16 Jan 2012 18:01:02 -0800

Library

Press play

Rust

B&W

Loving these shots from my old college roomie, @josephrlee!

asleepystudio:

Vancouver is totes cool.

shim - simultaneous browsing across devices

Mon, 16 Jan 2012 15:00:05 -0800

shim - simultaneous browsing across devices

Best CSS tabs I've seen to date!

Sat, 14 Jan 2012 15:01:05 -0800

Best CSS tabs I've seen to date!

"Given enough eyeballs, all bugs are shallow."

Fri, 13 Jan 2012 15:00:05 -0800

“Given enough eyeballs, all bugs are shallow.”

- Eric S. Raymond (http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s04.html)

Flickr architecture - great sharding strategies & ideas

Thu, 12 Jan 2012 15:01:06 -0800

Flickr architecture - great sharding strategies & ideas

Etsy Architecture

Wed, 11 Jan 2012 15:01:05 -0800

Etsy Architecture

On language hipsters, C++, and ego.

Tue, 10 Jan 2012 15:41:10 -0800

I read an article today on Hacker News titled “Web programs written in C++ are no big deal.”

Hot damn, says me. Maybe it’s a showcase of apps that utilize some of what C’s good for. Maybe it’s some examples of awesome sites that have lots of C/C++ in their backend. Maybe I’ll get inspired to dust off my C skills and hack out some cool stuff.

Nope — its instead a perfect example of language hipsterism and NIH syndrome, with some good points well hidden.

NIH, for those not acquainted, stands for “Not Invented Here.” It’s an antipattern, not a good thing, and can often happen as a result of overgrown ego’s. Do you really think your code is that much better than everyone else’s? Can you honestly not trust something you didn’t craft with your own two hands? Are you building a spaceship?

Language hipsterism is when your language of choice is cooler, better, and better in bed than everybody else’s. Almost every language is different. They all have their benefits, they all have their weaknesses.

There are some good points that are brought up. Shiny object syndrome is a real problem. Community churn, lack of peer review, all of these are things that you have to think about and weigh when you make a language, framework, or infrastructure choice.

The best developers I know have a giant collection of tools that they can apply to the task at hand, and are constantly learning new languages, techniques, and skills — not latching on to the one they’re most comfortable with and calling it a career.

Web apps written in C++ are no big deal. An app written in assembly is no big deal. What is a big deal is using the wrong tool for the job because of ego, stubbornness, or inertia. We have the greatest collection of tools for software development we’ve ever had — let’s make use of them.

Boo - object oriented javascript syntactic sugar!

Tue, 10 Jan 2012 15:01:05 -0800

Boo - object oriented javascript syntactic sugar!

Fountain codes

Sun, 08 Jan 2012 15:24:06 -0800

Fountain codes

Amazing JS Frameworks for 2012

Sat, 07 Jan 2012 15:19:05 -0800

Amazing JS Frameworks for 2012

Recruiting programmers

Fri, 06 Jan 2012 15:31:05 -0800

Recruiting programmers

Stallman was right. Damnit.

Thu, 05 Jan 2012 15:24:05 -0800

Stallman was right. Damnit.

file.js

Tue, 03 Jan 2012 15:20:05 -0800

file.js

Why I'm buying my bag from Timbuk2.

Tue, 03 Jan 2012 13:53:30 -0800

I’ve been looking at buying a new bag ever since I got a DSLR. I need something that can hold both a Macbook Air and a decent-sized SLR with a few lenses, all the assorted chargers and sundries, and not scream “steal me.”

The bag I use daily is an old Brenthaven laptop bag. I love it, but it just doesn’t have room for a camera in it. One of the things I really love is its strap and pad - it’s covered in this great rubbery coating that keeps it from wiggling at all. Even on top of a slick coat in the rain, with just a little corner of the pad hanging off of my shoulder, I’m confident that it’s not going to slip off and send all my beloved gadgetry to the ground.

My bag search got narrowed down to the Timbuk2 Snoop - I spent a while digging through reviews, and even ran to our local REI to actually feel some in my hands and see if I liked them. So far so good, but the strap worried me a touch. I’ll often just throw the strap hanging off my shoulder when I’m just going inside from the car or having a quick walk, and it felt like I’d really want a better non-slip pad.

So off to Twitter I went, only to be amazed with whoever’s running Timbuk2’s Twitter account.

Not only did they respond instantly, they actually retweeted it out to all their followers - within minutes I had a bunch of excellent ideas and suggestions.

Now that’s support done right, and they’ve made a fan for life. I can’t wait for my bag to show up.

PS: The bag’s on sale as well, snag one!