a tornado of razorblades

In my QCon talk Horizontal Scalability via Transient, Shardable, Share-Nothing Resources, I argued that memcached is the father of modern shardable resources. Today’s NoSQL key-value stores all owe some part of their inspiration to memcached. Even feature-rich datastores such as CouchDB or Cassandra also borrow a cornerstone idea from memcached: throw away some features historically associated with databases in order to make big gains in scalability and resiliency.

If your app needs to poll a remote API once an hour, or send out an email report every evening, what tool do you reach for? Probably cron. Triggering events at a given wall clock time is what cron is for, but it works better at the system layer (e.g. rotating logs on a server) than at the app layer (e.g. sending out a daily report to your app’s users). I’ve described all the ways cron could be improved for app clock events in a previous post.

Like many folks in the startup crowd, I’m a reader of Eric Ries' blog (some links), and I’ve read Steve Blank’s Four Steps to the Epiphany. What I didn’t know is that these guys have joined forces to build a movement they are calling “lean startups.” After attending the Startup Lessons Learned conference last week, I now believe this methodology is on its way to making a major impact on the world of entrepreneurship.

Web apps are increasingly focused on background jobs. In fact, the term “background job” almost seems inaccurate - the heavy lifting done by worker processes is often the meat of the app’s purpose. The web portion of the app, by comparison, does only the relatively lightweight work of putting job requests into queues, and later presenting the results of jobs as HTML or JSON.

Cron is a trusty tool in the unix toolbox for scheduling work to run at periodic intervals. In addition to system tasks, it’s common for app developers to use an app-specific crontab to run application tasks. For example, if your app is a feed reader, you might use a cronjob to fetch new feeds every three hours, and another cronjob to clean out old unread articles every night.

When you hear the term URL, what do you think of? Probably a web address - e.g., a publicly accessible HTML page such as http://google.com/ or http://news.ycombinator.com/. But URLs have a much wider application.

Inspired by the lean manufacturing revolution (and excellent books like Lean Thinking), I started with a first fundamental question: in a startup, what activities are value-creating and which are waste? Usually, new projects are measured and held accountable to milestones and deadlines. When a project is on track, on time, and on budget, our intuition is that it is being well managed. This intuition is dead wrong.

If you’ve been using polling to track Twitter search terms (totally random example), you may have wondered if there is a more efficient and reliable method. The Twitter streaming API is a potential solution.

A company with a culture of quitting does not have ex-employees; they have alumni. This is far more than a semantic distinction. An alumni relationship is positive; something that people can take pride in; and one that keeps the door open for further opportunities on both ends.