I’ve spent the past few days trying to develop a simple mathematical model to predict the expected availability of complex systems. In IT, we are often asked to develop and commit to service level agreements (SLAs). If the points of failure of the system are not analyzed, and then the system availability calculated, the SLA is flawed from the beginning. To complicate matters further, different people have different definitions of availability. For instance, does scheduled downtime for maintenance count against your system availability calculation?

Common Availability Definitions:

1. Availability = MTBF/(MTTR+MTBF) (Mean Time Between Failure, Mean Time To Recover). This is a classic definition of availability and is often used by hardware manufacturers when they publish an availability metric for a given server.
2. Availability = (Uptime + Scheduled Maintenance)/(Unscheduled Downtime + Uptime + Scheduled Maintenance). This is an IT centric availability metric where the business can support scheduled downtime after hours. This model works for some types of systems, such as a file server that isn’t needed at night, but it doesn’t work as well for websites, even though many web companies still use this for their SLAs.
3. Availability = Uptime/(Uptime + Downtime). This metric best applies to systems that are needed 24×7 such as e-commerce sites.

Availability is most often expressed as a percentage. Sometimes, people will refer to “four nines” (99.99%) or “five nines” (99.999%). To simplify things, the following table shows the minutes of downtime allowed per year for a given availability level:

(read more…)

My new job with StubHub came with a host of excellent benefits, including a shiny, new 401K with Charles Schwab. Schwab is generally known as a good, stable company with a strong online presence, so I was shocked by what arrived in the mail today. About a week after signing up for my 401K, I received a letter from Schwab titled “Confirmation of Personal Identification Number Change,” and right below the subject line is the password I had chosen for the website! To make matters worse, the letter came in an envelope from Charles Schwab labeled “Personal and Confidential,” ie. “STEAL ME.”

This letter got me thinking about all the supposedly strong security mechanisms employed by various online companies that I deal with that just make matters worse. The schwabplan.com PIN # confirmation is just one example. I used one of my common passwords expecting Schwab would treat it with the utmost care. To me, this would mean storing it in an encrypted, non-human readable form. Ideally, the password itself would not be stored at all. Instead, a hash of the password would be stored, and any time I entered my password, the hash of what I entered would be compared to the stored hash. This would protect my password from unscrupulous Schwab insiders, since statistics show that approximately 70% of security breaches occur from the inside. (read more…)

Two months ago, I had no idea who Hannah Montana was. My daughter is too young, thankfully, to care about Hannah and my nieces had not yet introduced me to the phenom. Now, she is my best friend. I love her! she rocks!

I should probably admit that I still really don’t have any idea who she is, and have never heard her music. The reason for my new found respect for Hannah is that two months ago, I changed my day job. I am now running technical operations for www.stubhub.com, a subsidiary of eBay.

If you’ve never heard of StubHub, click the banner on the left. StubHub is the leading secondary marketplace for concerts, sports events, and theater. If you want tickets to the World Series, the Super Bowl, or a sold-out concert, there is no better place than StubHub. And right now, Hannah Montana and Baseball Playoffs are the hot tickets.

People are going nuts for Hannah. As I write this, floor seats in Oakland, right in front of the stage are going for $1,500. There is also a luxury box with 20 tickets for over $11,000! This is the gotta have, must see, take me PLEEEAAASSSEEE!!!!! concert of the year. I love it!

I joined StubHub because it is truly my kind of company. First, it is a company with a solid foundation in the bricks-and-mortar world. People have been “scalping” tickets for a long time. By creating a neutral online marketplace, and backing it up with solid logistics and world class customer service, StubHub became the dominant player in the secondary ticket market. Second, it is a company that values its technology and its technologists. As such, it is a great place for an IT guy to work. Lastly, it is growing exponentially. The opportunity to design and build a highly-scalable, highly-available technical architecture was one I could not pass up.

Conceptually, I also love the free-market approach to ticket sales. StubHub does not take inventory of the tickets. We offer a secure place where fans can buy and sell tickets, and let the free market, not the ticket promoters set the market price. Hannah is a great example. The news is full of articles this week on parents complaining of being “gouged” by the ticket brokers. The Attorney General of Arkansas is investigating! What the people crying about the price seem to forget is the old laws of supply and demand. If they weren’t so desperate for the tickets, the price would fall.

I’ve neglected the blog lately, trying to get up to speed with the new gig. In the coming months, I have a bunch of articles planned based on the scalability challenges I am now facing. They should be worth the wait. In the mean time, visit StubHub, buy some tickets and go see Hannah. Let me know how you like the show.

PS> If you simply must see Hannah (in other words, you have a daughter), follow the StubHub advice for buying Hannah tickets. It may save you some money.