It was the shot heard round the hosting world. Last month, my good friends at RagingWire announced their latest offering, IronScale, which has the potential to fundamentally change the hosting business. At least, that’s what the press release and the voice mail I received from Doug Adams, their head of sales claimed. Now, I’ve been doing business with RagingWire for almost 8 years, and I often tell people they have the best designed/built/run data center in Northern California, so I know they offer great services. I’m one of their only three-peat customers (I’ve put three different companies into their facility) and I’ve never been disappointed. Still, I tend to discount terms like “game-changing” as marketing fluff. I’m a “show-me” kind of guy. So they did.

Today I had the pleasure of an on-site demonstration and walk through of the IronScale service. I am impressed. On the surface, it is a typical managed server hosting offering. You rent one or more dedicated servers in their data center and they provide the operating system, network, internet bandwidth, security, etc. Pretty common stuff, and pretty boring. Why did I drive to Sacramento on one of the hottest days of the year for this (110F)? Well, you have to look beneath the surface, which I did, to see what they are really offering. At what I saw was awesome.

IronScale makes some bold claims for their services. Among them:

• Dedicated physical hardware (Not VMWare or Zen virtualization)
• Less than 5 minute server deployment
• Instant reconfiguration of servers
• Instant backups
• No cost to reload the operating system
• Everything easily manageable through a web portal, including provisioning, console access, network configuration, firewall rules, backups, and bandwidth provisioning
• On demand Raid 50 storage
• Enterprise class network security

The first two bullet points are what instantly sets IronScale apart from the competition. Most hosting provides focus either on rapid deployment or dedicated hardware, but not both. If you want dedicated hardware, then provisioning time takes 24-48 hours, because it takes time for an engineer to Ghost/JumpStart a server and put it on the network. If you want rapid deployment, you use virtualization technologies like VMWare. RagingWire figured out a Door #3.

Some of what I learned today is under NDA, and the product is in Beta and patent pending, so I need to tread carefully, but here is my best explanation of what they are doing. The core of the product offering is commodity servers connected to an enterprise-class storage area network. Rather that laying an OS down on the physical server, they SAN-boot it. Provisioning a new server is as simple as taking a SAN snapshot of an existing server image, and assigning it to a new server. Each server is connected to a high-end Cisco switch with integrated firewall that provides security and isolation from every other server at the port level.

Simple concepts. The special sauce here is not the mix of technologies, but the software they’ve written to control it all, and do so simply and securely. Want to provision a new server? Grab one out of the pool assigned to you, pick the OS baseline image you want on it, name it, give it an IP address from the available pool, and boot it. Click the next tab and you can write firewall rules in an easy to understand format. The IronScale software translates into the appropriate Cisco commands and applies the configs to the firewall. Need more storage? Pick the server, assign more space, and the software updates the SAN to allocate space to that image. The running OS instantly recognizes it as new available space (no reboot required).

Need a bigger server? Shut down the running server. Assign the image to a bigger box, and reboot. Want to backup your servers before applying the latest updates from Microsoft on Patch Tuesday (you know, just in case…)? Take an instant SAN snapshot. If something goes wrong, revert back. No calls to customer service. No waiting for days or weeks. It is fast, and it is easy.

Basically, this IS virtualization but not like what you’ve seen before. Instead of VMWare, Zen, Cloud, Grid, pick your buzz word virtualization, this is storage-based virtualization combined with some really kick butt management software. The servers are dedicated but the storage is not. The beauty of this is, among other things, it isn’t OS restricted. Although the initial offering is based on Windows and RedHat, there is no reason they can’t support Solaris (x86), BSD, or other flavors of Linux in the future. They don’t need to hook into the OS to perform their services, so any OS that will run natively on the hardware will work. And because the servers they are running are very generic, driver support should not be much of an issue. All you need is the 1st OS image, and the SAN snapshots do the rest.

This service is clearly designed for the mid market. I would say their sweet spot is customers wanting 5 – 50 servers, but the service can scale up to hundreds, if not thousands of servers. For bigger companies, it would make a good platform for proof-of-concepts, development environments, or any situation where you need to be able scale capacity up or down rapidly and make frequent changes. What would push this into uber-cool status is if IronScale works out billing based on time-slices. So, if I want a pool of 50 servers to generate load for testing purposes, but I only need them 1 hour per night, or I need a compute farm for end-of-month processing, I could pay just for the time I use the servers and shut them down the rest of the time. Other companies are offering capacity on demand, but most of those are grid-based, and you need to modify your applications to take advantage. IronScale could do capacity on demand with zero customer modifications.

This is a Beta product, because they are still adding features and I’m sure fixing bugs in the management software, but the core offering is fully baked and ready for prime time. I expect in the future, they will layer on a plethora of additional services, such as advanced security scanning, server virtualization to increase utilization of the hardware, and database server clustering. I sincerely hope they also package their management software and start selling it to enterprises in the future. Based on what I saw today, I would buy it with some minor tweaks.

I am planning to beta test this offering in the next few weeks, so hopefully I’ll have more information, and some screen shots soon. In the mean time, I encourage you to check out the online demo and give IronScale a serious look. http://www.ironscale.com/videos/demo

I was reminded of Cheshire’s article and the underlying principles recently when working on an international WAN design. What Cheshire noted was that light signals pass through fibre optics at roughly 66% of the speed of light, or 200*10^6 m/s. Regardless of the equipment or protocols you use, your data cannot exceed that theoretical limit. This limit equals the delay between when a packet is sent, and when it is received, aka latency.

In the US, we tend to focus on bandwidth and carrier technology when ordering circuits, completely ignoring latency. For instance, when choosing between cable and DSL for your house do you ever ask the carrier for its latency SLA? Maybe you should. Using a cable connection a ping to www.google.com in Mountain View, CA from my house (137 KM) yields an average ping time (aka round-trip time or RTT) of 73ms. The theoretical latency for this distance (round trip) is 1.37ms meaning my cable connection is roughly 50 times worse than the theoretical limit. No surprise that Comcast focuses on bandwidth and not latency in its marketing.

Cable and DSL circuits in the US are generally not business class and do not carry any service level agreement (SLA) on latency or availability. Businesses who use these circuits for business critical services do so at their peril. Business circuits such as Frame Relay and MPLS do generally include latency SLAs, but understanding the difference between the SLA and your actual experience can have a massive impact the performance of your network. For instance, let’s say a carrier advertises a 55ms round trip SLA in the US. This SLA equals the maximum latency between any two points of presence (POP) on their network.

The coast-to-coast distance in the US is roughly 5,000KM for a theoretical latency of 50ms, so a 55ms RTT SLA is pretty good. But that doesn’t mean packets on your network will only take 55ms to cross the country. When designing your WAN you must also account for the latency added by your network equipment and your servers, and the distance between the carrier’s POP and your offices. As a result, a well designed US WAN will still experience 75-80ms ping times. A poorly designed WAN can experience much worse times.

Now consider creating an international WAN. In this case, you typically will receive multiple SLAs from the carrier for different parts of the network. For instance, when designing an MPLS connection between California and the UK, the SLAs would be approximately 55ms within the US plus 95ms to cross the Atlantic Ocean plus 21ms to connect within the UK. Add the latency of your network and you get ping times of 175ms to 200ms.

At this point you are probably asking yourself “so what?” Two tenths of a second is no big deal. The answer is the impact of latency on TCP windowing. Transmission Control Protocol (TCP) has a flow control mechanism that senses latency and bandwidth between two hosts and determines the rates which data will be transferred. The TCP window is the amount a unacknowledged data a sender can transmit before waiting for a TCP ACK. As the latency increases, the TCP window shrinks, meaning the sender sends less data before waiting for an ACK. This helps reduce the amount of data that will need to be retransmitted in case a packet gets lost. Smaller windows equals more packets, and more packets equals more data because each packet carries the overhead of a 40 byte TCP/IP header regardless of if the payload is 1 byte or 1500 bytes.

The result is what I call the “Sandbag Problem.” Let’s say the two of us are trying to fill sandbags. My job is to scoop sand into a container and hand the full container to you (data). Your job is to empty the container into a sandbag and hand the empty container (ACK) back to me. Occasionally you drop the container so I have to fill it again (Retransmit). If we were standing next to each other, the time it takes for me to hand the container to you, have you empty it, and hand it back to me (latency) would be very small. Now imagine there is a 6′ wall between us, and I need to hand the container over to you.

The wall changes several aspects of our filling operation. First, the size of the container must be smaller because I cannot lift the same weight over my head that I can lift at waist level. Second, the time to complete one cycle would increase because it takes longer to lift the container 6′ than it does 3′. Third, you would drop more containers so retransmissions would increase. As the wall gets taller, the problem gets worse. If the wall were 10′ tall, we would be throwing containers instead of lifting them, so they would need to be even smaller. The containers would be traveling 20′ round trip instead of 12′ so the delay would increase 75%. And we would need to send a lot more containers to move the same amount of sand.

TCP works just like the sandbag problem. As distance increases, the TCP window shrinks, the time between transmission and acknowledgement increases, and the number of packets required to move the data grows. One reason for this is the effect of TCP congestion avoidance algorithms on the window size. The result is that the effective “speed” of the link decreases exponentially as the distance increases, regardless of bandwidth. RFC 1323 TCP Extensions for High Performance provides for mechanisms to deal with part of this problem. One method is to tune the TCP window on your hosts based upon a calculation of Bandwidth Delay Product (BDP). BDP = bandwidth x delay. Example: A 2Mb/s E1 link between California and the UK would have a BDP of 2.048Mb/s x 200ms = 51,200 Bytes. This is the ideal TCP window to fill the pipe so that the sender is not sitting idle waiting for ACK packets. Most hosts have a TCP Window default size of 64KB so, in this scenario, no adjustments would be needed. But, if the connection were a 45Mb/s DS3, then the BDP would be almost 1,100KB. In this scenario, TCP windows would need to be adjusted to use the available bandwidth at peak efficiency.

For most network applications, anything over 100ms latency is noticeable to your end users. Time sensitive applications such as VOIP or video teleconferencing suffer the worst experience when delay is introduced. Added to this is the impact of jitter. Jitter is the delay caused when packets travel alternative paths to the destination, and either arrive out of order, or with varying intervals between them. Applications such as e-mail that are bursty and not time sensitive do not feel the impact of latency to the same degree. How much of a problem is this for you today? One way to measure latency on your network is to use your carrier’s looking glass tools. A list of major looking glasses may be found at: http://www.nanog.org/lookingglass.html.

When designing for latency in a WAN it is important to first understand the applications on the network. After the applications have been profiled, steps can be taken to mitigate the impact of network delay. In part 2 of this article, we will discuss methods of designing for latency mitigation.

With a 100% rebate, how can you lose? Order today.

With that said, I want to actively promote the concept of e-Tipping. e-Tipping is a way to pay the blogger back for the hard work they have put into their blog, similar to leaving a tip at a restaurant. There are several ways to leave an “e-Tip”:

1. Click the ads!!! – Most blogs these days have ads. If you like the article you just read, visit the site’s sponsors. The blogger will make, on average, about $.05 per click…not much of a tip, but it adds up when a lot of people are reading your blog. UPDATE 06/01/2007: It is against Google’s terms and conditions to directly ask people to click your ad links. I respect Google and their terms, and would not want to circumvent their business model. Most Google ads are contextually related to the blog article. So, if you find an article valuable, take the time to look at the ads. If you are interested in any of the products, by all means click the ad, but please do not click ads soley for the purpose of driving up click revenue for the blogger.
2. Donate – Many blogs offer paypal links. If you find the articles especially useful, make a small cash donation. This is often the best way to support a blogger if you want him to provide you with specific additional information.
3. Leave a Comment – Blogging can be a lonely business. Comments show you care. They also make articles seem more relevant to the next reader.
4. Digg/Slashdot/Link the article – Bloggers want traffic. The more the better. Also, Google ads pay for page views, as well as clicks. Help the blogger promote their site, and they will continue to create great content.
5. Read the rest of the blog – Chances are you found the blog from a link aggregator. If you find the article useful, click out to the parent site, and scan some of the other articles. You’ll probably find other articles of interest, worthy of your time and e-Tips.

In the past month, edgeblog has benefited greatly from other blogs and sites linking to us. As an e-Tip to them, let me encourage you to visit each of these. They are quality sites worthy of your attention. Like the comic said, “I’ll be appearing here every night this week, and don’t forget to e-Tip your blogger!”

-Bill

Please visit our friends and leave an e-Tip:

• http://www.gadgetworkshop.com
• http://www.msmobiles.com
• http://www.thedigeratilife.com/blog/
• http://www.reddit.com
• http://www.geekrant.org
• http://www.zoliblog.com/blog
• http://www.afarther.com/
• http://blog.ambersail.co.uk/wordpress/
• http://bloggerfodder.net/
• http://hangoversunday.com/
• http://arghwebworks.com/
• http://reddiggulo.us/
• http://www.taleslinger.com/
• http://gfxfor.us/

Digg This Story!

Which brings me to the title of this article. During my studies for the CEH exam, I was exposed to the seriously flawed CartIt.cgi shopping cart application. CartIt.cgi is a widely used shopping cart that stopped being developed last year. The reason this application is flawed is that it uses hidden fields within the HTML POST to submit the price and quantity when the user clicks on the add-to-cart button. Hidden fields are easy to manipulate. One of the easiest is to use a local proxy, such as Paros, to intercept the POST, effectively launching a man-in-the-middle attack. This allows you to change the price before it is submitted to the server.

Example:

Doing a simple Google search for cartit.cgi+plasma, I found a web site that sells plasma TVs (Which shall remain nameless to prevent being sued). The website thinks it is selling TVs for $7,599, but we can pay whatever we want by intercepting the POST and changing the price. If you think the company would catch this error, think again. Many companies outsource the fulfillment of orders, and never check the prices being charged. Note: I do not endorse e-shoplifting, so I did not complete the above transaction, but I know for a fact that the site will accept the order for $.99. Now, $.99 is extreme enough to *maybe* raise a flag. A simpler approach is to just move the decimal over 1 or 2 places. This way, if the company does notice, they will assume it was a processing error on their side. So maybe this article should be titled: “How to buy a 65″ plasma for $75.99.”

Another simple search for CartIt reveals that many hosting companies are still actively supporting CartIt.cgi. For example, IM1 Web Hosting calls CartIt “a powerful e-commerce solution for merchants and professional Webmasters…CartIt is an extensible, scalable shopping cart system that can handle just about any product or product combination you throw at it.” Disgraceful.

Note also that the shopping cart displayed above was deemed secure by VeriSign, Control Scan, BBB Online, Mastercard, & Visa. How much confidence do you have in those programs now??? Hopefully not much.

The exploit described above is not unique to CartIt. There are many shopping carts that use hidden POST fields. A shopping cart should allow the user to submit the SKU and the quantity, but never the price. The price should be queried from a database. The point here is that if you do not know how your applications work, you cannot rely upon their security. If you are using a shopping cart provided by your hosting company to run your site, we recommend you check it for these exploits. Failing to do so can be hazardous to your bottom line.

-Bill

Digg This Story!

Editor’s note: The techniques described in this article are for educational purposes only. We do not encourage or endorse the manipulation of 3rd party web applications to change the price. E-Shoplifting is a crime.

According to our friends over at MS Mobiles, the situation changes Thursday, the 16th of November. Cingular is set to ship 2 new smartphones:

• The Samsung Blackjack is a Q killer. It offers G3 HSDPA broadband speeds, which are more than double the speeds of Verizon’s “high-speed” broadband. It is also a quad-band phone, making it well suited for world travelers.
• The HTC Hermes, aka Cingular 8525 is the latest version of its popular pocket PC phone, which offers full touch screen PDA functionality. It is similar to the Verizon XV6700, but like the Blackjack, the 8525 offers HSDPA broadband. The 8525 also has a 2 megapixel camera and 802.11 support.

Both of these have great multimedia capabilities, such as streaming audio and video, but they really shine as business tools. If you have Microsoft Exchange 2003, these phones are a must-have. In Exchange 2003, Microsoft beefed up its wireless activesync. With a Windows-based phone and an accessible Exchange 2003 web access server, you can synchronize e-mail, contacts and calendars over the cellular network. With Exchange SP2, Exchange can push data to your phone, but this is a huge battery drain. The better option is to set your phone to synchronize every 5-15 minutes. Exchange combines with a Windows Smartphone blows away the capabilities of blackberrys and you don’t need any extra software to roll this out to your entire company.

I’m going to order the Blackjack as soon as it is released on Thursday, and hope to have a real-world review up soon.

After reading this book, I decided to write my own updated list of 10 Immutable Laws of Information Security. These 10 rules represent years of experience, hundreds of projects, and countless mistakes:

1. There is no such thing as perfect security – Systems designed by humans are vulnerable to humans. Bugs exist. Mistakes are made. The things that make your computers useful, i.e. communication, calculation and code execution also make them exploitable. Information Security is the management of risk. A good infosec design starts with a risk profile, and then matches solutions to the likely threat.
2. If you’re not part of the solution, you are part of the bot net – Failing to protect your systems is no longer an option. Firewalls, anti-virus, and patch management are required as a cost of doing business. Every system you fail to protect will quickly become a launching point for more attacks against others. Wide spread attacks such as Code Red and Nimbda spread because basic security mechanisms were not employed. Your mistakes threaten my systems, and my mistakes threaten you.
3. Your defenses must be perfect every time; the attacker only needs to be lucky once – See rule 1. Attackers look for the easy way. The best firewall in the world will not prevent a hard drive from being stolen. Your security policy must take a holistic approach to your systems, and then minimize the impact of an exploit. A good place to start are the 10 security domains identified by the ISC2: Access Control, Application Security, Business Continuity & Disaster Recovery, Cryptography, Risk Management, Compliance, Operations Security, Physical Security, Security Architecture & Design, and Telecommunications & Network Security. Analyze each of these areas against the 11 Security Dimensions in the Open Source Security Testing Methodology Manual(OSSTMM) and you’ll be on your way to a solid defense: Visibility, Access, Trust, Authentication, Non-Repudiation, Confidentiality, Privacy, Authorization, Integrity, Safety & Alarm.
4. Your data center is only as secure as your administrator’s PC – These days, most data centers have good physical security, but none of that matters if the administrator has full remote control of his systems. Install a key-logger on the admin box, and you own the network. Forcing privileged users to sit in an unrestricted cube farm with the rest of your employees is just asking for trouble.
5. An unsupervised janitor is the richest guy in your company – See rule 4. As I’ve discussed before, a USB key with U3 and a PC with AutoPlay is all it takes to get passwords, install software, and generally 0wn a PC. Couple that with your administrator’s terminals and you have a recipe for disaster. Would you really trust your janitor to do the right thing if I offered him $1,000 to plug a USB drive into a PC for 10 minutes and then bring it back to me? Physical security extends beyond the data center to include every system that has privileged access. How secure are your admin’s home PCs? Your CIO’s?
6. Everybody Lies – Your users lie when they say they didn’t open that attachment. Your administrators lie when they say they’ve verified all your backups. Your vendors lie when they say their solution will fix all your problems. The attacker on the phone claiming to be a help desk agent who needs your password is lying. Good security minimizes the capability to lie, and the impact of the lie.
7. Usability increases security – The best security controls are the ones that are mandatory and transparent to the end user. The worst controls are difficult to use and require the user to change his/her behavior. Automatically redirecting your web pages to pages that use SSL increases privacy while being effortless on the part of your user. Requiring a user to have 36-character password with special characters, and forcing them to change it every 7 days, may seem more secure but it forces the user to write the new password down and tape it to their monitor just so they can remember how to log in. Don’t confuse complexity with security. Usually, the opposite is true.
8. It is easier to design security upfront, than to bolt it on later – Often, small changes to an application or a network can yield big security returns. Making these changes once the system is in production, however, can be very costly. Adding a security review to the early stages of your projects will prevent many future headaches.
9. If a defense can fail, it will – Murphy was right! Build redundancy and defense-in-depth into every design. Focus both on preventing a failure and on minimizing its impact. Storing confidential data encrypted inside a database will minimize the loss if the database authentication fails. Adding anti-virus firewalls to your network will help stop the spread of WORMS from personal (unprotected) laptops. Always assume the worst case and plan accordingly.
10. A motivated attacker will always trump a diligent defender – See rule 1. If the bad guy wants in, and has enough motivation, he will get in. Period. Why do the best protected networks of the DOD and FBI still get compromised? Because the motivation to get in is high, and the attacker has unlimited time. Fortunately, the reverse is also true: An unmotivated attacker will always lose to a diligent defender. Hackers are lazy and they go after the low hanging fruit first. Minimize your public profile, and you will reduce the number of attacks. A web server that is filtered by a firewall and only allows port 80 & 443 looks a lot less attractive than an unprotected web server that also responds to a couple dozen other ports. Reducing the number of attack vectors reduces the number of attacks and attackers.

That’s my list. Ignore it at your peril! Leave me a comment with your top laws, and thanks for stopping by.

-Bill

Digg This Story!