Would you risk driving at speeds up to 100mph (on the Autobahn of course ;o) for a distance equivalent to driving from Los Angeles to New York (2,500 miles) in a concept car which has only been driven across the car park – oops parking lot (my Britishisms got the better of me!)? The answer is you should not if you want to arrive safely and in a reasonable time-frame. Yet I have seen exactly that analogous situation many many times web applications get rolled out before anyone has made sure that they will actually work with more than 1 user.
As I have mentioned a few times in my blog posts, I am fortunate to have spent the past 8 years traveling to many parts of the USA and other parts of the World looking at ColdFusion and JRun applications. Some of the things I have found are mind-blowing in the sense that they actually ever reached Production, which they all did. Here are a few examples.
- Scheduled Tasks being instantiated in the Application.cfm file with no conditional code. The result being that tasks were being instantiated as they were running with every single request, the result was a very unstable application.
- A JRun Java app where there were debug writes to the std-out log throughout the code in Production. The std-out log grew so fast it was virtually unreadable after 30 minutes. The log was growing by 3gb per hour.
- NIC’s and Switches both set to autosense, the result being that all devices in the infrastructure, Web-CF Servers, Database Servers etc were pegged at 10mbps non-duplex. When I arrived on site the client had a 12 Web Server, 3 DB Server farm up that was dying at 500 concurrent users and needed to support 5,000 concurrent users. That site currently supports over 6,500 concurrent users with 6 Web CF servers after we fixed the settings. So it is not always code that causes issues.
My main point here is that we should never use a Production environment or put anything there until everything has been very thoroughly tested. In the cases above, all these serious issues would have been discovered with a proper code deployment regime which includes testing everything and I do mean everything; first. There was a previous blog post I made relating to testing code before deploying it.