AnandTech: A Brief History of Time
by Jason Clark on July 26, 2004 6:52 PM EST- Posted in
- IT Computing
AnandTech 5.0
After 5 years on the ColdFusion platform, the time had come to move on to what we think is a better platform for the future of the site - Microsoft .NET. The decision to move away from ColdFusion was a tough one in some ways. ColdFusion offers a tremendous advantage over other languages, thanks to its RAD (Rapid Application Development) roots, but the trade offs were cost and stability. Cost was definitely the major factor in our decision. Our server platform is standardized on Microsoft Windows, so .NET is free for us, while ColdFusion costs us $1299 USD per server. As we continue to grow, and keep up with the newer versions, the costs rise. One could argue that we would make up for those costs with the time that ColdFusion saves the developer. However, we found that not to be the case, and we'll explain why later.We did quite a bit of testing prior to beginning development, writing a skeleton framework and load testing it and comparing the results to ColdFusion. The results were attractive with .NET definitely, offering some serious performance gains (more to follow on our benchmark results). The architecture of the new AnandTech.com site consists of a VB.NET API behind the ASP.NET user interface; the API can be exposed as a web service with the flick of a switch, if necessary. We chose VB as it is a more natural progression from ColdFusion syntax, and offers the same performance and features as C# does. VB.NET is a completely different animal than VB; although the syntax may be similar in some ways, it offers a heck of a lot more than VB ever did.
Writing the API is what took the most time, as we had to port over our back-end framework from ColdFusion into our new VB.NET API. Our first step was to write a few functions that handled some of the tedious work in writing any web application: a database class that handles all the database calls, which reduces the amount of lines of code we had to write for each call; a few list functions; and, some other little functions that ColdFusion had built in. Those functions took a few hours at most to write, and covered the main functions that we had taken for granted in ColdFusion. After those functions were written, we found that the RAD argument started to diminish, especially with the slick IDE that Microsoft offers. Visual Studio .NET is, bar none, the best IDE that we've seen. It offers a robust development environment with extensive function insight features, which really speed up the development process. If Macromedia were to develop this kind of IDE for ColdFusion, I think the differences between the two platforms would most definitely start to diminish.
The next major piece of work that we did was to transition the interface from HTML to CSS. This was a major undertaking, but once the framework was complete, the rest just fell into place. The old site was all HTML, and had a few too many tables in it. The result was a slower rendering site, and combined with the ads, it only made the situation worse. As a result, we took a 130K site down to around 45K in HTML. Needless to say, the page rendering times are night and day in comparison.
Hardware used in version 5.0
5 x Dual AMD Athlon MP 1900+ w/ 1GB Memory
So, after we were finished, how does it perform? Take a look below.
Note: These benchmarks compare the old anandtech site to the new anandtech site, not ColdFusion to .NET. There were many changes that resulted in the performance increase, the platform was just one of them.
The graph above depcits how many requests per second the server was handling during the load test.
The graph above depicts how long the site takes (under load) to deliver the last byte to the client.
Facebook
Twitter
RSS
67 Comments
View All Comments
PorBleemo - Wednesday, July 28, 2004 - link
vda #30:You forget that their servers are a little more powerful then 166Mhz.
JasonClark - Wednesday, July 28, 2004 - link
felix,No where in the comparison did we say the performance comparison was CF vs .NET, well aware that there are some differences in page layout along with architecture. The comparison was the old site to the new site. For us to do a fair comparions the code would have to be quite similar, and even then it is still not apples to apples. Overall, platform, css and a re-architecture of the site brought us some large gains (as illustrated).
Yes we had already invested in licensing, but with some JRUN related issues that went unresolved, and the cost of future licensing .NET was a better choice for us.
RAD is great, but honestly with Visual Studio 2003, the argument isn't as great as you might think.
fbaum - Wednesday, July 28, 2004 - link
Hey ho...
Just noted that your interface from pure HTML to CSS doesn't really result in an apples to apples comparison of cf to vb.NET performance. No matter how you slice it, larger pages = slower rendering site. Taking down pages from 130K down to around 45K in HTML would yield huge performance gains without changing your app server, and wouldn't have required you to redevelop code, just presentation.
Got to agree, the VS IDE is slick. But I'm a RAD fan and a cf fan, I like having useful builtin functions to call upon, and .NET coding sometimes makes development that should be easy harder than it should be.
Why rewrite? Your server licenses are already bought, right?
Disclaimer: my email address ends in @macromedia.com, consider my comments accordingly
-Felix
kenny4269 - Wednesday, July 28, 2004 - link
Nice article, always interesting to read behind the scenes stuff.vda:
I work as a computer tech and have one thing to say about Windows. Outside of the security holes (which ANY OS can have), most Windows problems are caused by the user. Keeping up to date with patches, antivirus, not opening attachments, etc. all all things that MUST be done on ANY OS (open source or not). If you download BonziBuddy on your Windows PC or some poorly written Linux app, you can still have problems.
Don't get me wrong, I like Linux, but don't tell me it's perfect choice for everything just because it's open source.
JasonClark - Tuesday, July 27, 2004 - link
sonixDivx, the performance gain was a combination of our new layout (removing the news), and the architecture of our .NET version. But, the way we wrote the .NET site is similar to writing a ColdFusion app with CFC's as the backend, very similar. I'm not sure we'll do a comparison of the two, since the two sites are fairly different in layout and content.Cheers
JasonClark - Tuesday, July 27, 2004 - link
vda:What happens when a 18 year old genius has the source code to the operating system that your entire system architecture relies on? Then exploits a vulnerability in the source code (yes unix developers make mistakes, enough that redhat network sends out a constant stream of e-mail :)).
Then that exploit (which was easy to find as you had source code) takes your network down and costs you thousands every hour? There is another side to the open source argument, I'm not saying I don't believe in the movement, but your argument is one-sided.
BTW, the registry has never been corrupt for us, and that's what emergency backup disks are for btw, I certainly hope you don't run linux assuming it will never break :).
Phiro: we use Linux Virtual Server (vda are you reading this? LINUX virtual server :)). We use 5 web servers run the site, we could run off of 1 except on extremely busy days. We like redundancy.
vda - Tuesday, July 27, 2004 - link
> In regards to linux, our expertise lies in windows and it hasn't given us any grief what so everProblems do happen with every OS. Now, when Windows hangs on you, what do you do? When registry got corrupted, what do you do?
Nothing. You can do nothing because you have no source of the damn thing. You have next to nothing in regard to debugging facilities. No strace. No gdb. No kernel debugger. Disassembling stuff is
possible but illegal.
It does not matter for "yesterday's advanced user"
style admins and web designers, for they are Gates' target audience.
For me, it does matter.
vda - Tuesday, July 27, 2004 - link
> Code is as bloated as the developer makes it, experience, experience...Hehe. Our corporate Internet gateway have
lots of stuff. Tons of network utils
(nmap, mtr, tcpdump, iptables, traffic shaper)
and services (*transparent chaching web proxy*,
www, mail, secure ftp, DNS server, DNS cache).
It runs on Pentium 166 and 40Mb of RAM.
Try to install Windows NT on that.
--
vda
vda - Tuesday, July 27, 2004 - link
I once said this:'What happens when you read some doc and either it doesn't answer your question or is demonstrably wrong? In Linux, you say "Linux sucks" and go read the code. In Windows/Oracle/etc you say "Windows sucks" and start banging your head against the wall.'
To my surprize, it even propagated into some .signature files :)
Aaron - Tuesday, July 27, 2004 - link
A well written C++ .Net ISAPI filter on IIS beat out Apache in any confguration last I checked.Anandtech Team: Are you looking to add any additional developers to your team? I am a senior .net dev / usability designer / MS SQL Server DBA / systems architect. Work on your application would make a welcome addition to my resume. Email me if you are interested.
aaron.dezeeuw #at# gmail.com