Low End Mac Reader Specials
TypeStyler For Mac OS X is Now Shipping! Download The Free Fully Functional 60 Day Tryout at www.typestyler.com
OWC: Mercury On-The-Go FW800+USB2 up to 1.0TB. Bus Powered, no external power supply needed. Macworld Editors Choice, CNET Very Good Starting from $99.97, 500GB $159.99. Click here
Don't install Parallels to play poker online! Poker Mac will show you how
to download and install a native Mac poker application such as Full
Tilt Poker Mac.
Laptop Hardware Provided by TechRestore - Overnight Mac & iPod Repairs.
Compare products like desktop computers, apple laptops, apple macs, and LCD Monitors side by side! All the information and reviews to make the best purchasing decision for new mobile phones, sat nav systems, or MP3 players. The Ciao online shopping community makes searching products easy for you.
Mac Musings
The Dark Side of Testing
or How Microsoft Always Wins
Dan Knight - 1999.04.19 - Tip Jar
I have an issue of Byte magazine from many, many years ago on benchmarking. It's a topic the computer industry has followed with keen interest for decades.
Benchmarking allows you to compare two or more items (computers, printers, operating systems, hard drives, applications, etc.) using the same test.
Ideally, benchmarks are created in such a way that they are not biased toward one of the products being tested. Also ideally, the products being tested have not been tweaked to perform better when running the benchmark than when doing real world tasks.
A favorite benchmark for the Macintosh community was Byte magazine's BYTEmark. This was the famous test that showed that the G3 delivered over twice the integer performance of the Pentium II at the same clock speed.
Windows NT vs. Linux 5.2
Why in the world am I writing about Windows and Linux in Mac Musings?
Quite simply, because Linux is now getting the Macintosh treatment.
By that, I mean that some reviews and benchmarks are being deliberately handled so the Microsoft product is the hands down winner. You can do this in Photoshop, for instance, by picking just the right settings for certain filters. Change those settings just a bit, then the Mac wins. But most readers don't know that.
Benchmarks have to appear as objective, whether they are or not. That's probably part of the reason Mindcraft labels their comparison of Windows NT 4.0 and Red Hat Linux 5.2 as a white paper, a term that tends to imply objectivity.
In reality, Microsoft commissioned the study and probably would not have allowed publication of results that didn't show Windows NT as the winner.
Although the benchmarks themselves appear objective, the problem is that the systems tested are not comparable: the Windows NT server was tweaked for optimum performance, but the Linux server was not (see NT beats Linux...maybe). In fact, some have commented that Mindcraft chose the worst possible settings at times.
Another problem is that Mindcraft didn't test over the 10Mbps or 100Mbps wiring common in most offices.
How do I know? Quite simply because their peak performance summary shows Windows NT capable of a 286.7Mbps throughput - nearly three times what 100Base-T can handle. Of course, Red Hat Linux could also saturate a 100Base-T network, providing peak throughput of 114.6Mbps.
Unless you have Gigabit ethernet, either Windows NT 4 or Red Hat Linux 5.2 can serve up data faster then your network can handle it.
Mindcraft's performance analysis also shows that Linux and Windows NT are evenly matched with 1 to 32 clients. Unless you have more than 32 active clients (not just connected, but moving data back and forth with the server), a highly optimized Windows NT server and an unoptimized Linux 5.2 server provide comparable performance.
And things look a lot worse on the web side. The tweaked NT server provides about twice the performance of an unoptimized Linux server.
Of course, we also want to see how realistic the results are. With 288 active threads, Windows NT serves web content at 22.4MBps. Red Hat Linux peaks at 5.9MBps with 160 threads.
Now go and convert megabytes per second to megabits per second. Windows is serving up web pages at almost 180Mbps - twice the speed of 100Base-T and far faster than any commonly known internet connection. Fine for offices with Gigabit ethernet, but for those connecting from the outside world, so what?
Unoptimized Linux, meanwhile, trucks along at 47Mbps. That's half the bandwidth of 100Base-T, almost 5 times the bandwidth of 10Mbps ethernet, and far, far more than most internet connections are able to handle - even the fastest cable modems peak out at about 30Mbps.
Truth in Testing
Be sure to read the fine print at the end of the white paper. It provides all sorts of revealing information.
Not only did Microsoft fund this study, it also helped Mindcraft set up the Windows server for optimum performance. As already mentioned, the Linux server was not optimized at all.
Further, what was tested wasn't a simple single processor system with a regular hard drive, but a four processor system using RAID drives. Windows NT has been optimized for such an environment, while Linux has not:
- "We started the tests using Red Hat Linux 5.2 but had to upgrade it to the Linux 2.2.2 kernel because its Linux 2.0.36 kernel does not support hardware RAID controllers and SMP at the same time." (In other words, RAID support and multiprocessing [SMP] support are very new features of Linux.)
The big difference is that Microsoft Windows NT is a commercial product with a huge corporation to back it up. When they are funding a report, they will provide all the technical assistance needed for Windows to perform at its best.
Linux, a freeware operating system, has no such infrastructure. To optimize Linux, you have to learn it, not simply depend upon someone else to tweak the settings for you. Comments in the Mindcraft white paper show they were not willing to invest the required time and effort into learning Linux. Instead, they complained that they couldn't get the help they wanted from the volunteer Linux community, although the kernel source code did contain hints on tuning and configuring Linux.
Conclusion
You've seen it before with the Mac. You'll see it again with Mac OS X Server, despite the fact is has already been called the fastest web server under $5,000.
With Linux running on a multitude of hardware platforms and Intel working with Apple to port OS X to Merced, Microsoft is running scared. Even if the Department of Justice doesn't break up the company, their stranglehold of the personal computer industry is being challenged by two powerful upstarts.
While the Mac OS and Linux combined might only be 15% or so of the installed base (the Mac OS is about 12%), that number continues to grow. And it can only grow at the expense of Windows.
Between the evangelistic Mac users preparing to leap into the Unix-derived Mac OS X and the technical gurus adopting Linux in the nerve centers of businesses, we can confidently predict that the Windows market share and installed base will slowly erode.
And when we do get some objective benchmarks showing tuned Linux, Mac OS X, and Windows NT performance, the Mindcraft report will be clearly seen for what it is: Microsoft propaganda in the guise of an objective study.
Further Reading
- NT faster than Linux in tests, Slashdot, 4/13
- Web and File Server Comparison: Microsoft Windows NT Server 4.0 and Red Hat Linux 5.2 upgraded to the Linux 2.2.2 Kernel, Mindcraft, 4/13
- NT
beats Linux...maybe, ZDNet, 4/15
A study commissioned by Microsoft found that, "Highly tuned NT beats a barely tuned, if that, Linux. But, there's more to it...."
Dan Knight has been using Macs since 1986, sold Macs for several years, supported them for many more years, and has been publishing Low End Mac since April 1997. If you find Dan's articles helpful, please consider making a donation to his tip jar.
Recent Mac Musings
- IDE Is Dead; Long Live SATA!, 11.04. SATA has displaced parallel ATA. While IDE hard drives haven't disappeared, the best deals are in SATA hard drives.
- The Future of Personal Computing: Personal Servers and Low Cost Portables, 11.02. With WiFi everywhere, virtual network computing, and remote access, your iPhone, iTouch, iTablet, or MacBook Air becomes a gateway to your home or office computer.
- The Late 2009 Mac mini Value Equation, 10.21. We called the Mac mini 'the best value in desktop Macs' two months ago, and the refreshed Mac mini only improves that value.
- The Late 2009 MacBook Value Equation, 10.21. The redesigned consumer MacBook uses unibody construction, gains LED backlighting and battery life, but loses FireWire.
- More in the Mac Musings index.
Links for the Day
- Mac of the Day: 17" iMac G4/800 MHz, July 2002 - The iMac 'grows up' with a 17" 1440 x 900 display.
- Group of the Day: LisaList supports Lisa users.
- November 8 in LEM history: 99: OS 9: I think I like it - 01: The simplified Mac life - Soured on Windows - Flea market Mac - 02: Little room for improvement in new 'Books - Combo drive upgrade for iceBooks - 04: Re-Porter - 05: Fix the old iMac or buy a Mac mini? - Apple's Copland project - 06: MacBook Core 2 - MacBook value equation - Cheap is as cheap does - 07: Problems with Classic mode in Tiger - The G4 Power Mac that won't run Leopard
- Support Low End Mac
Recent Content on Low End Mac
- Quad-Core CPU Makes Sense in MacBook Pro, OS X 10.6 Causing Overheating, Overseas Power, and More, The 'Book Review, 11.06. Also Late 2009 MacBook reviewed, how to add RAM to new MacBook, 18.4in Acer notebook used Intel i7, and SanDisk SSD chosen for Sony VAIO X.
- Dumping Macs for Google Apps, SSD in iMac, Late 2009 iMac Performance Problems, and More, Mac News Review, 11.06. /newsrev/09mnr/1106.html
- WiFi Paranoia, iMac-O-Lantern, Magic Mouse Does Click, Free Clipboard Managers, and More, Charles W. Moore, Miscellaneous Ramblings, 11.05. Also strange time stamps, problem with ColorIt on Intel Mac, and the story behind OS X 10.5.4 install discs.
- QuickTime X in Snow Leopard Imports, Trims, and Publishes Video Quickly and Easily, Alan Zisman, Zis Mac, 11.04. The long, slow process of importing video into iMovie to edit it, then render it to another format, is history as QuickTime X does that much more quickly.
- More links in our archive.
Recent Deals
- Best Mac Pro Deals, 11.03. Used 2.66 GHz 4-core, $1,300; 3.0 8-core. $2,299; refurb 2.66 4-core Nehalem, $2,149; 2.93, $2,549; 2.26 8-core, $2,799; 2.93, $4,999.
- Best iPhone Deals, 11.03. New 8 GB iPhone 3G, $$99; refurb 16 GB 3GS, $149; new, $199; 32 GB, $299.
- Best 12" PowerBook G4 Deals, 11.03. Used 867 MHz SperDrive, $348; 1 GHz, $499; 1.33 Combo, $298; SD, $559; 1.5 Combo, $448; SuperDrive, $589.
- Best Power Mac G3 and PCI Video Card Deals, 11.02. Used beige 300 MHz, $25; G4/366, $49; blue & white 350, $80; 400, $90; 450, $105; PCI video cards from $15; shipping additional.
- Best Power Mac G4 and AGP Video Card Deals, 11.02. Used 400 MHz, $50; 733 MHz, $69; 933 MHz, $209; 1.25 GHz dual, $299.
- Best 15" MacBook Pro Deals, 11.02. Used 2.0 GHz, $800; 2.2, $900; 2.4, $1,000; refurb 2.53, $1,449; 2.66, $1,699; 2.8, $1,949; 3.06, $2,169; new 2.53, $1,579; 2.66, $1,799; more.
- Best Mac mini Deals, 10.30. Used 1.33 GHz G4 mini, $379; 1.42, $389; 1.5, $419; 1.83 GHz Core Duo, $350; Core 2, $439; new 2.26 GHz nVidia, $580; 2.53 GHz, $770; Server, $990.
- Best G4 iBook Deals, 10.30. Used 12" 1.07 GHz Combo, $225; 1.33 GHz, $298; 14" 1 GHz, $349; 1.33 GHz, $398; 1.42 GHz SuperDrive, $498.
- Best Classic Mac OS Deals, 10.30. System 6.0.8 floppies, $10; 7.1, $12; 7.5, $20; 7.5 CD, $4; 7.6 $13; 8.1, $11; 8.5, $20; 8.6, $90; 9.0, $20; 9.2.2, $30.
- More deals in our archive.
About LEM | Support | Usage | Privacy | Contacts
Navigation
Used Mac Dealers
Apple History
Video Cards
Email Lists
Favorite Sites
MacSurfer
MacMinute
MacInTouch
MyAppleMenu
InfoMac
Macs Only!
The Mac Observer
Accelerate Your Mac
RetroMacCast
PB Central
MacWindows
The Vintage Mac
Museum
DealMac
DealsOnTheWeb
Mac2Sell
ramseeker
Mac Driver Museum
JAG's House
System
6 Heaven
System 7 Today
the pickle's Low-End
Mac FAQ
Abandonware
Petition
Mac vs. PC Info
Affiliates
The Apple
Store
Mac
Connection
B&H
MacMall
TechRestore
ExperCom
Crucial
Memory
batteries.com
Advertise
MacMinute
MacInTouch
MyAppleMenu
InfoMac
Macs Only!
The Mac Observer
Accelerate Your Mac
RetroMacCast
PB Central
MacWindows
The Vintage Mac
Museum
DealMac
DealsOnTheWeb
Mac2Sell
ramseeker
Mac Driver Museum
JAG's House
System 6 Heaven
System 7 Today
the pickle's Low-End
Mac FAQ
Abandonware
Petition
Mac vs. PC Info
Mac Connection
B&H
MacMall
TechRestore
ExperCom
Crucial Memory
batteries.com
