PC Magazine May 31, 1994 v13 n10 p125(23)
------------------------------
(Includes related articles on Editors' Choice winner, highlights, suitability to task, price/performance index, microprocessors, performance tests, and summary of features) (overview of 14 evaluations of RISC workstations)
Seymour, Jim
Yegyazarian, Anush
Flanagan, William P.
10P3T1.XLS How the chips stack up.
10P3T2.XLS Summary of Features: RISC workstations.
IBM's $15,525 RISC System/6000 Powerstation 25T wins the Editors' Choice designation over 13 other workstations evaluated. This system, the first workstation to employ the 66-MHz PowerPC 601 chip, proves itself to be the most versatile, delivering a good balance of floating-point and integer math operations. The three Sun SPARC-based Unix workstations offer excellent performance in AutoCAD, and Silicon Graphics' Unix-based Indy excels in graphics capability. Carrera's Hercules 200 NT turns in a top performance under floating-point-intensive applications, and the Pentium-based systems deliver good integer capability but fall short on floating-point tasks. The NT-based systems cannot compare to the Unix systems because of the paucity of accompanying software.
PC Magazine takes a look at whether the worlds of RISC workstations and PCs are converging.
We know: You're tempted. After all, who wouldn't be seduced by stories about 100-MHz and 200-MHz RISC chips? You hear talk about RISC being the future of desktop computing. You see newspaper stories about hotshot traders on Wall Street relying on their RISC machines to move millions of dollars. You watch in amazement as the ballroom scene in Beauty and the Beast swirls by, knowing that the impossibly difficult animation was done on a RISC workstation.
And you wonder: Is it time? Time to bail out on that pedestrian Intel-based box on your desk and finally move up to what the real computer jocks keep on their desks: RISC workstations. We're tempted, too. And we wonder.
For years, the answer for the general computing public has been simple enough: Not yet. Not yet, because those workstations cost too much, they ran only Unix, and there just weren't enough mainstream applications available--or if there were, they were never a match for the hot DOS and Windows programs.
But things have gotten complicated since we last took up the RISC question ("The New PCs," June 15, 1993). First, RISC workstation prices have fallen 50 percent and more over the past two years--to a level where many are competitive with the prices of high-end Intel- based x86 boxes. Second, while general applications running under any of the various flavors of Unix are still thin on the ground, they do exist, and many of them have familiar names: CorelDRAW, Lotus 1-2-3, WordPerfect, AutoCAD, and others. The problem is that few of these applications run under all varieties of Unix, and those that do cover most of the bases are older versions of the software.
Then there's Microsoft Windows NT. Microsoft's advanced operating system runs on important RISC architectures such as MIPS' R4000 family or Digital Equipment Corp.'s Alpha series--and soon, Hewlett-Packard's PA-RISC chips and the PowerPC. Now, at least in theory, you can have your RISC and eat it too, so to speak. You don't have to confine yourself to the world of Unix applications, so you don't need to tackle Unix's near-vertical learning curve and the grief of setting up and working under Unix.
Is considering purchasing a RISC workstation finally a no-brainer move? The answer is still . . . Not Yet.
We rounded up 11 of the most popular RISC workstations and divided them into two test groups, based on operating system. Those that ran Unix include the DEC 3000 Model 300X AXP Workstation, Hewlett-Packard's HP 9000 Model 712/80i, IBM's RISC System/6000 POWERstation 25T, the Silicon Graphics Indy, and three systems that use Sun's SPARC architecture: the two-processor Integrix SWS10/512 50MHz SPARCstation 10, Sun's two-processor SPARCstation 20 Model 502, and Tatung's Super COMPstation 10 Model 51. The group of units that ran Microsoft Windows NT includes Carrera's Hercules 200, the AcerFormula 4400, the DeskStation Tyne v4633x, and the NeTpower NeTstation 300. (For a complete rundown of the CPUs involved, see the sidebar "How the Chips Stack Up.")
To place these systems in context, we also included two well-regarded Pentium PCs: the Dell OmniPlex 566 and ALR Evolution V/60, each of which received an Editors' Choice award in an earlier story. Both units ran the x86 version of Windows NT; the ALR Evolution ran under UnixWare and SCO Unix as well. And because Digital Equipment Corp. was in the process of revamping its AXP line of Alpha-based NT workstations, we tested a 150-MHz NT system for comparison only--the DECpc AXP 150. To facilitate the comparison of same-CPU systems, the performance results and reviews are arranged in alphabetical order according to the processor manufacturer.
The minimum hardware requirements for systems included in this roundup were suitably demanding: 32MB of system memory, a 400MB hard disk, a 15-inch display, accelerated video adapters supporting at least 800-by-600 resolution, a CD-ROM drive, and a keyboard and mouse. All this hardware had to be available for a system cost of under $22,000.
We had two questions on our minds while testing these systems. First, can a RISC workstation offer the ease of setup and range of software we've taken for granted in the PCs we use every day? And second, which systems best fulfill the role of a good all-around engineering workstation?
This kind of cross-platform testing is always complicated by the lack of truly equivalent benchmark tests. For example, instead of using our familiar DOS and Windows benchmark tests, we had to use a matrix of seven tests that measure performance in different areas. But all tests were not available or able to run on all systems. The SPECint92 and SPECfp92 scores, for instance, provide an excellent indication of a system's computational ability but run only on Unix workstations at this time. We also ran several cross-platform applications--CorelDRAW, FrameMaker, PV-Wave, Picture Publisher, Pro Engineer, and AutoCAD; but again, not all the systems ran all the software.
Be warned: Just because applications have the same name and are packaged in a similar box, this hardly means that they are the same or that their publishers have devoted the same effort to fine-tuning each release for its intended platform. Though several prominent software vendors in the DOS/Windows world have Unix releases of some of their products, the Unix versions rarely receive as much care in terms of performance-tuning as the much more profitable DOS and Windows versions.
Even within the Unix world, a given application is rarely available to all users. For example, WordPerfect runs under SCO Unix, but not for the DEC system, and only in an older version for the Silicon Graphics Indy. The curious result is that a stellar application first created in the Unix-dominated minicomputer world cannot run under all the flavors of Unix. As our tests show, performance varies from machine to machine among the various Unix versions and Windows NT.
In many ways, RISC systems and the x86 CISC systems we're more accustomed to using are very similar. With the exception of chips in HP's PA-RISC family, both kinds of CPUs rely on both internal caches and external caches to eliminate the data bottleneck caused by the fact that the CPU is operating much more quickly than the system bus. And the two chip-design philosophies have borrowed freely from one another; there are many RISC-like features, such as pipelining, in current CISC designs. The other subsystems are also similar, though workstations typically demand more resources in every category. For instance, workstations historically have pushed the envelope of graphics speed and quality; they also have sported large color monitors in addition to high-performance, high-resolution display adapters--and as far back as a decade ago, when most PC owners were considering IBM's then-new color EGA video standard that displayed 16 colors at 640-by-350.
Because workstations routinely surrender 300MB or more of their hard disks to the Unix operating system, hard disks with capacities of a half-gigabyte or more are most common. SCSI-2 is the dominant hard disk interface, and memory demands are heavy, with 16MB a bare minimum of RAM for running current versions of Unix; 32MB or more is much more common. Finally, Unix workstations are almost always networked, so built-in Ethernet connections are the norm. Unix itself has tried to put on a prettier, friendlier face in recent years; Sun's version of Unix, Solaris, has probably gone furthest in developing an attractive GUI that will feel familiar to any Windows user.
But the biggest difference between today's RISC workstations and today's high-end PCs is the possibility that a RISC workstation will have two (or more) CPUs installed. Symmetrical Multiprocessing, or SMP, is a powerful and highly cost-effective way to increase system performance. Engineers estimate that the best-designed scalable RISC chips add about 80 percent more raw processing power with each chip, at a total system price increase of only about 15 percent. Multiprocessing is trickling down from the server level of engineering and just coming to x86 machines--mainly on Windows NT systems. It is already a fixture in the Unix/RISC world. Two of our test machines--the Integrix SWS10/512 and Sun's SPARCstation--had two RISC processors each.
The bottom line is that we don't see RISC workstations as threats to midrange and entry-level PCs. Clearly, the lure of the RISC world is speed, speed, and more speed--which translates into competition for high-end x86 PCs. In designing, building, benchmarking, buying, and using computers, it's important to remember that there is speed, and then there is speed. The biggest advantage of RISC workstations over CISC-based x86 designs has always been in floating-point math performance rather than in integer-math calculations. Since few DOS/Windows applications rely on floating-point math, and current Pentium designs offer parity in terms of integer operations, the performance advantage of RISC machines' floating-point unit (FPU) doesn't deliver much improvement on general business applications.
Does that matter to you? If you use technical, graphical, or scientific software that is highly dependent on FPU operations, Pentiums still may not be fast enough: You can get substantially faster performance from many RISC workstations for just slightly more than what you'd pay for an all-out 100-MHz Pentium PC.
Chances are that if you need what a RISC machine does best you probably already have one. The world of Unix/RISC workstations is built around specialized software. And RISC users have plenty of incentive to learn Unix's esoterica, to cope with the relatively difficult setup of Unix/RISC machines, and to pay the incremental cost of a Unix/RISC box over a high-end x86-DOS/Windows machine.
Conversely, if you need what an x86 PC does best you probably already have one. There are simply no Unix analogues to such high- performance Windows software as Quattro Pro or Excel. The PC business, at whatever processor-power level, has been built around the idea of ever-faster horizontal applications.
Certainly, Windows NT may change this. With the availability this summer of the next release of NT--code-named Daytona--NT will shake off some of its first-release lethargy and begin to attract speedy NT-specific applications. Some of those will be NT ports of scientific and technical applications that are now successful in the Unix/RISC world, but many more are likely to be rewrites of existing Windows business applications.
Since the majority of computer users want access to Windows apps, makers of Unix/RISC workstations have tried to bolster their push into the mainstream marketplace by fostering the development of two software-based emulation schemes--Sun's Wabi and Insignia Solutions' SoftWindows--to allow some Windows 3.1 applications to run under Unix. But both extract a price. Wabi supports just 13 Windows 3.1 applications and runs them slowly when compared with native Windows. Far from a stable product, Wabi gave us a lot of trouble on our tests, and we remain skeptical about its general usefulness. SoftWindows, though at the time of our testing still in beta form, installed and performed much more reliably--but still ran Windows apps very slowly.
As our testing shows, running emulation reduces the performance of an expensive RISC workstation to that of a sub-$1,000 486SX/25 PC-- if that. As with the dog who sang, the most remarkable thing about both Wabi and SoftWindows may be that they can do what they do at all, rather than that they can do it very well.
There is another way to bring Windows to RISC workstations: Do it via hardware. Apple's new Power Macs support the existing body of Mac software designed to run on Motorola 68k chips, as well as new software written especially for PowerPC CPUs; this is achieved by encoding 68k emulation in ROM. Similarly, the expected PowerPC 615 chip, coming in about a year from IBM, will probably be able to perform both x86 and PowerPC work.
The key to success in this market is to seek the best economy of scale. This year, Intel is expected to ship over 40 million x86 CPUs, of which more than 6 million will be Pentiums of various speeds. The total RISC market is much, much smaller: Less than 300,000 units were shipped worldwide in 1993. The clear winner among workstation makers is Sun Microsystems, which has garnered 44 percent of the market; Hewlett-Packard follows with 21 percent, IBM with 13 percent, and several other vendors with less than 10 percent apiece.
Along with the overwhelming numerical advantage that Intel has in microprocessor volume, there is a pricing paradox in fast CPUs: Pentiums sell for about twice the price of RISC chips, but RISC chips offer up to two times the raw processing power of Pentiums. Intel's fatter profit margins on x86 chips are the envy of the RISC chip producers; Intel enjoys that price premium because of its x86 chips' compatibility with tens of thousands of DOS and Windows applications. But this scenario is going to change quickly as the prices of 60-MHz and 66-MHz Pentium chips drop; Intel will be shipping in bulk the higher-performing and more profitable 90-MHz and 100-MHz Pentiums by the time you read this (even faster Pentiums may be available by the end of the year).
Meanwhile, the PowerPC alliance will be trying to turn the economy-of-scale argument in their favor. By contrast with the roughly 250,000-unit worldwide RISC market last year, the PowerPC group alone may ship 1.4 million chips this year--boosted by Apple's rapid conversion to PowerPC. In addition to desktop computers, we'll see PowerPC processors in automotive engine-management computers and in a new generation of TV set-top decoders being developed for the much-talked-about 500-channel future. It seems odd to look to these ancillary non-desktop computer markets for the volume needed to drive PowerPC prices down, but clearly computer users are going to be major if indirect beneficiaries of the move to more powerful embedded processors.
But the CPU accounts for only about one-fifth of a system's cost, at most. On the whole, RISC workstations are more expensive than PCs-- the result of proprietary design and a lack of direct competition.
If your world revolves around Pro Engineer, AutoCAD, or any of the other specialized workstation applications, you might be wondering which workstation to buy. Because of limited availability of applications under the many different varieties of Unix, it is certainly a lot harder to choose a workstation than it is to choose a PC. The first step must be to look at what flavor of Unix a system supports and what applications are available.
As you read the reviews that follow, take a look at the performance charts that begin on page 150 for an indication of how well each of these systems handled a specific set of tasks. The low-level benchmark tests, such as the SPECmarks, Khornerstone, and Graphstone, tell part of the story. Look also at the results of our applications tests, and you'll get a good picture of each system's capabilities. Several workstations excelled in specific performance areas that their manufacturers have been able to exploit over the years. IBM's PowerPC-based RS/6000 is the clear leader in graphics, but the two-processor SPARCstation, from Sun, led the way on our test of AutoCAD-based 2-D and 3-D graphics. The processor with the most raw computational power is Digital's Alpha chip, which graced both the DEC 3000 (in the Unix group) and the Hercules 200 (in the Windows NT group).
Based on the virtues of SCO Unix and UnixWare, the available applications and performance comparisons, the clear conclusion is that x86 systems cannot compete in the technical areas, where RISC- based workstations currently excel. While the ALR Evolution trailed the pack in Unix testing, both it and the Dell OmniPlex ranked in the middle of the pack under Windows NT. Pentium's obvious advantage came into play when we tested native Windows Excel code on the Pentium machines versus Excel under the Windows emulations provided on the RISC machines. As expected, the RISC systems plodded, while Pentium soared.
It's clear that if you want to run x86 applications--the DOS and Windows business applications that are the heart of the software market--your best choice today for price and performance is an x86- based computer. The whizzy-speed claims of the RISC workstation vendors pale next to the machines' inability to run popular applications at anywhere-near-acceptable levels of performance.
But with PowerPC CPUs with hardware x86 emulation coming, and with the growing maturity of Windows NT, that could change. Anyone who's worked with a Pentium machine may wonder just why we need this much speed. All but the most processor-intensive programs run very quickly on a Pentium PC. For almost all of today's Windows and DOS apps, we simply don't need RISC speed. But that will change. Coming up very soon are complex speech-recognition programs, which will allow us, for example, to dictate letters to our PCs--in a variety of languages. Speech recognition, especially in anything like real time, takes huge amounts of processing power. And graphics applications are getting bigger and much more video subsystem intensive. Anyone who's watched a Pentium PC slowly redraw an Adobe Photoshop image will tell you that today's fastest PCs aren't nearly fast enough.
There's also desktop teleconferencing. Intel CEO Andrew Grove jokes that Intel is investing heavily in development of desktop conferencing software (such as the company's new ProShare package) because "it absorbs CPU cycles like crazy," pushing Intel's customers toward ever-faster chips. Desktop conferencing has already overwhelmed standard phone lines--it demands fast ISDN lines--and if you want full-screen images, at standard 30-frame-per-second TV speed, you're going to buy a faster CPU, too. Other intersections with telephony--the growing field of computer telephone integration, or CTI--will demand much more power as well. Both Microsoft and Novell, for example, are telling customers to get ready to dump their on-site PBX switchboards before long, replacing them with telephony servers on their LANs.
Maybe some of those apps will demand the power of RISC chips. And maybe the developments in Windows NT, the PowerPC alliance, and other corners of the computing world will, over the next two years or so, change the equation. But for now, the question of whether it's time to move up to RISC has two answers. If your needs are high-end graphics and computational ability, maybe a RISC workstation is the right choice. But for the general business app user, the answer remains unchanged: Not yet.
The phrase "East is East, and West is West, and never the twain shall meet" goes a long way towards explaining the polarization present in desktop computing. PCs offer the flexibility to run thousands of diverse applications, while workstations run very specialized programs but run them well. A high-end PC can't match the processing ability of a RISC workstation, but none of the workstations we looked at can touch a PC in ease of setup and use.
Within the Unix and Windows NT workstation worlds, all the
systems we tested had positive attributes:
* The three Sun SPARC-powered Unix workstations were finely tuned for excellent
results in AutoCAD.
* Carrera's Hercules 200 NT system stood out under floating-point-intensive
applications like Pro Engineer.
* Silicon Graphics' Unix-based Indy impressed in terms of graphics capability.
* The DeskStation Tyne v4633x, the AcerFormula
4400, and the NeTpower NeTstation 300--powered by MIPS CPUs--all ran NT
well.
* Hewlett-Packard's HP 9000 Model 712/80i Unix system did well on applications
like FrameMaker and Pro Engineer.
* Digital's DEC 3000 Model 300 AXP, powered by Digital's 200-MHz Alpha CPU,
showed its processor prowess on the Khornerstone and Dhrystone tests.
* The Pentium systems--ALR and Dell in NT and ALR in Unix--provide the integer
capability to run head-to-head with the best workstations but can't keep up on
floating-point tasks.
The IBM RISC System/6000 POWERstation 25T, however, which stood out as the most versatile of the systems we looked at, deserves an Editors' Choice. The $15,525 RS/6000 is the first workstation to use the 66-MHz PowerPC 601 processor. Systems of higher and lower capability and price will augment the line over the next few years. The chip itself provides a good balance between integer math and floating-point operations. Clearly focused on graphics, the RS/6000 led on the Xmark and Graphstone tests by 30 percent. It had average numbers in each application test, where the results were closely spaced.
Because of the lack of software we felt that none of the NT systems were on a par with the Unix systems.
IF ULTRA-FAST CPUs impress you, you may have a RISC workstation in your future. But be warned: Compatibility with the gargantuan library of DOS and Windows applications can be had only through emulation software that saps much of a system's power.
WORKSTATIONS ARE workstations and desktop PCs are desktop PCs. Though each type of system has its own audience and adherents, desktop PCs will continue to be the systems of choice for most users.
THE PREDOMINANT OPERATING system of PCs and workstations have their individual drawbacks. Unix remains hard to learn, Windows NT is still in its infancy, and Windows 3.1 lacks the power and flexibility of a workstation OS.
SAME NAME BUT NOT SAME performance can be expected when you cross the line between DOS/Windows applications and workstation applications. PC DOS/Windows applications boast a larger audience, so the software publishers can expend extra efforts at optimization; Unix and Windows NT applications are few, and they are rarely fully optimized.
Computational ability EXCELLENT
Expandability GOOD
Engineering/scientific/CAD FAIR
Graphics/publishing POOR
These 14 workstations--6 running Windows NT and 8 running Unix--offered varying levels of performance in all Suitability to Task categories.
The Computational ability score indicates each system's CPU performance. This score is based on results of the Stanford test suite, which measures each system's multithreading ability by running up to 16 concurrent processes; the single-precision and double-precision Whetstone tests, which simulate floating-point programs; and the Dhrystone, a CPU-intensive test that simulates an applications development environment by using integer-based calculations.
When rating Unix-based workstations, we also take into account the results of our SPECint92 and SPECfp92 tests, which are standardized tests of a system's processor.
Expandability refers to a system's potential for growth. Considerations include the maximum number of processors supported, the maximum amount of external cache installable, hard disk options, maximum amount of RAM installable, available drive bays and bus slots, number of ports, power-supply wattage, and the number of power connectors.
The Engineering/scientific/CAD score reflects a system's ability to run CAD/visualization packages. This score is based on the results of applications tests using Pro Engineer and PV-Wave. When rating Unix-based systems, we also take into account each workstation's performance in AutoCAD.
A good Graphics/publishing score reflects more than just a good graphics subsystem; good floating-point-unit and hard disk performance are also key. We consider the results of the Graphstone--a graphical benchmark test--and the Khornerstone, which measures each system's graphics, hard disk, and CPU performance in an application environment. For Windows NT systems, we considered the results of tests on which we time operations of common graphics-related tasks using Picture Publisher for NT. For the Unix-based systems, scores are also based on the results of tests run using CorelDRAW and FrameMaker.
Workstations are often designed to do a few things expertly, sacrificing performance in other areas. A workstation may be a bargain if its area of expertise matches your needs, but its overall performance may pull it down in terms of our rating. Our Editors' Choice winner--IBM's RISC System/6000 POWERstation 25T--was the best all-around workstation.
Thanks to prices that fall below $10,000, the tested Windows NT workstations generally offer more bang for the buck than the tested Unix workstations, which range in price from about $13,000 to $21,195.
Carrera's Hercules 200 and the DeskStation Tyne v4633x are the performance bright spots among the NT workstations. The non-Intel-based Unix workstations are pricier than the NT systems, but the performance is generally better. Consider the DEC 3000 Model 300X AXP Workstation, the HP 9000 Model 712/80i, and IBM's RS/6000 if you are looking for a good value. The NT-based ALR Evolution V/60 did better on our price/performance index primarily because of superior graphics abilities.
Seventy-five percent of each system's performance/feature score was based on benchmark test scores. The system's features determined the remaining 25 percent. You can find more data in the Hardware Library on ZiffNet under 10BANG.XLS.
Analysis written by Tami D. Peterson
The overall performance of RISC workstations was not dominated by any one phase but is an amalgam of low-level testing and applications performance.
What the Numbers Mean
We ran two SPECmark benchmark tests: SPECint92, a measure of integer operations, and SPECfp92, a measure of floating-point operations. These tests run only under Unix. The three workstations in this lineup with Sun SuperSPARC CPUs--the Integrix SWS10/512 50MHz SPARCstation 10, Tatung's Super COMPstation 10 Model 51, and Sun's SPARCstation 20 Model 502--led the pack on the SPECint92 test. The two processors carried by the Integrix and Sun entries can partially explain the impressive performance of these systems. To see whether compilers would make a difference, we tested Sun's unit with both the preinstalled SunPro compiler and the Apogee compilers that came with the Integrix and Tatung units. While the Apogee compilers proved more capable, they did not improve performance dramatically, pointing to system design as a final differentiator.
The Silicon Graphics Indy (with a 100-MHz R4000SC CPU) and Digital's DEC 3000 Model 300X AXP Workstation (with a 175-MHz Alpha CPU) were at the bottom of the heap on the SPECint92 test--further proof that there are many other factors to consider besides CPU speed when looking at workstations.
On the SPECfp92 test, a similar scenario emerged among the units with SunSPARC architectures: The Integrix and Tatung units both outperformed Sun's SPARCstation. But contrary to what it achieved on the SPECint92 test, the DEC 3000 was number one on the SPECfp92 test by a wide margin. Silicon Graphics' Indy showed an improvement; it outperformed the ALR Evolution V/60 under both SCO Unix and Novell UnixWare. UnixWare's FORTRAN compiler appears better tuned for the Pentium than SCO's compiler does: With UnixWare, the ALR Evolution topped its own performance under SCO Unix by more than 20 percent.
The Unix-only Xmark tests produce an overall graphics score for each system. IBM's RISC System/6000 POWERstation 25T (with a 66-MHz PowerPC 601 CPU) turned in the highest score. This unit has a hot video subsystem based on IBM's GXT 150 graphics controller, which helped the RS/6000 outperform Hewlett-Packard's HP 9000 Model 712/80i--itself an impressive graphics machine--by about 40 percent. The ATI Mach32 card in the ALR Evolution (under either SCO Unix or UnixWare) did not help this system on our Xmark tests, thanks to the lack of fully optimized drivers.
The Graphstone tests, which execute a variety of drawings, are a gauge of video performance. As on the Xmark tests, IBM's RS/6000 took first place, and the HP 9000 took second. The top-performing NT workstation was the DeskStation Tyne v4633x, which uses an Appian VL-Bus card with 2MB of VRAM.
The Khornerstone tests measure each system's CPU, floating-point-unit (FPU), and disk performance. With the Alpha's exceptional floating-point ability, the DEC 3000 captured the top spot on these tests. The ALR Evolution, under UnixWare, overcame a low disk score to take second place. Under SCO Unix, the ALR Evolution could not overcome this hurdle: It finished dead last. The Alpha-based Carrera Hercules 200 and DEpc AXP 150, running Windows NT, could not perform these tests because of a lack of FORTRAN compilers.
The Dhrystone tests are CPU-intensive tests that measure programming application performance. Based on these tests, the Alpha platform is a programmer's dream: The DEC 3000 captured the number-one Dhrystone score, followed by Carrera's Hercules NT workstation. The ALR Evolution received a high score under both SCO Unix and UnixWare. Apparently, the C compilers in both flavors of Unix are about equally optimized for the Pentium.
The single-and double-precision Whetstone tests simulate FPU-intensive applications. The DEC 3000 ranked number one on both Whetstone tests. The HP 9000, the ALR Evolution, and Sun's SPARCstation also showed potential for running a variety of 32-bit FPU-intensive applications. The single-precision version was the only test on which Sun's workstation far outpaced its Integrix and Tatung clones. The Sun compilers made the difference.
On the double-precision Whetstone test, Sun's compilers did not provide the SPARCstation with the same performance edge. Most other workstations' scores on this 64-bit test were more or less in line with their 32-bit performance. The DEC 3000 maintained its lead, and the ALR Evolution (in both Unix configurations) turned in strong scores. The Alpha-based Carrera Hercules and DECpc AXP 150 could not run these tests because no FORTRAN compilers were available.
The Stanford test suite exercises processor-level functions concurrently, demonstrating a workstation's ability to manage multithreaded applications.
All the NT workstations we tested contain CPUs with built-in floating-point units that can operate at full CPU speed. This explains the overwhelming performance lead of Carrera's Alpha-based 200-MHz Hercules 200 on the Stanford Floating-Point tests. In view of the fact that the three MIPS-based machines--the AcerFormula 4400, DeskStation Tyne v4633x, and NeTpower NeTstation 300--boast internal clock speeds of 133 MHz or higher, the ALR Evolution, with its 66-MHz Pentium, did an impressive job of keeping up the pace.
In our Unix group, Digital's DEC 3000 Model 300X AXP workstation did well here. But as more processes were added, it was outperformed by the Integrix SWS10/512 50MHz SPARCstation 10 and by Sun's SPARCstation 20 Model 502, both of which pack two 50-MHz SuperSPARC chips. Half as fast as the Integrix were the ALR Evolution V/60 (running UnixWare), Hewlett-Packard's HP 9000 Model 712/80i, IBM's RISC System/6000 POWERstation 25T, and the Silicon Graphics Indy.
On the Stanford Integer tests, the 133-MHz MIPS-based DeskStation Tyne and the 200-MHz Alpha-based Carrera Hercules showed strong integer performance--especially with multiple processes. On the Unix side, most of the workstations maintained fairly equal integer performance when handling up to eight concurrent processes, but Tatung's Super COMPstation 10 Model 51, with one 50-MHz SuperSPARC chip, had difficulty handling more than one process. The Integrix SWS10/512 and Sun's SPARCstation did well at balancing higher loads.
Within both workstation classes (NT and Unix), system performance on the Stanford Recursion tests sorted out roughly by CPU capacity and speed. Among the NT systems, the MIPS-based workstations turned in the best scores, along with the Alpha-based systems, grouping together at every process load. The 60-MHz Pentium-based ALR Evolution brought up the rear. Among the Unix machines, the dual- SuperSPARC-powered workstations by Integrix and Sun led the pack. The Dell OmniPlex 566, with its 66-MHz Pentium, could not run any of the Stanford tests; this had more to do with the unit's system design at the time of testing than with the Pentium itself.
The Windows NT workstations in this roundup ran the standard Windows version of Microsoft's Excel spreadsheet program at a higher video resolution than workstations with either SoftWindows or Wabi installed because of the limitations of those two Windows emulators. Even so, the two Intel Pentium-based systems--the ALR Evolution V/60 and the Dell OmniPlex 566--led the pack because they ran the Windows applications in native mode.
Under SoftWindows, the Integrix SWS10/512 50MHz SPARCstation 10 (with two SuperSPARC/50 processors) and the RISC System/6000 POWERstation 25T (with an IBM PowerPC 601/66 processor) far outpaced the competition. Tatung's Super COMPstation 10 Model 51 with a single SuperSPARC/50 processor ran Excel about 23 percent faster within Wabi than within SoftWindows.
The HP 9000 Model 712/80i and Silicon Graphics Indy were both slowest when running Excel under SoftWindows. At the time of our testing, however, SoftWindows was available only in beta form. The DEC 3000 and the ALR Evolution could not run these tests because neither SoftWindows nor Wabi was available on these platforms.
When we correlated core component performance with the ability to run CorelDRAW, none of the systems met our expectations. IBM's RISC System/6000 POWERstation 25T, which headed the charts in all our graphics subsystem tests, lagged on our CorelDRAW test, although the results were tightly spaced. And since disk access played a big role in that test, we expected the Silicon Graphics Indy, with its fast disk subsystem, to perform better. Since the Integrix SWS10/512 produced good integer scores throughout all the low-level tests, its first-place finish makes some sense. Yet its runner-up on the CorelDRAW test--the HP 9000 Model 712/80i--was not very strong on integer functions. Because so much of the performance on this test was erratic, we can only surmise that the CorelDRAW ports to these varying Unix platforms are far from equal.
Contrary to their showing on the AutoCAD tests, where workstations based on Sun SuperSPARC chips performed best, the Integrix SWS10/512, Sun SPARCstation, and Tatung COMPstation were virtually tied for last place on the FrameMaker test. Since the Integrix and Sun workstations both have two 50-MHz SuperSPARC CPUs and ran at virtually the same speed as the single-processor Tatung COMPstation, clearly FrameMaker's code is not designed to scale CPUs.
Since FrameMaker is not available for an Alpha-based unit,
we could not test the DEC 3000. We tested the ALR Evolution with FrameMaker
running under SCO Unix and Unixware, but since only an earlier version of
FrameMaker was available for these operating systems, the test results were not
directly comparable.
Pro Engineer makes heavy demands on a unit's video and
floating-point system. But Carrera's 200-MHz Alpha-based Hercules 200, packing a
Compaq QVision video card, managed to perform well against some stiff
competition from the Unix camp. On our Graphstone tests, the inherent drawing
ability of the QVision card did not amount to much, but the EISA slot's speed
advantage in transmitting data to the Hercules display subsystem paid off. We
were unable to test Pro Engineer on the Intel-based NT workstations because it
would not run properly, and we were also unable to test it on the ALR because it
was not available for Intel-based Unix.
Strong video performance and a fast disk are vital in PV-Wave, a highly visual scientific analysis program. It's no surprise that the DEC 3000 Model 300X AXP workstation, the Unix system most often at the top of floating-point charts, took first place in the PV-Wave test. The Silicon Graphics Indy, offering the best disk performance and good video, took second place. The IBM RISC System/6000 POWERstation 25T and the HP 9000 Model 712/80i, which showed well- rounded performance in all three categories--CPU, graphics, and disk--were front-runners in the Unix field as well.
Tops among the NT workstations were the DeskStation Tyne v4633x, with the best video performance in its group, and the Hercules, with fast video and CPU scores. We were unable to test the ALR, because PV-Wave was not available for Intel-based Unix systems.
With three of our tested versions of Unix not supported in the current release of AutoCAD--Digital's OSF/1, as well as SCO Unix and UnixWare for the Intel Pentium--it is hard to say whether the top- performing models would otherwise keep their positions. According to our tests (the 2-D and the newer, more intense 3-D test), you would probably want to run AutoCAD on a system with either one or two SuperSPARC chips: The Integrix SWS10/512, Sun SPARCstation, and Tatung COMPstation, all running Solaris 2.3, show that AutoCAD is well tuned for their environment.
Carrera's Hercules 200 unit performed exceptionally well on the Picture Publisher test. The two Intel Pentium NT workstations--the ALR Evolution and Dell OmniPlex--also did well. Judging by the collective low-tier performance of all three MIPS machines (the Acer, DeskStation, and NeTpower), Picture Publisher has so far been better ported to the Intel platform.
David Wilson of Workstation Labs (Humbolt, Arizona) performed the low-level testing.
The SPECmark scores are based on tests written in C and FORTRAN. Results are reported for both integer operations (SPECint92) and floating-point operations (SPECfp92). Currently, the SPECmark tests run on Unix operating systems only.
The Xmark score is a weighted geometric mean of a series of X11PERF graphics tests. These Unix-based performance tests are similar but not comparable with PC Magazine Labs' Windows-based Graphics WinMark benchmark test.
The Graphstone tests comprise 125 routines carried out with 13 different graphics types; they indicate overall graphical performance. The tests are written in C, and the raw scores are measured in operations per second.
Khornerstone results are calculated from 21 tests--written in C and FORTRAN--that stress each workstation's CPU, FPU, and disk subsystem.
Dhrystone 2.0 is a series of CPU-intensive exercises written in C. These tests are useful for measuring the performance of systems' programming applications.
The single-and double-precision Whetstone tests are designed to simulate the FPU-intensive programs used in engineering and scientific fields.
At PC Labs, we ran ten "Stanford" test routines designed to assess a system's multitasking capability. Each routine is run as a single process and then as 2, 4, 8, and 16 concurrent processes. We classified the tests into three groups (floating-point, integer, and recursive operations); each point plotted on our graphs represents a product's average timing within the group of tests.
The Stanford Floating-Point test suite consists of three parts. Mandel uses the Mandelbrot fractal algorithm--which represents a form of matrix multiplication performed on complex numbers--to plot the Mandelbrot set through the points (-4.4, -2.4, 1.6, 2.4). Floating-Point Matrix Multiply and Fast Fourier Transformation are two standard gauges of a CPU's floating-point performance.
The Stanford Integer test suite consists of Puzzle, intMM, and Tree. Puzzle uses a processor-bound algorithm employing many nested loops, and intMM performs a matrix multiplication on integer arrays. The Tree algorithm dynamically constructs a balanced binary tree with 5,000 nodes, providing a good indication of the operating system's speed at allocating data-storage memory. These integer-intensive looping routines are a good measure of a CPU's raw instruction-processing speed.
The Stanford Recursion test suite consists of Towers, Queens, Quick Sort, and Bubble Sort algorithms. The Towers algorithm solves the classic "towers of Hanoi" problem by employing a highly recursive, best-fit routine. The Queens algorithm solves the similar "eight queens" problem using a recursive approach as well. The Quick Sort and Bubble Sort algorithms recursively implement the standard sort routine found in C-language libraries. All algorithms use numeric-compare processes on simple data structures with no floating-point math involved. These small routines provide a good indication of a CPU's efficiency at handling the overhead and state changes involved in function calls, which are important parts of modern structured code.
We run 16-bit Windows emulation tests on both the NT and Unix workstations. On the Windows NT platforms, we use NT's native Windows emulation mode and run our WinApp 16 macro for Microsoft Excel for Windows. On select Unix platforms, we use Insignia Solutions' SoftWindows or Sun's Wabi as the mode of emulation and run the same Excel micro.
To help gauge real-world performance, we also run a number of native applications tests on the Windows NT-and Unix-based systems. We first looked for software compatible with all three NT processors and all six versions of Unix represented. At test time, only Pro Engineer (from Parametric Technology Corp.) and PV-Wave (from Visual Numerics) covered the RISC territory.
Pro Engineer is an advanced CAD/CAM application, and we run macros that cycle through the process of designing and building tools, from blueprints to die stage to finished manufacturing. The NT and Unix workstations all execute the same macros.
PV-Wave is a sophisticated graphical application for scientific analyses. We perform rendering and mathematical functions, including magnetic resonance imaging (MRI) and real-time photo decomposition. Because of a slightly different PV-Wave interface under NT, we were unable to run exactly the same functions on the NT workstations that we ran on the Unix workstations.
We were able to run several tests only on the Unix workstations. With Corel Corp.'s CorelDRAW, Version 3.0, an intricate drawing consisting of 683 separate objects is color-separated and printed to disk; other tasks are performed as well. Using Frame Technology's FrameMaker, Version 4.0, a professional desktop publishing application, we test each workstation's ability to manage complex documents by performing a series of functions within a 200-page document. The AutoCAD 2-D and 3-D tests were developed by the AutoCAD Users Group of San Diego; these tests, which are run with Autodesk's AutoCAD, Release 12, measure elapsed times for a variety of common CAD operations.
Just one other NT application was available at the time of testing across the Alpha, Intel, and MIPS platforms: MicroGrafx's Picture Publisher, Version 4.0. With this application, we perform tests similar to those run on the Unix platforms using CorelDRAW.
List price (tested configuration): We report the prices for the workstations configured the way we tested them. Some vendors provide an estimated selling price to more accurately reflect what customers should expect to pay for the system.
A description of each high-end workstation's processors is followed by the CPU clock speed (in parentheses).
Internal cache refers to the cache built onto the processor. For most of the systems we tested, the processor cache is divided between data and instruction cache; other systems unify data and instruction cache.
Under hard disks, we indicate the number of hard disks included in the test system, along with their capacity and type.
Power supply (and number of connectors): We list the wattage of each power supply and the number of connectors for each. The number of connectors determines how many devices the workstation can power.
Bus architecture (primary, secondary) refers to the buses that connect the I/O slots to the CPU.
Processor upgrade path: Processors can be seated in several different types of sockets, which determine a computer's CPU upgradability. These sockets can be standard, LIF (low insertion force), ZIF (zero insertion force), or TAZ (tool-activated ZIF); the upgrade chip can also be integrated onto a daughterboard.
We indicate the BIOS version (or date) for each system. The BIOS, which performs a computer's initial boot and test routines, may affect a unit's performance on our benchmark tests. Many Unix workstations, however, do not have BIOS programs that are directly comparable to those we find in the PC world.
Flash BIOS: Flash ROM stores the BIOS code, replacing conventional ROM storage for this purpose. Unlike ROM, flash memory can handle both reads and writes, making the BIOS upgradable for future changes. Many Unix workstations do not have flash ROM.
Installable external RAM cache (per processor): We list the maximum amount of external RAM cache each processor can support.
Cache architecture: In a direct-mapped cache architecture, each address in main memory is assigned to exactly one location in the cache. Each location in the cache, however, can accept data from many main memory addresses--though not from more than one at once. This many-to-one relationship between main memory and cache can sometimes lead to a data bottleneck. Some of the vendors address this problem by using a proprietary cache architecture, such as Unified secondary cache, TI SuperCache, or Harvard Architecture.
Cache write design: For write-back cache, data that has been changed is written back to the cache; main memory is updated only when the cached data is replaced, thus minimizing delays. Write-through caches write updated data to both cache and main memory. Although write-through cache is relatively easy to implement, it can introduce delays when the processor must wait to complete writes to the slower main memory. Buffered write-through cache is like regular write-through cache except that the changes are stored in buffers until the system bus is free.
The disk controller location refers to the location of the disk controller circuitry. The controller can be located either on a card or on the system's motherboard.
CD-ROM drive: For those tested systems that included a CD-ROM drive, we indicate the drive manufacturer and model.
The display circuitry location refers to the location of the video circuitry that directs video performance. The circuitry can be located either on a card or on the motherboard.
We indicate whether each system supports audio and SCSI-2.
We report each system's maximum graphics support. This may require the addition of a graphics card.
Network options include a built-in Ethernet or Token-Ring interface.
Carol A. Venezia
Now that Intel is rolling out new speedier, low-voltage Pentiums, RISC CPU vendors are responding with lower voltages and faster clock speeds.
ALPHA: Digital Equipment Corp.'s Alpha AXP 21064 CPU is RISC's speed demon, operating at up to 275 MHz. The 21064 is a full 64-bit design that can execute up to two instructions per clock. But sales have been weak, in part because there are few applications for this platform.
HP PA-RISC: Hewlett-Packard's PA-RISC architecture has long been a force in the workstation market. The flagship member of this family is the PA-7100, a 32-bit superscalar design running at 100 MHz. This year, HP rolls out the PA-7100LC, a new low-power superscalar CPU that runs at up to 100 MHz.
IBM PowerPC: Currently available from the PowerPC family is the PowerPC 601, a 3.6-volt, 32-bit design running at speeds up to 80 MHz and able to execute up to three instructions per clock cycle. The future for this CPU family includes the forthcoming low-power 603. But, much like the Alpha, the PowerPC's workstation future lies in applications development.
MIPS: MIPS Technologies' R4000 CPU family includes the high-performance R4400 series and the new R4600 series. The nonsuperscalar R4400's top clock speed is currently 150 MHz, and the R4600 runs at internal clock speeds of up to 133 MHz. The R4000SC is on the low end of the R4000 family and has separate data and instruction caches of 8K each.
Sun SuperSPARC: Sun Microsystems' SPARC CPUs includes MicroSPARC II, a
low-power, uniprocessor version; SuperSPARC, capable of multiprocessing; and
UltraSPARC, a 64-bit design slated for 1995. Of the RISC vendors, Sun may be the
most vulnerable: It has the users and the apps, but it doesn't offer the speed
or floating-point performance of its competitors.
------------------------------
Type
Hardware Review
Evaluation
Company
Carrera Computers Inc.
Advanced Logic Research Inc.
Dell Computer Corp.
Acer America Corp.
DESKSTATION Technology Inc.
Netpower Inc.
Digital Equipment Corp.
Hewlett-Packard Co.
International Business Machines Corp.
Silicon Graphics Inc.
Integrix Inc.
Sun Microsystems Inc.
Tatung Science and Technology Inc.
Product
Tatung Super COMPstation 10/51 (SPARC-based microcomputer)
Carrera Computers Hercules 200 (Alpha-based system)
Adavanced Logic Research Evolution V/60 (Pentium-based system)
Acer America AcerFormula 4400 (MIPS-based
system)
Dell Computer Dell OmniPlex 566 (Pentium-based system)
DeskStation Technology Tyne v4633x (MIPS-based
microcomputer)
NetPower Netstation 300 (MIPS-based system)
DEC Alpha 3000 300X AXP (Alpha-based microcomputer)
HP Apollo 9000 700 712/80i (HP PA-RISC-based system)
IBM RS/6000 POWERstation 25T (PowerPC-based microcomputer)
Silicon Graphics IRIS Indigo Indy (Graphics system)
Integrix SWS10/512 50MHz SPARCstation 10 (SPARC-based microcomputer)
Sun Microsystems SPARCstation 20 502 (SPARC-based microcomputer)
Topic
Evaluation
Workstations
Microcomputer
Record #
15 350 222
******************************
*RISC workstations: ready for the desktop?
PC Magazine: May 31, 1994
COPYRIGHT Ziff-Davis Publishing Company 1994
******************************