12
for class on Tuesday Oct. 24, 2000
Please re-read Sections 2.6 through 2.9 of the Patterson & Hennessy text, together with the attached material from SPEC and be prepared to discuss the following:
The Standard Performance Evaluation Corporation (a.k.a. SPEC) is a group of computer manufacturers that agree on a set of standard benchmarks and on measurement methods for evaluating their machines. Execution times are compared with the times on a "reference machine," which is now a certain SPARC workstation. Before this organization was formed (in the late 1980's) competing companies often used different benchmarks to tout their machines, and/or suspicious figures of merit like MIPS (not the company, the metric) and/or questionable ways of combining a set of benchmark numbers.
The SPEC Website, http://www.spec.org, contains a great deal of information and links, including benchmark results for a large number of machines. Attached to this sheet (if there's a staple in the upper lefthand corner) is an excerpt from a SPEC FAQ about the CPU2000 benchmarks, together with four CINT2000 results pages from four different machines that were tested this year--one Pentium III (from Dell), one UltraSPARC (from Sun), one MIPS (the R12000 from SGI), and one Alpha (from Compaq). You should become very comfortable with the format of these reports.
Manufacturers, customers, and others rely on these figures to understand computer performance. What do you think of the SPEC approach to measuring and reporting performance?