Ultra Enterprise 3000 Web Server
This Certified Performance Report shows that Novell's BorderManager FastCache can increase the perceived Web-serving capacity of a Sun Ultra Enterprise 3000 system more than sevenfold. "Perceived Web-serving capacity" is the maximum number of HTTP GET requests per second (rps) that a specified server environment can satisfy. The table below summarizes the peak performance of each server environment tested.
We used the Ziff-Davis WebBench 1.1 benchmark for the performance measurements in this report. Our testing started with a baseline test of a Sun Ultra Enterprise 3000 in a stand-alone configuration using static HTML data. It's peak perceived Web-serving capacity was 550 rps. We then tested a Web server environment with a Compaq ProLiant 6000 running the BorderManager FastCache between the Ultra Enterprise 3000 and the load generating clients. This configuration gave a perceived Web-serving capacity of 3288 rps for static HTML data. The third environment we tested put an Intel MB440LX DP server running the BorderManager FastCache between the Ultra Enterprise 3000 and the load generating clients. The performance of this configuration peaked at 4055 rps for static HTML data. Figure 1 shows the connection rate performance curves for these Web server environments. Similarly, Figure 2 shows the corresponding network throughput for these environments. In the figures in this report, the term "threads" means the number of WebBench requestors simultaneously making requests of the server environment. The number of threads is not indicative of the number of real users that can access a Web server environment; it is shown here to help those who want to reproduce these tests.
Figure 1: BorderManager FastCache Static Connection Acceleration
Figure 2: BorderManager FastCache Static Throughput Acceleration
We also tested the Ultra Enterprise 3000 baseline Web server environment and the ProLiant 6000 accelerated environment with a mix of 80% static HTML data and 20% dynamic data. The dynamic data was generated by a CGI program running on the Ultra Enterprise 3000. The baseline performance peaked at 86 rps with eight threads. The accelerated environment reached 1067 rps at 16 threads. Figures 3 and 4 show the connection rate and throughput, respectively, for these two environments.
Figure 3: BorderManager FastCache Mixed Connection Acceleration
Figure 4: BorderManager FastCache Mixed Throughput Acceleration
Mindcraft tested three Web server environments:
Table 1: Sun Ultra Enterprise 3000 Configuration
Table 2: Compaq ProLiant 6000 Configuration
Table 3: Intel MB440LX DP Configuration
Figures 5, 6, and 7 show the test lab configurations used. Each client system was configured as shown in Table 4. When threads were added during the testing, they were added from each subnet in order to balance the load. Each client system always ran four threads of the WebBench client program for all tests.
Table 4: Client Systems
The networks were dedicated to this testing only. The fast Ethernet hubs we used were made by Bay Networks.
Mindcraft followed the standard WebBench 1.1 test procedures. For all of the static tests, we modified the ZD_STATIC_KEEPALIVE_V11.TST test suite file to use four threads per client, to change the ramp down time to 10 seconds, to change the test time to120 seconds, and to test using 16 to 144 threads in increments of 16 (we tested to 152 threads only for the baseline test to determine where the Web server's performance fell off). All of the static tests used the ZD_STATIC_V11.WL workload file unchanged. The static data consists of 10 classes of 10 files, where each file is the same size. The file sizes start at 256 bytes and double in size for each class up to 128 KB for the last class. The 100 files of static data total 2.5 MB. The weighted average file size for the ZD_STATIC_V11.WL workload is 8,348 bytes.
For all of the mixed tests, we modified the ZD_UNIX_SIMPLE_CGI20_KEEPALIVE.TST test suite to use four threads per client, to change the ramp down time to 10 seconds, to change the test time to 120 seconds, and to test using 16 to 144 threads in increments of 16. All mixed tests were done with keep-alive turned on. All of the mixed tests used the ZD_UNIX_SIMPLE_CGI20.WL workload file unchanged. The mixed test data consists of 10 classes of 10 files, where each file is the same size, and one class of dynamic data. The file sizes for the static data are the same as for the ZD_STATIC_V11.WL workload; however, the access percentages for each class of file is lower to allow for the dynamic data. The weighted average file size for the static data in the ZD_UNIX_SIMPLE_CGI20.WL workload is 6,735 bytes. The dynamic data is generated by a CGI program written in C that runs on the Web server system.
We didn't use the Intel system for the mixed tests because of time limitations for the test work. Also, we only tested the Compaq-based accelerator only up to 64 threads for the mixed tests because the Sun system reached its the peak request rate at 32 threads and because of test time limitations. If we were to have tested with more threads, we expect that mixed data performance would have improved beyond the 1067 rps reported because the Sun's CPU was only 12% utilized and the Compaq's CPU was less than 40% utilized.
The purpose of these tests was to measure the peak performance improvement offered by Novell's BorderManager FastCache. Our performance measurements of the baseline configuration, a stand-alone Sun Ultra Enterprise 3000 running Netscape's Enterprise Server 3.0, were done with configuration parameters set to obtain the maximum performance.
When analyzing the performance of a Web server, there are four primary factors to look at: the CPU utilization of the server system, the the disk subsystem performance, the networking subsystem performance, and the Web server software performance. The Sun system's overall CPU utilization reached 97% and 81% at the peak response rate for static static and mixed data, respectively. For the static data tests, the CPU was the main factor limiting performance of the Sun system since it was fully utilized and the other performance factors were not at their limits. For the mixed data test, the CPU was not fully utilized because of the disk overhead needed to launch the CGI program that generated the dynamic data. However, it was the largest factor limiting overall performance. Because the static data was small enough to fit into RAM, the performance of the disk subsystem played almost no part in the measured performance. In addition, the network usage peaked at less than 20 Mbits/second per subnet, well below the theoretical peak of 60 to 70 Mbits/second for a half-duplex 100Base-TX network. So the network did not limit the Sun's performance.
We can conclude that the primary performance-limiting factor was the Sun's CPUs, affected, of course, by the efficiency of its operating system and the Netscape Web server software. Given the performance limitation of the baseline Web server, it is a good candidate for performance improvement using the BorderManager FastCache.
The BorderManager FastCache is able to improve the perceived performance of the Sun system by the factors shown in Table 5. This improved performance is real for people accessing a Web site accelerated by Novell's BorderManager FastCache. The performance of the Web server system itself, however, does not improve. What happens is that BorderManager FastCache is able to serve the files requested directly from its cache rather than getting them from the Web server. This means that even when there are requests for non-cacheable data, such as dynamic data generated by a CGI program, BorderManager FastCache can still give significant perceived performance improvements. In fact, the acceleration factors in Table 5 show that BorderManager FastCache is more than twice as effective for Web sites with a mix of static and dynamic data than it is for Web sites with only static data. This increased effectiveness comes from off-loading the Web server from processing almost all of the requests for static data.
Table 5: BorderManager FastCache Acceleration Factors
Mindcraft, Inc. conducted the performance tests described in this report between July 21 and August 2, 1997, at Novell's Superlab in Provo, Utah.
Mindcraft certifies that the results reported herein fairly represent the performance of systems tested as measured by Ziff-Davis's WebBench 1.1 test suite running the ZD_STATIC_V11.TST workload. Our test results should be reproducible by others who use the same test lab configuration as well as the computer and software configurations and modifications documented in this report.
|Copyright © 1997-98. Mindcraft, Inc. All rights reserved.
Mindcraft is a registered trademark of Mindcraft, Inc.
For more information, contact us at: firstname.lastname@example.org
Phone: +1 (408) 395-2404
Fax: +1 (408) 395-6324