Netscape Enterprise Server 2.0.1
March 10, 1997
This report compares the performance of Netscape Enterprise Server 2.0.1 and Lotus Domino Web Server 1.5a serving HTML text. All tests were done on a single-processor Hewlett-Packard NetServer 5/133 LS2 running Microsoft Windows NT 4.0.
Three sets of tests were run: Netscape Enterprise Server serving HTML files, Lotus Domino serving the same files, and Lotus Domino serving from a Notes database containing the HTML files.
The WebStone 2.5 benchmark was used to test the system. All three configurations were tested over the range of 4 to 24 WebStone client processes. We restricted the testing to small loads because at higher loads the Domino server experienced very high error rates and lower performance (see data summary). In addition, the Netscape server was tested with loads from 30 to 240 WebStone client processes.
Figure 1 shows the peak connection rates measured in the range from 4 to 24 client processes. Figure 2 shows the corresponding data throughput rates. A table summarizing the peak performance follows Figure 2.
This Performance Report was commissioned by Netscape Communications Corporation to compare the performance of their Enterprise Server 2.0.1 to the performance of Lotus Domino Web Server 1.5a. The WebStone 2.5 run rules were followed, with two exceptions: the fileset used was the Silicon Surf fileset from WebStone 1.1, and the range of loads used was from 4 to 24 WebStone client processes.
The Silicon Surf fileset was chosen because the standard WebStone 2.5 fileset includes one data file that is five megabytes in size, and we were not confident that we could load this file into the Notes database properly. The fileset was translated from the format used by WebStone 1.1 to that used by WebStone 2.0 and WebStone 2.5. The narrower range of loads was used because even at loads of 20 and 24 WebStone client processes tests of the Domino server gave error rates of 61 to 442 errors per second. Other servers we have tested give error rates of 0.0 to 0.1 error per second at the highest attainable load. See the data summary for the fileset used and the data summary tables.
The NetScape Enterprise Server was also tested at loads ranging up to 240 WebStone client processes.
Mindcraft, Inc. conducted the performance tests described in this report between March 3 and March 7, 1997, in the Netscape performance laboratory in Mountain View, California. Mindcraft used the WebStone 2.5 benchmark to measure performance.
Mindcraft certifies that the results reported herein fairly represent the performance of the Netscape Enterprise Server 2.0.1 and of the Lotus Domino Web Server 1.5a running on a Hewlett-Packard NetServer 5/133 LS2 under Microsoft's Windows NT Server 4.0 operating system as measured by the WebStone 2.5 benchmark. Our test results should be reproducible by others who use the same test lab configuration as well as the computer and software configurations and modifications documented in this report.
This analysis is based on the complete WebStone benchmark results for the tests described here.
The WebStone 2.5 benchmark stresses a system's networking ability in addition to other aspects of server performance. The best way to see if there is unused capacity on a server computer running WebStone is to look at the CPU utilization. It was 100% for all of the tests when the peak performance was attained. The maximum network bandwidth used was 33.5 Mbits/second, for the Netscape server with 240 WebStone client processes. This is well below the maximum expected available bandwidth of a 100Base-TX Ethernet, and below the aggregate performance of the seven 10Base-T Ethernet connections to the client systems. Therefore, the networks did not significantly limit the performance of the Web servers as measured in these tests.
Figures 3 and 4 show the network throughput and connection rates, respectively, of the Lotus and Netscape servers running on a Hewlett Packard Web Server LS2. The performance curves are nearly flat for all three configurations over the range 8 through 24 clients. Above this load level the performance of the Lotus Domino server degrades rapidly, with increased latency (Figure 5) and greatly elevated error rates. The performance of the Netscape Enterprise Server increases slightly with increasing demand, up to the maximum test load of 240 WebStone client processes.
The WebStone load that a server computer can support depends on four primary factors:
For all the tests presented here the network throughput was well below the maximum capacity of the network hardware. The server platform was the same for all the test runs. Each server used all available CPU cycles when peak performance was measured. This leaves only Web server efficiency as the determining factor of the observed performance.
Mindcraft followed the standard WebStone 2.5 run rules with the following changes:
The following basic set of procedures was used for performing these tests:
The data files used for the static HTML testing were the default "Silicon Surf" fileset, described in filelist.ss as distributed with the WebStone 1.1 benchmark. The fileset was translated from the format used by WebStone 1.1 to that used by WebStone 2.0 and WebStone 2.5. See the data summary for the translated fileset file.
This fileset was designed to represent a real-world server load, based on analysis of the access logs of Silicon Graphics, Inc.'s external Web site, http://www.sgi.com. Netscape's analysis of logs from other commercial sites indicated that Silicon Surf access patterns were fairly typical for the Web when they were designed.
The Silicon Surf model targets the following characteristics:
In order to cause the Netscape Enterprise Server to use all
available CPU cycles on the server computer system, we used seven
WebStone client systems. One of these systems also served as the
Webmaster, controlling the WebStone driver. The test lab network
configuration used for this work is shown below:
The tests described in this report were performed on an isolated LAN that was quiescent except for the test traffic.
|Copyright © 1997-98. Mindcraft, Inc. All rights reserved.
Mindcraft is a registered trademark of Mindcraft, Inc.
For more information, contact us at: firstname.lastname@example.org
Phone: +1 (408) 395-2404
Fax: +1 (408) 395-6324