Netegrity SiteMinder 4.61
with Microsoft Active Directory
AuthMark Performance Details

By Bruce Weiner
(PDF version, 131 KB)
April 18, 2002

Contents

Executive Summary
Login Results
Extranet Results
Conclusions
Test Methodology
    iLOAD MVP
    AuthMark
Server Set Up
Load Generators

Disclosure

Netegrity sponsored the testing in this report. Mindcraft, Inc. conducted the performance tests described in this report at Microsoft's test lab in Redmond, Washington.

This part of the white paper details the Login and Extranet Scenario performance characteristics of Netegrity SiteMinder 4.61 SP2 with Microsoft Active Directory.

Detailed Login Results

Table 1 summarizes the Login Scenario performance as a function of the SiteMinder Policy Server configuration and the number of entries in the directory. All of the Login Scenario tests used one Policy Server; only the number of CPUs was varied as shown in Table 1.

The SiteMinder Policy Server is the control point for all authentication and authorization. Our tests were structured to push the Policy Server system as close as possible to 100% CPU utilization. The CPU Utilization by Server Type column in Table 1 shows that the SiteMinder Policy Server was performing at its maximum. Except for the Login 1 test, all user credentials for the entries tested were cached in the SiteMinder Policy Server, which accounts for the Active Directory (AD) server CPU having been idle during the tests. At the same time, the Web server CPUs had plenty of performance left in them, so they were not performance bottlenecks for the tests. The load generator (LG) systems were sufficient to create a workload on the servers that drove the Policy Server to its peak performance.

Each of the three networks (load generators to Web servers, Web servers to Policy Servers, and Policy Servers to directory servers) had enough bandwidth available that they were able to support the highest load without limiting overall performance.

Table 1: SiteMinder 4.61 Login Performance

The Login 1 and 2 tests show the performance benefit that caching user credentials can bring. The two test configurations are identical except for the use of a second CPU in the Active Directory server for the Login 1 test; we needed to add this CPU in order to maximize the Policy Server performance. You can see that enabling caching in the Login 2 test increases performance 1.95 times that of the Login 1 test.

The Login 2, 3, and 4 tests (the gray shaded rows in Table 1) show how SiteMinder's performance scales as Policy Server CPUs are added. Going from one to two Policy Server CPUs in the Login 3 test improves performance 1.64 times over that in the Login 2 test. The Login 4 test uses four Policy Server CPUs and improves performance 2.62 times over the single CPU Login 2 test. As we added Policy Server CPUs, we also needed to add Web servers to create a load that maximized the performance of the Policy Server. Figure 1 shows SiteMinder's performance for the Login 2, 3, and 4 tests by the number of Policy Server CPUs.

Figure 1: SiteMinder Login Scalability for 1,000,000 Users

The Login 5 test used a directory with 20 million entries in it and tested 1 million active users. Comparing the performance for the Login 5 test with that of the Login 4 test, which used a 1 million-entry directory and tested only 10% of the number of active users used for the Login 5 test, you can see that SiteMinder continues to deliver high performance even as the number of active users increases by a factor of 10.

Detailed Extranet Results

The Extranet Scenario simulates customers or suppliers logging into a private Web site and obtaining information they are authorized to receive. It measures the combination of one user authentication and 10 authorizations for access to resources (these 11 operations constitute one Extranet sequence). We report the total operations per minute, which is 11 times the number of Extranet sequences. The Extranet Scenario, because it uses a more realistic mix of operations than the Login Scenario, provides a better basis for comparing access control and identity management solutions. You can find a more complete description of the Extranet Scenario below.

The Extranet test was done with SiteMinder configured to cache user credentials, as one would do if the servers were on the inside of a private network with firewalls to protect access to systems that store passwords and other sensitive information. We call this a full-cache configuration.

Table 2 shows the Extranet Scenario performance metrics and CPU utilizations for each type of system. We used a single directory server with one CPU for this test. The Active Directory server CPU was essentially idle during the tests because we used a full-cache configuration.

Table 2: SiteMinder 4.61 Extranet Performance

Conclusions

The benchmark results lead us to conclude that:

• SiteMinder 4.61 with Microsoft Active Directory sets the per CPU performance standard for 1,000,000-user directories against which other Windows 2000 Policy Server-based products will be measured.
  
• SiteMinder 4.61 on Microsoft Windows 2000 using Active Directory demonstrates the performance required to support  20,000,000 users.
  
  
• The SiteMinder 4.61 Policy Server on Windows 2000 delivers outstanding scalability across four processors.

Test Methodology

Mindcraft^® tested the performance of SiteMinder with Active Directory using our iLOAD MVP™ tool to run the AuthMark ™ Benchmark Login and  Extranet Scenarios. We describe these tests in this section to further your understanding of the performance results discussed above.

iLOAD MVP Overview

iLOAD MVP is a general-purpose, script-driven capacity planning, benchmarking, and regression testing tool. The major components of iLOAD MVP are:

• A Control Center that manages client systems, controls test script execution, and reports on test results.
• Multithreaded client load generators that execute test scripts to simulate users accessing a server.
• Test script generation programs.
• Test data generation programs.

iLOAD MVP provides the capabilities needed to test high-performance servers with a small number of client systems. Its capabilities include:

• The ability to simulate a large number of simultaneous user sessions. The number of user sessions is limited only by the client OS, the amount of memory and the performance of the client systems.
• Support for HTTP 1.0 and 1.1 as well as LDAP V3.
• Support for authentication and authorization.
• Support for SSL.
• Custom test scripts.

The AuthMark Benchmark

The AuthMark Benchmark is designed to test the performance of products that provide authentication and authorization services in support of Web servers. Authentication is the process of verifying who a user is; it typically occurs when a user logs in. Authorization is the process of verifying that an authenticated user is allowed to see or to use a particular resource. In the case of a Web server such resources include HTML files, graphic files, and programs that generate Web pages dynamically.

AuthMark simulates a large number of users accessing Web servers via their browsers. This approach permits AuthMark to test authentication and authorization performance independent of the technology used to provide those services.

AuthMark consists of several test scenarios to determine various aspects of performance for authentication and authorization systems under different circumstances.

AuthMark Login Scenario

The AuthMark Login Scenario focuses on testing authentication. We call it the Login Scenario because authentication is done the first time a user accesses a protected part of a Web site, just like a login. The HTTP 1.0 and 1.1 protocols define the steps a browser follows for authentication. Some of the steps are visible to you and others are not. It is important to understand what happens during a login in order to understand what the Login Scenario measurements mean.

Login Process

The following simplified sequence will walk you through the login process to show you how it works using the HTTP 1.0 and 1.1 protocols:

1. When you click on a link or enter a URL in your browser, your browser sends the requested URL to the Web server.
2. The Web server determines that you must be authenticated before it returns the resource at the requested URL. Typically, the authentication requirement is specified as part of the Web server's configuration or via an authentication/authorization product connected to the Web server.
3. The Web server sends back a "401" HTTP response to your browser indicating that you are not authorized to see that requested resource.
4. Your browser pops open a window and asks you to enter your user ID and a password.
5. After you enter your user ID and password, your browser stores them in memory and associates them with the protected space (called a realm ) containing the URL you requested.
6. Your browser then resends a request for the same URL but this time it includes an HTTP authorization header containing your user ID and password.
7. This time the Web server checks your user ID and password to see if they match the authentication information in the authentication system. If they do, you are authenticated.
8. Now that you have been authenticated, the authorization system checks whether or not you are authorized to access Web pages in the realm. If you are authorized, the Web server sends the Web page you requested.

Notice that the URL you clicked on or entered is actually sent twice (in steps 1 and 6). This means that the authentication system is used twice—first, it finds out that the requested URL requires the user be authenticated, then it processes the authorization header when the request is resent.

Once a user has been authenticated, a Web browser automatically sends the authorization header whenever the user requests a URL in the same realm requiring authentication.

Login Scenario Configuration

Table 3 shows the AuthMark Login Scenario configuration parameters we used.

Table 3: AuthMark Login Scenario Configuration Parameters

The number of user sessions active during a given test run is determined by the length of the test and the number of logins. Sessions are not logged out once created. Instead, each session remains quiescent after login. 

Running the Login Scenario

The basic steps for running the Login Scenario are:

1. Generate the data to fill the security database. iLOAD MVP provides a tool to generate realistic data for the LDAP V3 organizationalPerson object class and Netscape's inetOrgPerson object class. It also includes tools to load the same data into an LDAP directory, which was used for this test.
2. Load the security database with the user data.
3. Generate the test scripts for the Login Scenario. iLOAD MVP provides a tool to do this. These scripts drive iLOAD MVP to simulate user interaction with the Web server(s).
4. Load Web pages on the Web server(s). There are 100 Web pages each of which is 14 KB in size for the Login Scenario.
5. Load and configure the user management system or authentication/ authorization system.
6. Run the benchmark.

The Login Scenario test script selects users randomly from the user database (see Table 3 for the numbers we used for this test). The tester is free to select the number of load generator systems and the number of iLOAD MVP client threads to use. These are called the load generators.

The tester selects the number of load generators to get the highest performance possible from the authentication/authorization system being tested. In order to obtain peak performance, the tester may need to use multiple Web servers and directory servers.

The tester is permitted, but not required, to do a warm-up run of the test scenario in order to get the servers to a state that would more likely represent the state they would be in during normal operation. For this benchmark, we warmed-up the servers by running each test script in its entirety.

Extranet Scenario

The Extranet Scenario simulates an environment where users must login to a Web site and where all access requests require authorization. This scenario depicts a more realistic usage pattern than the Login Scenario.

The Extranet Scenario test execution starts with the same operation sequence as the Login Scenario (steps 1 - 8 above) and continues with the following operations:

9. The test client requests another resource.
10. The authorization services check the validity of the user and and that the user is authorized to have access to the resource.
11. If the user is authorized, the resource is returned.
12. The test client then requests additional resources.

The Extranet Scenario sequence consists of a total of 11 client operations per user session:

• The first resource request (which does two operations): the login request and the authorization for resource requested; and
  
  
• Nine more resource requests that require authorization.

SiteMinder checks the continuing validity of an authenticated user each time a resource access request is made to ensure that the user session has not been revoked.  However, the user is not re-authenticated. As a result, the user does not see a new login request as long as the resources being accessed are in the Internet domain in which the user has been authenticated.

For the Extranet Scenario, we warmed-up the servers by running the test script in its entirety.

Extranet Scenario Configuration

Table 4 shows the AuthMark Extranet Scenario configuration parameters we used.

Table 4: AuthMark Extranet Scenario Configuration Parameters

Sessions are not logged out once created. Instead, each session remains quiescent after all of its requests have been satisfied. 

Running the Extranet Scenario

The basic steps for running the Extranet Scenario are:

1. Generate the data to fill the directory. iLOAD MVP provides a tool to generate realistic data for the LDAP V3 organizationalPerson object class and Netscape's inetOrgPerson object class. It generates the data in standard LDIF format so that it can be loaded directly into an LDAP directory
2. Load the directory with the user data.
3. Generate the test scripts for the Extranet Scenario. iLOAD MVP provides a tool to do this. These scripts drive the iLOAD clients to simulate user interactions with the Web servers.
4. Load Web pages on the Web servers. There are 100 Web pages each of which is 14 KB in size.
5. Load and configure the authentication/authorization system.
6. Run the benchmark.

The Extranet Scenario test scripts select users randomly from the user directory (see Table 4 for the numbers we used for these tests). The tester is free to use any number of load generator systems each running any number of iLOAD MVP client threads (called load generators).

The tester selects the number of load generators to get the highest performance possible from the authentication/authorization system being tested. In order to obtain the peak performance, the tester may need to use multiple Web servers and directory servers.

The tester is permitted, but not required, to do a warm-up run of the test scenario in order to get the servers in the state they would be in during normal operation. For this benchmark, we warmed-up the servers by running the test scripts in their entirety.

Server Hardware and Software Set Up

Mindcraft used three types of servers for these tests: Web, Policy and Active Directory servers. Table 5 shows the summary of the server configurations we used for each test. The detailed configurations of the SiteMinder Policy and Active Directory servers are shown in Table 6 and Table 7, respectively. Tables 8 and 9 show the Web server configurations. Figure 2 shows how the various servers were connected for the tests; however, you will need to refer to Table 5 to see which systems were used for each test.

For half of the tests we used a Giganet cLAN CL5300 30-port 1.25Gb/s switch and the associated CL1000 1.25Gb/s Host Bus Adapters for the network between the Web servers and the Policy Server. However, part way through our testing the CL5300 failed. So, we changed to an HP ProCurve 1600m 16-port 100Base-TX switch and used the 100Base-TX NICs in the servers for the Login 4, Login 5, and Extranet tests.

Table 5: Servers and CPUs

Table 6: SiteMinder Policy Server Configuration - HP NetServer lxr 8500

Table 7: Active Directory Server Configuration - HP NetServer lxr 8500

Table 8: Web Server A Configuration - 4 x HP NetServer lt 6000r

Table 9: Web Server B Configuration - 2 x IBM eServer xSeries 330

Figure 2: Test Lab Configuration
(Some servers are not used for all tests. See Table 5 for details)

Load Generator Systems

We used a variety of load generator systems for these tests. Figures 2 and 3 show the specific load generator systems we used for each test. Tables 11 through 15 detail the configuration for each of the load generator systems.

Table 11: Dell Optiplex GX400 Configuration

Table 12: Dell Optiplex GX50 Configuration

Table 13: IBM NetVista 6578RAU Configuration

Table 14: HP Workstation x2000 Configuration

Table 15: Dell PowerEdge 6350/550 Configuration

NOTICE:

The information in this publication is subject to change without notice.

This publication does not constitute an endorsement of the product or products that were tested. This test is not a determination of product quality or correctness, nor does it ensure compliance with any federal, state or local requirements.

Mindcraft is a registered trademark of Mindcraft, Inc.

Product and corporate names mentioned herein are trademarks and/or registered trademarks of their respective companies.