Mindcraft Certified Performance

Netegrity SiteMinder 4.51
AuthMark Performance

By Bruce Weiner
(PDF version, 62 KB)
March 26, 2001
Update to March 22, 2001 version


Executive Summary
   Mindcraft Certification


Netegrity Inc. sponsored the testing in this report. Mindcraft, Inc. conducted the performance tests described in this report at Sun’s test lab in Menlo Park, California.


We thank Sun for providing the systems used for the tests and the support staff who helped configure the servers.

Executive Summary

Netegrity SiteMinder 4.51 delivers 112,202 logins per minute and 20,179 Extranet Sequences (221,969 operations) per minute for 1,000,000 users, achieving the highest overall and per CPU performance that we've seen

Netegrity SiteMinder 4.51 delivers dramatic performance improvement over earlier versions and sets a new performance standard for authentication/authorization products. Throughout our tests, SiteMinder outperformed all other similar products we've tested both in overall performance and in performance per policy/security server CPU.

Mindcraft® tested Netegrity SiteMinder 4.51 running on a mix of Sun Enterprise servers. For these tests, we used Mindcraft’s iLOAD MVP™ test tool running the AuthMark Login and Extranet Scenarios.  

Login Scenario

The Login Scenario represents the type of load commonly seen at portal sites. It simulates users accessing protected resources via Web servers. The Login Scenario assumes that 10% of a portal's user population logs in concurrently to use portal resources. All tests were done using a 1,000,000-user directory with 100,000 active users. 

The Login Scenario measures the combination of one user authentication and one authorization for access to a resource (called a Login). The Result Analysis section in the second part of this white paper explains the benchmark results.

The SiteMinder Policy Server is the control point for all authentication and authorization. Our tests were structured to push the Policy Server systems as closely as possible to 100% CPU utilization. Table 1 summarizes the Login Scenario performance as a function of the SiteMinder Policy Server system(s) configuration. The Scaling Factor in Table 1 shows how much faster a configuration is compared to a single system with one CPU using one directory server, the smallest configuration. 

Table 1: SiteMinder Login Performance Scalability - 1,000,000 Users

per second

per minute
Logins/ minute/ Policy Server CPU Scaling factor Policy Server(s) CPU Utilization

Policy Server configuration

660 39,588 39,588 - 100% 1 system, 1 CPU


64,959 32,480 1.6 97%

1 system, 2 CPUs

1,256 75,372 25,124 1.9 88% 1 system, 3 CPUs


75,626 37,813 1.9 98%

2 systems, 1 CPU

1,829 109,747 27,437 2.8 75%- 80% 2 systems, 2 CPUs
1,870 112,202 18,700 2.8 55% 2 systems, 3 CPUs

The CPU utilizations for the Policy Server configurations with one system, three CPUs and with two systems, two and three CPUs show that we did not have enough load generators and Web servers, or fast enough ones, to drive the Policy Servers to full CPU utilization. If the lab had enough load generators (which ran at 70% CPU utilization for the highest performance tests) and Web servers available, we fully expect that SiteMinder would have achieved more logins per minute than it did.

Figure 1 shows SiteMinder's Login performance from Table 1 by server configuration.

Figure 1: SiteMinder Login Scalability for 1,000,000 Users

Extranet Scenario

The Extranet Scenario measures the combination of one user authentication and 10 authorizations for access to resources (these 11 operations constitute one Extranet sequence). The Extranet Scenario, because it uses a more realistic mix of operations than the Login Scenario, provides a better basis for capacity planning purposes.

Table 2 compares the SiteMinder Extranet Scenario performance to that of the Login Scenario for the same hardware configuration - one Policy Server with one CPU. The results in Table 2 demonstrate that the SiteMinder Policy Server performs authorizations several times faster than authentications. It is not possible to calculate the exact performance difference because the CPU utilization of the Policy Server CPU was 50% for the Extranet test and was 100% for the Login test. The CPUs in the Web servers averaged 95% utilization while the load generator CPUs averaged 70% utilization. This means that the Extranet performance would have been significantly higher, if there were enough load generators and Web servers available in the lab to drive the Policy Server's CPU to full utilization. 

Table 2: SiteMinder Extranet and Login Performance - 1 Policy Server with 1 CPU


Extranet Scenario

Login Scenario




Authorizations/minute 201,790 39,588

Total operations/minute

221,969 79,176


The benchmark results lead us to conclude that:

  • Netegrity SiteMinder 4.51 outperforms all other products we've tested so far for the AuthMark Login and Extranet Scenarios.
  • SiteMinder 4.51 delivers the highest Login and Extranet performance per policy/security server CPU of any product we have tested to date.
  • SiteMinder delivers outstanding performance scaling as CPUs and Policy Servers are added to a configuration.

Mindcraft Certification

Mindcraft certifies that the results reported accurately represent the performance of Netegrity SiteMinder 4.51 running on Sun Enterprise servers configured as specified herein and as measured by AuthMark benchmark.

Our test results should be reproducible by others using the same test lab configuration, the same Sun server configurations, and the same software configurations documented in this white paper.

 Analysis and Test Details


  • Added load generator system CPU utilization to the Login and Extranet Scenario commentary.
  • Added lack of additional load generator systems as another reason that the Policy Servers CPUs were not fully utilized.


The information in this publication is subject to change without notice.


This publication does not constitute an endorsement of the product or products that were tested. This test is not a determination of product quality or correctness, nor does it ensure compliance with any federal, state or local requirements.

Copyright © 2001. Mindcraft, Inc. All rights reserved.
Mindcraft is a registered trademark of Mindcraft, Inc.
Product and corporate names mentioned herein are trademarks and/or registered trademarks of their respective owners.
For more information, contact us at: info@mindcraft.com
Phone: +1 (408) 395-2404
Fax: +1 (408) 395-6324