Baltimore Technologies
SelectAccess 3.1
AuthMark Performance

By Bruce Weiner
(PDF version, 62 KB)
August 6, 2001

Contents

Executive Summary
   Conclusions
   Mindcraft Certification
Analysis
Methodology
Configuration
iLOAD MVP
AuthMark

Disclosure

Baltimore Technologies sponsored the testing in this report. Mindcraft, Inc. conducted the performance tests described herein at Baltimore’s facilities in San Mateo, California.

Executive Summary

Baltimore Technologies SelectAccess 3.1 delivers the highest overall and per CPU AuthMark Login and Extranet performance we've measured to date — 205,359 logins per minute and 7,065 logins per minute per CPU as well as 354,112 Extranet operations per minute and 11,782 Extranet operations per minute per CPU

Baltimore Technologies SelectAccess 3.1 broke all previous performance records for logins and raised the bar for Extranet performance while providing outstanding performance scaling. Mindcraft^® tested SelectAccess 3.1 running on Sun Enterprise servers. For these tests, we used our iLOAD MVP™ test tool running the AuthMark™ Login and Extranet Scenarios.

Login Scenario

The Login Scenario represents the type of load commonly seen at portal sites. It simulates users accessing protected resources via Web servers. The Login Scenario assumes that 10% of a portal's user population logs in concurrently to use portal resources. All tests were done using a 1,000,000-user directory with 100,000 active users. 

The Login Scenario measures the combination of one user authentication and one authorization for access to a resource (called a Login). Table 1 summarizes the Login Scenario performance as a function of the SelectAccess policy server system(s) configuration. The Scaling Factor in Table 1 shows how much faster a configuration is compared to a single system with one CPU using one directory server, the smallest configuration.

Because the SelectAccess policy server is the control point for all authentication and authorization, our tests were structured to push the policy server systems as close as possible to 100% CPU utilization. The Result Analysis section in the second part of this white paper includes an analysis of the effect of CPU utilization on the Login Scenario benchmark results.

Table 1: SelectAccess Login Performance Scalability - 1,000,000 Users

* - Look at Login Performance Analysis in the second part of this report for the computation of the logins/minute/total CPUs and at the hardware configurations for more details on the test environment.

Figure 1 shows SelectAccess's Login performance from Table 1 by server configuration.

Figure 1: SelectAccess Login Scalability for 1,000,000 Users

Extranet Scenario

The Extranet Scenario measures the combination of one user authentication and 10 authorizations for access to resources (these 11 operations constitute one Extranet sequence). The Extranet Scenario, because it uses a more realistic mix of operations than the Login Scenario, provides a better basis for capacity planning purposes.

Table 2 shows the SelectAccess Extranet Scenario performance for three configurations. The CPU utilizations shown are the average for each type of server. CPU utilizations for the Extranet tests are analyzed in detail in the second part of this white paper.

Table 2: SelectAccess Extranet Performance - 1,000,000 User Directory

Conclusions

The benchmark results lead us to conclude that:

• Baltimore Technologies SelectAccess 3.1 delivers the highest overall AuthMark Login and Extranet performance we've measured to date.
• SelectAccess 3.1 achieves the highest Login performance per policy/security server CPU and the best Login and Extranet performance per CPU, for all CPUs used, that we have measured to date.
• SelectAccess delivers outstanding performance scaling as CPUs and policy servers are added to a configuration.

Mindcraft Certification

Mindcraft certifies that the results reported accurately represent the performance of Baltimore Technologies SelectAccess 3.1 running on Sun Enterprise servers configured as specified herein and as measured by AuthMark benchmark.

Our test results should be reproducible by others using the same test lab configuration, the same Sun server configurations, and the same software configurations documented in this white paper.