Baltimore Technologies sponsored the testing in
this report. Mindcraft, Inc. conducted the performance tests described
herein at Baltimore’s facilities in San Mateo, California.
Baltimore Technologies SelectAccess 3.1
delivers the highest overall and per CPU AuthMark Login and Extranet
performance we've measured to date 205,359 logins
per minute and 7,065 logins per minute per CPU as well as 354,112
Extranet operations per minute and 11,782 Extranet operations per
minute per CPU
Baltimore Technologies SelectAccess 3.1 broke all previous performance
records for logins and raised the bar for Extranet performance while
providing outstanding performance scaling. Mindcraft®
tested SelectAccess 3.1 running on Sun Enterprise servers. For these
tests, we used our iLOAD MVP™ test
tool running the AuthMark™
Login and Extranet
The Login Scenario represents the type of load commonly seen at
portal sites. It simulates users accessing protected resources via
Web servers. The Login Scenario assumes that 10% of a portal's user
population logs in concurrently to use portal resources. All
tests were done using a 1,000,000-user directory with 100,000 active
The Login Scenario measures the combination of one user authentication
and one authorization for access to a resource (called a Login).
Table 1 summarizes the Login Scenario
performance as a function of the SelectAccess policy server
system(s) configuration. The Scaling Factor in Table 1 shows how
much faster a configuration is compared to a single system with
one CPU using one directory server, the smallest configuration.
Because the SelectAccess policy server is the control
point for all authentication and authorization, our tests were structured
to push the policy server systems as close as possible
to 100% CPU utilization. The Result
Analysis section in the second
part of this white paper includes an analysis of the effect
of CPU utilization on the Login Scenario benchmark results.
Table 1: SelectAccess Login Performance
Scalability - 1,000,000 Users
|| 1 system, 1 CPU
1 system, 2 CPUs
|| 1 system, 3 CPUs
||1 system, 4 CPUs
2 systems, 2 CPUs
|| 2 systems, 4 CPUs
* - Look at Login
Performance Analysis in the second part of this report for
the computation of the logins/minute/total CPUs and at the hardware
configurations for more details on the test environment.
Figure 1 shows SelectAccess's Login
performance from Table 1 by server configuration.
SelectAccess Login Scalability for 1,000,000 Users
The Extranet Scenario measures the combination of one user authentication
and 10 authorizations for access to resources (these 11 operations
constitute one Extranet sequence). The Extranet Scenario, because
it uses a more realistic mix of operations than the Login Scenario,
provides a better basis for capacity planning purposes.
Table 2 shows the SelectAccess Extranet Scenario
performance for three configurations. The CPU utilizations shown
are the average for each type of server. CPU utilizations for the
Extranet tests are analyzed
in detail in the second part of this white paper.
Table 2: SelectAccess Extranet Performance
- 1,000,000 User Directory
|Total CPUs used
The benchmark results lead us to conclude that:
- Baltimore Technologies SelectAccess 3.1 delivers
the highest overall AuthMark Login and Extranet performance we've
measured to date.
- SelectAccess 3.1 achieves the highest Login
performance per policy/security server CPU and the best Login
and Extranet performance per CPU, for all CPUs used, that we have
measured to date.
- SelectAccess delivers outstanding performance scaling as CPUs
and policy servers are added to a configuration.
Mindcraft certifies that the results reported accurately represent
the performance of Baltimore Technologies SelectAccess 3.1 running
on Sun Enterprise servers configured as specified herein and as
measured by AuthMark benchmark.
Our test results should be reproducible by others using the same
test lab configuration, the same Sun server configurations, and
the same software configurations documented in this white paper.
The information in this publication is subject to
change without notice.
MINDCRAFT, INC. SHALL NOT BE LIABLE FOR ERRORS OR OMISSIONS CONTAINED
HEREIN, NOR FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES RESULTING FROM
THE FURNISHING, PERFORMANCE, OR USE OF THIS MATERIAL.
This publication does not constitute an endorsement
of the product or products that were tested. This test is not a
determination of product quality or correctness, nor does it ensure
compliance with any federal, state or local requirements.