RSA Security Inc. sponsored the testing in this
report. Mindcraft, Inc. conducted the performance tests described
in this report at Sun’s test lab in Menlo Park, California.
We thank Sun for providing the systems
used for the tests and the support staff who helped
configure the servers.
RSA ClearTrust 5.0.1 delivers the highest
Login and Extranet performance per Authorization/Policy Server
CPU we've measured: 60,144 Logins and 221,980 Extranet operations
per minute per CPU
RSA ClearTrust's Login performance scales in two dimensionsas
the number of managed users increases from 1 million to 10 million
and as CPUs are added to the Authorization Server. RSA ClearTrust
sets a new Extranet performance standard for policy-server-based
identity and access management products. It achieved 392,117 Extranet
operations per minute and 12,649 Extranet operations per minute
per total CPUs in all of the servers used in the test.
Mindcraft® tested RSA ClearTrust 5.0.1 running on Sun
Fire servers. For these tests, we used Mindcraft’s iLOAD
MVP™ test tool running the AuthMark™
Login and Extranet
The Login Scenario simulates the type of load commonly seen at
Web portals where users access protected resources. The Login Scenario
assumes that 10% of a portal's user population logs in concurrently
to use protected resources. These tests were done using both
a 1,000,000-user and a 10,000,000-user directory with 100,000 and
1,000,000 users logging in, respectively. The Login Scenario
measures the combination of one user authentication and one authorization
for access to a resource (called a Login).
The RSA ClearTrust Authorization Server, which performs functions
similar to Policy Servers in other products, is the control point
for all authentication and authorization. Our tests were structured
to push the Authorization Server systems as closely as possible
to 100% CPU utilization. Table 1 summarizes
the Login Scenario performance for RSA ClearTrust. It also shows
normalized metrics based on the total number of CPUs in the configurations
tested and the number of CPUs in the single Authorization Server
that was used for these tests. More configuration
details are in the second part of this white paper.
The Scaling Factor/Total CPUs in Table 1 shows how much faster
on a per-CPU basis a configuration is than an Authorization Server
with one CPU, the smallest Authorization Server configuration. It
is computed by dividing the Logins/Minute/Total CPUs for a configuration
by that for the single-CPU Authorization Server configuration. A
metric close to 1.0 indicates linear scaling. A metric above 1.0
means that the configuration provides better than linear scaling
while a metric significantly below 1.0 means that the configuration
scales less than linearly.
For the 1,000,000-user directory, the Scaling Factor/Total CPUs
metric shows that performance scales linearly as the Authorization
Server is expanded from one to four CPUs. For the 10,000,000-user
directory, per-CPU performance also scales linearly.
Table 1: RSA ClearTrust Login Performance
1 Million and 10 Million Users
Figure 1 shows RSA ClearTrust's Login
performance from Table 1 by Authorization Server configuration and
RSA ClearTrust Login Scalability for 1 Million and 10 Million Users
The second part of this white
paper provides a detailed analysis
of the benchmark results.
The Extranet Scenario measures the combination of one user authentication
and 10 authorizations for access to protected resources (these 11
operations constitute one Extranet sequence). The Extranet Scenario,
because it uses a more realistic mix of operations than the Login
Scenario, provides a better basis for capacity planning purposes
and for evaluating how products will perform when deployed.
Table 2 shows the Extranet Scenario performance
of RSA ClearTrust with a 1,000,000-user directory. As with the Login
Scenario, the Extranet Scenario simulates the activities of 10%
of the number of users in the directory, which in this case is 100,000
The Scaling Factor/Total CPUs in Table 2 shows how much faster
on a per-CPU basis the two-CPU Authorization Server configuration
is than the one-CPU configuration. It is computed by dividing
the Extranet Operations/Minute/Total CPUs for the two-CPU configuration
by that for the one-CPU configuration. This metric clearly shows
that performance scales better than linearly as the Authorization
Server is expanded from one to two CPUs. This more-than-linear scaling
occurred in part because we were able to saturate the Authorization
Server in the one-CPU configuration while having excess Web server
CPU capacity. In other words, we could have used fewer Web server
CPUs and achieved the same level of performance for the one-CPU
Authorization Server configuration. In addition, we did not have
enough client performance to saturate the CPUs in the two-CPU Authorization
Server configuration, which is not uncommon given the exigencies
of performance testing.
The RSA ClearTrust Extranet performance shown in Table 2 demonstrates
the excellent overall performance, per-CPU performance, and scalability
that it delivers.
Table 2: RSA ClearTrust Extranet Performance
- 1 Million Users in the Directory
(20,180 + 201,800)
(35,647 + 356,470)
These AuthMark Benchmark results lead us to conclude
- RSA ClearTrust 5.0.1 delivers the highest Login
performance per Authorization (Policy) Server CPU that we've measured
so far, 60,144 Logins/minute/Authorization Server CPU.
- RSA ClearTrust 5.0.1 outperformed all other policy-server-based
identity and access management products we tested with the Extranet
Scenario both in total Extranet operations per minute (392,117)
and in Extranet operations per minute per total CPUs (12,649).
- RSA ClearTrust 5.0.1 delivers outstanding linear performance
scaling as Web servers and Authorization Server CPUs are added
to an installation.
- RSA ClearTrust 5.0.1 provides excellent, predictable
performance for moderately sized communities of 1,000,000 users
as well as for large communities of at least 10,000,000 users.
Mindcraft certifies that the results reported herein accurately
represent the performance of RSA ClearTrust 5.0.1 running on Sun
Fire servers configured as specified herein and as measured by the
Our test results should be reproducible by others
using the same test lab configuration, the same Sun
server configurations, and the same software
configurations documented in this white paper.
The information in this publication is subject to
change without notice.
MINDCRAFT, INC. SHALL NOT BE LIABLE FOR ERRORS OR OMISSIONS CONTAINED
HEREIN, NOR FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES RESULTING FROM
THE FURNISHING, PERFORMANCE, OR USE OF THIS MATERIAL.
This publication does not constitute an endorsement
of the product or products that were tested. This test is not a
determination of product quality or correctness, nor does it ensure
compliance with any federal, state or local requirements.