DirectorySmart 4.7 with
Windows 2000 Active Directory
on ProLiant ML570 Servers:
Total Cost of Ownership and Performance

By Bruce Weiner
(PDF version, 83 KB)
September 19, 2001

Contents

Executive Summary
TCO
Extranet Summary
Login Summary
Conclusions
Certification
Extranet Details
Login Details
Configurations
iLOAD MVP
AuthMark

Disclosure

Compaq, Microsoft, and OpenNetwork Technologies sponsored the testing in this report. Mindcraft, Inc. conducted the performance tests described in this report at OpenNetwork Technologies in Clearwater, Florida.

Executive Summary

OpenNetwork Technologies DirectorySmart 4.7 with Microsoft Windows 2000 Active Directory running on Compaq Proliant ML570 systems delivers outstanding performance at low cost using 1,000,000-user and 15,000,000-user directories.

Mindcraft's tests show that DirectorySmart with Active Directory provides enterprise-class performance. Our test results include:

• For a 1,000,000-user directory, 309,227 Extranet operations/minute and 14,056 Extranet operations/minute/CPU, the highest we've measured by 19%.
  
  
• For a 15,000,000-user directory, 294,390 Extranet operations/minute and 13,381 Extranet operations/minute/CPU.

Performance is an important consideration when evaluating which product to buy. However, performance comes at at cost. So, in this paper we present not only performance measurements but also normalized performance metrics based on the total cost of ownership for each solution tested.

We used the Mindcraft iLOAD MVP™ test tool running the AuthMark™ Login and Extranet Scenarios to test performance of the access control and identity management solutions covered in this paper.

Total Cost of Ownership

Total Cost of Ownership (TCO) represents the costs to acquire, install, maintain, and use a solution. TCO includes the following costs:

• The cost of buying all of the hardware used including computers and networks (we excluded cable costs and the cost of the load generator systems, which are not part of the solution).
• The cost of licensing all of the software used.
• Training class costs.
• Hardware maintenance cost for the evaluation time period.
• Software maintenance cost for the evaluation time period.
• Personnel costs for training, installation, and time spent supporting the solution for the evaluation time period.

In this paper, we evaluate TCO for a three-year period. Also, we use two metrics to help you compare these results to others and to make purchasing decisions and project justifications: TCO/Performance and Annual TCO/User.

TCO/Performance is a price/performance metric that is useful for comparing performance results of different solutions because it normalizes performance based on the cost to own the solution. A lower TCO/Performance metric is better than a higher one because the solution with the lower metric costs you less per unit of performance than one with a higher metric.

Annual TCO/User is simply the annualized TCO (TCO divided by three) divided by the number of users in the directory. Using Annual TCO/User metrics based on the same number of users, you can make informed purchasing decisions and Extranet project justifications. Annual TCO/User should be used only with solutions that meet your performance requirements. A lower Annual TCO/User metric means that the solution costs you less per user each year than one with a higher metric. So a smaller Annual TCO/User is better.

There is a caveat to using the Annual TCO/User metric: it is affected significantly by the number of users in the directory being tested. Therefore, when comparing Annual TCO/User metrics be sure that they were based on the same number of users.

The TCO spreadsheet for these tests shows how we arrived at the TCO and calculates the TCO/Performance and Annual TCO/User metrics. We have set up the TCO spreadsheet so that you can enter your own costs and even evaluate the TCO/Performance and Annual TCO/User for other solutions.

Extranet Scenario Results Summary

The Extranet Scenario simulates customers or suppliers logging into a private Web site and obtaining information they are authorized to get. It measures the combination of one user authentication and 10 authorizations for access to resources (these 11 Extranet operations constitute one Extranet sequence). We report the total operations per minute. The Extranet Scenario, because it uses a more realistic mix of operations than the Login Scenario, provides a better basis for comparing access control and identity management solutions. You can find a more complete description of the Extranet Scenario in the detailed paper.

Figure 1 shows the Extranet Scenario performance of DirectorySmart 4.7 with Active Directory for tests with 1,000,000-user and 15,000,000-user directories. The X-axis shows the total number of CPUs used in all of the servers. We did not test a six-CPU configuration with a 15,000,000-user directory.

Figure 1: DirectorySmart 4.7 with Active Directory Extranet Performance

Table 1 shows the Extranet Scenario performance and TCO metrics. The per CPU performance is the highest we've measured to date by almost 20%. The 1,000,000-user directory test results show:

• Doubling the number of servers doubles performance, demonstrating this solution scales linearly.
• Doubling the number of Web and directory servers improves TCO/Performance more than 42%, giving you more bang for your money.
• Doubling the number of servers increases the Annual TCO/User metric only 15%, making this a cost-effective way to double performance.

Table 1: DirectorySmart 4.7 with Active Directory
Extranet Performance and TCO Metrics
(Lower TCO metrics are better)

The performance of the 15,000,000-user directory test is notable because Active Directory had to access disk for each user authentication, whereas all of the user authentication data was cached in memory for the 1,000,000-user directory tests.

The detailed paper provides additional information on these and other Extranet tests.

Login Scenario Results Summary

The Login Scenario simulates users requesting and receiving the first Web page at a protected Web site. It measures the combination of one user authentication and one authorization for access to a protected resource (called a Login). We report Logins/minute. The Login Scenario assumes that 10% of the user population in a directory logs in concurrently to use resources. So, for the tests with a 1,000,000-user directory, 100,000 users did a Login. For the test with a 15,000,000-user directory, 1,500,000 users did a Login. Login Scenario performance results should be considered best-case performance. The detailed paper explains the Login Scenario more thoroughly.

Figure 2 shows the Login Scenario performance of DirectorySmart 4.7 with Active Directory for tests with 1,000,000-user and 15,000,000-user directories. The X-axis shows the total number of CPUs used in all of the servers. We did not test a 32-CPU configuration with a 15,000,000-user directory.

Figure 2: DirectorySmart 4.7 with Active Directory Extranet Performance

Table 2 presents the Login Scenario performance and TCO metrics. For this scenario, the 1,000,000-user directory tests show:

• Linear scaling; performance doubles as the number of servers doubles.
• TCO/Performance improves more than 40% by doubling the number of Web and directory servers, giving you more bang for your money.
• Annual TCO/User increases only 17% when the number of servers doubles, making this a cost-effective solution for doubling performance.

Table 2: DirectorySmart 4.7 with Active Directory
Login Performance and TCO Metrics
(Lower TCO metrics are better)

For this scenario too, the performance of the 15,000,000-user directory test was impacted because Active Directory had to access disk for each user authentication.

Additional information on these and other Login tests is available in the detailed part of this paper.

Conclusions

These results lead us to conclude that:

• OpenNetwork Technologies DirectorySmart 4.7 with Microsoft Windows 2000 Active Directory on Compaq ProLiant ML570 servers sets the TCO/Performance and Annual TCO/User standards against which other Web access control and identity management solutions will be measured.
• OpenNetwork Technologies DirectorySmart 4.7 with Microsoft Active Directory delivers the highest AuthMark Extranet Scenario performance per CPU that we have measured to date.
• DirectorySmart with Active Directory on ML570 servers offers a low cost-per-user access control and identity management solution for 1,000,000 users, 15,000,000 users, and more.
• DirectorySmart with Active Directory delivers outstanding performance scaling.

Mindcraft Certification

Mindcraft certifies that the reported results accurately represent the performance of OpenNetwork Technologies DirectorySmart 4.7 using Microsoft Active Directory and Windows 2000 Server running on Compaq ML570 systems configured as specified herein and as measured by the AuthMark Benchmark.

Our test results should be reproducible by others using the same test lab configuration, the same server configurations, and the same software configurations documented in this white paper.

Mindcraft is a registered trademark of Mindcraft, Inc.

Product and corporate names mentioned herein are trademarks and/or registered trademarks of their respective companies.