Load tested equals road tested - Industry Trend or Event

Would you buy a car that hadn't been road tested for performance and reliability? Not likely. The same should hold true for mission-critical client/server applications. Just as Detroit bangs, burns, spins and speeds their vehicles before rolling them onto dealers' lots, IS groups and software publishers need to exercise their systems, too.

This means emulating real-life conditions in a realtime production environment, prior to deployment. Load testing tools, which are fast becoming the star performers of the testing market, are helping developers do just that.

Client/server systems that slow down or fail in high-usage situations cause numerous problems, from lowered productivity to costly loss of business. "With the growth of enterprise-wide client/server applications, deployment problems have become much larger," says Wing Wong, technical director at the consulting division of Coopers & Lybrand, Edison, N.J. "No one wants to publish systems that are less than foolproof."

Load testing becomes more important as applications grow in complexity and in number of users, according to Dick Heiman, senior analyst, application development tools at International Data Corp., Framingham, Mass. "The ramifications of [system] failure are more severe today. Large, mission-critical, network-based application systems that run stock exchanges, banks and airports cause real havoc if they go down," he says.

Product liability is another concern, says Heiman. Because more products have built-in software components, companies are open to lawsuits if those components fail to perform correctly. "Testing how the system behaves for five users is easy to do. Testing for usage by 5,000 people is extremely difficult," Heiman says.

The three-tiered architecture of the more sophisticated second-generation client/server systems adds to their vulnerability. These applications are more functionally rich and usually access and update more files than first-generation two-tier models. First-generation client/server systems typically did little more than provide information retrieval services against a relational database. Second-generation applications are far more crucial to the businesses they serve.

Another threat to the corporate environment, according to Zohar Gilad, director of enterprise product marketing at Mercury Interactive Corp., Sunnyvale, Calif., is the Internet. Controlled access to corporate systems by well-trained users is a thing of the past. "The Web's risk factor is exposure," says Gilad. "With the shift in how people are accessing information, users are no longer insulated by the technical and operations staff from system problems. They now can have direct access to a company's systems and data. Protecting corporate security and prestige requires improved testing."

Under controlled conditions, load testing simulates a system's real-world operational environment. It can be employed to determine what happens to a system if memory is exceeded, or if too many users log-on.

Load testing is performed by stressing all the components of a system -- hardware and software alike. These tests typically simulate heavy activity, including generating many calls to the DBMS, creating a lot of traffic on the network, and generating numerous transactions for the application server. As the load on the system increases incrementally, the load testers measure such "whole system" activities as throughput, responsiveness and resource utilization on the desktop, server, network and file systems.

Load testing software is an offshoot of GUI-based test tools. "Many customers recognized the possibility of a natural connection between automated functional testing and load testing," says Wong of Coopers & Lybrand. "IS groups that employed GUI test tools to automate functional tests began asking that load testing be tied in to those products." Load test tools take advantage of the automated capture and playback techniques of functional test tools. Utilizing functional test scripts, load testing tools create volume on the system.

Developers can perform load testing in two ways, says Wong: "One type runs by means of a remote control from a desktop. With this type of tool, an agent, operating from a designated master control workstation, controls all the scripts across all the desktops. The second type uses a proxy method in which the server is used to simulate activity from multiple workstations. Such tests might simulate SQL calls from a large number of workstations to test the performance of a DBMS." Each approach has advantages and disadvantages. Without physically testing every device on the system, the test is not a perfect gauge of real-world conditions. However, hooking up and testing hundreds or thousands of PCs to exactly duplicate the real environment is nearly impossible. IS departments have to be prepared to compromise. Nevertheless, the tests can significantly decrease the risks inherent in client/server deployment.

Developers frequently tack load testing on to the end of a project, just prior to systems deployment. However, mature IS groups may employ load testing at far earlier stages. These groups, with an eye toward "the big picture," use load tests to see if a proposed system ar- chitecture will be able to perform according to requirements. "Some developers test before development even starts, perhaps using a prototype of the system to see if a particular database management system can support the estimated number of transactions that will be processed by the application," Wong explains.

Testing prior to development can help determine how certain elements in the environment will behave before full resources are committed to the project. According to George Scott, director of client service at Softbridge Inc., Cambridge, Mass., these tests might answer such questions as: Is the proposed server big enough to handle all the PCs connected to it? Can the server manage the simultaneous log-ons of all users on the network? Will minimum performance requirements be met under heavy traffic conditions? Despite its practicality, pre-development load testing has downsides -- namely, expense. Creating a prototype's test scripts can take several weeks, and the cost of the effort may dampen senior management's enthusiasm. "Nonetheless," says Wong, "IS departments will do these tests for applications that handle a lot of users; for example, realtime transaction processing systems or systems with high transaction rates that go against a database or that transmit data on a low bandwidth."

Regardless of when load testing is performed, there's the expense of the test product itself. "Load test tools can be expensive, especially when figured on a per-license basis," says Wong. "With many products costing $2,000 to $3,000 per license, IS departments are hard-pressed to justify installing the tool on multiple workstations." Still, many would agree that the price of testing tools is a relatively small price to pay for reliable systems. "If you are dealing with big projects and big dollars, you need to look across the whole life of a system. From that perspective, investing in test tools, no matter what their cost, makes a lot of sense," says Mike Maslowski, director of technical architecture and customer systems at AT&T, Dallas. Maslowski opted to perform load testing to help get the company's Sales Troubleshooting Call Center, a mission-critical client/server system, under control.

According to Maslowski, the Sales Troubleshooting Call Center system had been built for the retail phone sales business. The application, developed with PowerBuilder, accessed a Sybase database from Windows-based clients. AT&T had leased the maintenance of the system to a third party, so no in-house group would claim responsibility for it. Meanwhile, the system's performance was rapidly deteriorating, Maslowski says.

"After acknowledging that the Sales Troubleshooting Call Center system was broken, we decided to initiate the process of its restoration," says Maslowski. "Our plan called for the use of both load testing and regression testing. We used [Dallas-based] AutoTester's AutoTester product for the regression tests, and Softbridge's Automated Test Facility [ATF] for the load testing."

AT&T chose ATF because it was the least "intrusive" tool of that genre, says Maslowski -- its footprint only minimally affected the outcome of the performance tests.

"Using ATF, we figured out the breaking points in the system and were able to fine-tune it," says Maslowski. By opting for load testing, he said, his team was able to get all users back online in time for the holiday season.

Load testing is equally important for commercial software developers whose livelihood depends on delivering credible products. At Software AG, Reston, Va., load testing helped stabilize its Natural product, a development environment for the Adabas DBMS. Running automated tests improves the efficiency of testing procedures as well as the quality of the application, says Lon Kight, the firm's advisory QA specialist. Software AG uses AutoTester products.

"We use AutoTester's Distributed Test Facility [DTF] because it allows the same test scripts to run across multiple platforms, testing the performance of the system," he says. "Cross-platform portability of tests is also important because Natural runs on a lot of very old legacy platforms -- on mainframes as well as on Unix boxes and PCs." With its ability to drive multiple test stations from a central console, Kight adds, "DTF ensures that Natural can survive real-world usage." For its part, Client/Server Laboratories, an independent testing company in Atlanta, primarily uses tools from Mercury Interactive when performing tests for its clients, according to Curtis Franklin, director.

"Although we use a variety of test products, we've begun to specialize with tools from Mercury Interactive," Franklin says. "One of our client companies is trying to decide on an architecture for future growth. This com- pany wants to find out how several of its candidate architectures will respond under high volumes, which can range from 500 to 1,000 users. We are using Mercury's LoadRunner to test these architectures, incrementally adding users to the system's load." Franklin says LoadRunner sits between the client and the server, capturing all SQL traffic. This ranges from the normal transaction set to delays, timings and other network activity.

Given the intricacies of client/server systems, it's likely that load testing products will increasingly penetrate IS groups moving to new platforms. "IS managers are desperate for any kind of certainty they can get," says Franklin. Automated testing, say proponents, can provide this kind of certainty, and will give IS developers confidence to rev up their client/server motors.

Representative testing tools

Company               Product       Comment

COPYRIGHT 1996 Wiesner Publications, Inc.
COPYRIGHT 2004 Gale Group