Object-oriented unit testing - HP's Waltham Div - technical

Object-Oriented Unit Testing

ALTHOUGH OBJECT-ORIENTED ENVIRONMENTS are being used more frequently in software development, little has been published that addresses object-oriented testing. This article describes the processes and experiences of doing unit testing on modules developed with an object-oriented language. The language is C+ + and the modules are for a clinical information system. Because the system must acquire real-time data from other devices over a bedside local area network and the user requires instant information access, extensions were made to the language to include exception handling and process concurrency. We call this enhanced version Extended C+ +. Test routines were developed and executed in an environment similar to that used in development of the product. This consists of an HP 9000 Series 300 HP-UX 6.01 system.

Unit Testing

Unit testing is the first formal test activity performed in the software life cycle and it occurs during the implementation phase after each software unit is finished. A software unit can be one module, a group of modules, or a subsystem, and depending on the architecture of the system, it is generally part of a larger system. Unit tests are typically designed to test software units, and they form the foundation upon which the system tests are built. Since software units and unit tests are fundamental entities, unit testing is critical to ensuring the final quality of the completed system.

The unit testing process involves test design, construction, and execution. The test design activity results in a test plan. Because the primary intent of unit testing is to find discrepancies between unit specifications and the coded implementation, the unit specification is the primary reference for the test plan. Test construction involves building the test cases based on the test plan, and test execution involves performing the tests and evaluating the results.

Both structural (white box) testing and functional (black box) testing techniques are used in unit testing. Since structural testing requires intimate knowledge of the design and construction of the software, unit testing requires intense developer involvement in the process.

Objects

An object is the fundamental building block in an object-oriented environment and it is used to model some entity in an application. For example, in an office automation system, objects might include mail messages, documents, and spreadsheets. An object is composed of data and methods. The data constitutes the information in the object, and the methods, which are analogous to procedures and functions in non-object-oriented languages, manipulate the data. In most applications, there are many objects of the same kind or class (e.g., many mail messages, devices, etc.). C+ + defines the data and methods for these similar objects in a data type called a class. Each object in an object-oriented language is an instance of a particular class. Also in C+ +, a data item is referred to as a member and the methods, member functions.

One of the main differences between object-oriented and procedural languages (non-object-oriented languages) is in the handling of data. In a procedural language environment such as Pascal, C, or Fortran, system design is based on the data structures in the system, and operations are performed on data passed to procedures and functions. The primary data items are typically global and accessible to all the modules in the system. In an object-oriented environment, the object's internal data structures and current values are accessible only to the methods within the object. The methods within an object are activated through messages passed from other objects. The messages indicate the method to be activated and any parameters required.

Fig. 1 illustrates these differences in architecture between object-oriented systems and procedural-language-based systems. In Fig. 1A, to move a graphical item (objx) from one location to another, the message move__item(xnew,ynew) is sent to the object instance objx to perform the operation. The current location and geometric characteristics of the item are contained in the data structures of objx. The methods in objx will handle the transformation and translation of the item to a new location. Fig. 1b depicts the same operation in a procedural language environment. The graphical items's data structure and current values are kept in a global data structure which is accessible to all the modules in the system.

Objects and Unit Testing

The issues related to objects and unit testing include:

* When should testing begin? In a procedural language environment, a complete unit may not exist until several functions or procedures are implemented. In an object-oriented environment, once a class has been defined and coded, it can be considered a complete unit and ready for use by other modules in the system. This means that unit testing must be considered much earlier in an object-oriented environment.

* What testing techniques should be used? Since the paradigm of object-oriented programming emphasizes the external behavior of data abstractions rather than the internals, one would expect to employ only black box, functional testing techniques. However, a more robust testing structure employing complete path testing is actually needed.

* What should be tested? In an ideal situation, the answer to this question would be that all classes should be completely path tested, particularly for critical application systems. However, the resources required to meet this goal may be substantial and, in working towards it, trade-offs are likely to be made nonetheless, refinements can be added to the testing process that simplify labor intensive phases and improve chances that a minimal set of tests will be executed.

* Who should do unit testing? To answer this question, we need to consider what is being tested and the expertise required of the tester. Remember that units are typically modules that eventually become part of a larger system and only the developers know the detailed internals of the units they are responsible for building. As a result, an independent tester or a developer who is not involved in the design and generation of code for a specific class may find it difficult to perform adequate testing on that class. For example, a developer may design a data base class which is intended to make it easier for a user to perform transactions in a data base. The methods within the data base class are responsible for performing the data base interface tasks. An independent tester who is unfamiliar with the way in which these low-level functions work would certainly be ineffective in testing the internals of this class.

In the clinical information system, knowledge of Extended C+ + was sufficient to become an effective tester for certain classes in the system. This was because of the formulation of generic classes. A generic class in the clinical information system is a class that provides general functionality. It can be considered an extension of the language's built-in data types that fills a utilitarian purpose for other components of the system. Strings and linked lists are examples of objects that provide such universal functionality.

To build on this generic concept, parameterized type classes were introduced. Parameterization permits a general definition of a class to be extended to create a family of type-safe classes, all with the same abstract behavior. For example, suppose we design a class called Array which contains pointers to some object. Through parameterization, we can extend this class definition to create arrays that point to characters, arrays that point to strings, or arrays that point to any other type of object (Fig. 2). The testing of a parameterized type class can provide a high level of reliability for a growing family of similar classes. From the experience gained in testing generic classes, we have developed an approach to the testing of other C+ + classes.

Test Process

The tasks associated with the testing process for objects are the same as for regular unit testing: design, construction. and test execution.

Design. during the design phase, the tester determines the test approach, what needs and does not need to be tested, the test cases, and the required test resources. The inputs required to conduct the design phase for objects include:

* The header and source files of the target class (the class being tested), and a well-defined specification of the class. An example of a class specification is shown in Fig. 3.

* An analysis of the effects of inheritance on the target class. When a class uses another class as a base to build additional functionality, it is said to be derived from that class and consequently inherits data and methods from the base (parent) class. If the target class is derived, we want to know if the base class has been thoroughly tested. Provided that the functionality of the base class has been proven, any member function of the target test class that leverages directly from a base class member function will require minimal testing. For example, the specification in Fig. 3 shows that String is derived from a parameterized class called Sequence. The functions that String inherits from Sequence (AddFirst, Capacity, etc.) require only a basic functionality test.

* The cyclomatic complexity metric.sup.5 of the individual member functions belonging to the target class. The complexity measure and its accompanying testing methodology play a key role in the implementation of this test strategy. Through their use, we can ensure that all the independent paths in the target member functions are tested. Fig. 4 shows an example of path test cases for the member functional Lowercase. In our project, the predicate method of calculating cyclomatic complexity has been built into the Extended C + + parser.

* A hierarchy or structure list which shows member function dependencies. In simple terms, what member functions call what other member functions of this class? Private member functions, which are not accessible to the end user directly, should also be included in this list. For example, in Fig. 5, the function operator + performs its task by invoking the Append function, which indicates that Append should be tested first.

* The signals or exceptions that are raised (not propagated) by each function. Extended C + + includes linguistic support of exception handling, which permits a special kind of transfer of control for processing unusual but not necessarily erroneous conditions. These signals should not be confused with HP-UX operating system signals. Signals are defined for the various member functions in a class specification. For example, the specification for String indicates that the member function Delete-String raises a signal called InvalidStarthIndex if the StartIndex parameter passed to the member function is not valid. The last step of the design phase is to determine the approach to use of verify the test results. There are a number of options in this area. One approach is to print out the expected results for a test case and the actual results generated by the target class test to two different files. At the end of the test, the two files can compared using the standard UNIX tool diff (see Fig. 6). A second option for results verification uses similar ideas, but may require less actual programming time. An expected results file can be constructed by hand and the file can be used for comparison with actual target class output. If these two approaches prove impractical because of the behavior of the class being tested, a third alternative might be to include the expected observations in a test plan using the class pecification as a basis for deriving these observations.

Fig. 7 shows an excerpt from the test plan for the class String. A test plan is the culmination of the test design process, and in addition to guiding test activities, it is an excellent respository of information regarding what was done to test an object.

Construction and Execution. The strategy for developing test cases for execution is to determine all the paths in a module that require test coverage, and then to create test cases based on the class specification (black box approach) and certain features in the code (white box approach). The white box strategy is based on the structured testing methodology resulting from McCabe's work (see article on page 64 for a discussion of the use of the McCabe complexity metric in our division). In this methodology, test cases are created to execute each decision path in the code. In the clinical information system, except for paths that contained code for exception handling, test cases were written to ensure complete path coverage of each member function. Exception handling situations were dealt with separately because they disrupt the normal control flow of a program. Based on the class classification and source code, test cases designed to ensure path coverage were derived using the other well-understood methodologies of equivalence partioning and boundary-value analysis.

In creating test cases for valid equivalence classes, realistic input values for the member functions were preferred over those that lacked relevance from an application standpoint. For example, if the primary use for our sample String class is to hold a single line if information on an electronic index card, we might expect it to hold, on average, 12 to 50 characters. Our test case would be to create a string of 40 characters rather than 140.

Boundary-value analysis dictates that tests be built that create objects of extremes. Instances of null strings (size = 0) should respond as any non-null string would unless the specification states otherwise. Clearly, a null string appended with the value abc should yield the same result as the string abc appended with a null value. At the other extreme, tests should exist to stress objects (usually in size) beyond all expectations of normal use. For example, in HP-UX, main memory is managed in pages of 4096 bytes. Therefore, it should be valid to create a string that holds 4097 characters.

Tests to invoke exception handling capabilities were also included in class test suites. Boundary-value conditions were used to invoke these facilities. For example, if an exception is encountered when we index beyond the legal boundary of a string, the test case invokes the exception by trying to access the character just past the end of the string, not ninety-nine characters past it. Special care must be taken in coding exception test cases, because if a signal raised by a member function is not handled correctly, an aborted test program may result.

There are other areas for test cases that do not show up using the structured technique. For example, the effects of implicitly and explicitly invoking a class's constructor and destructor functions should be examined for consistency. Initialization and casting operations should also be tested. In addition, defects have been discovered by applying associativity rules to member functions. That is, if string s1 is null, and string s2 is not null, s1 > s2 should yield the same results as s2 < s1. In addition, the use of the object itself as a member function input parameter proved valuable in uncovering subtle implementation errors. For instance, given s1 is a string, the test s1.Append(s1) becomes a legitimate and creative way of triggering certain test conditions. Much of this type of testing can be integrated into standard testing without creation of separate tests.

Results

The methodology presented here was applied to testing several generic classes after the development group had completed their testing using black box testing techniques. The results show the shortcomings of strict black box testing. Even though development group testing was extensive and appeared to be thorough, defects were still uncovered.

Defects were found in each of the generic classes tested. The number of defects found seemed to be related to the composite (total) complexity of all of the class member functions and more directly to the number of noncomment source statements (NCSS) contained in the source and include files. The general relationship of complexity to defects is shown in Fig. 8a, and the correlation between defects and the NCSS of each class is shown in fig. 8b. Each point represents a generic class. On average, a defect was uncovered for every 150 lines of code, and correspondingly, the mean defect density exceeded 5.1 per 1000 lines. Only the code contained in the source and include files for each class was counted for this metric. Code from inherited functions was not considered. These defect rates pertain to a small set of actual product code produced during the early stages of development. Another interesting relationship was observed when the NCSS values of source and test code were compared (see Fig. 9).

Conclusion

There is a cost associated with class testing. A significant investment of time is required to perform the testing proposed here. Assuming testers are already competent with the object-oriented environment, they must acquire familiarity with McCabe's complexity concepts as well as a basic understanding of the class tested. Because testing so far has taken place concurrently with development, time estimates for the testing phase have been somewhat inconsistent and do no yet suggest any clear conclusions. Fig. 10 summarized the metrics we have collected thus far. (The classes are listed in the order they were tested).

In the object-oriented environment, objects and their definitions, rather than procedures, occupy the lowest level of program specification. Therefore, it is necessary to focus on them when implementing a thorough test methodology. Practices used in testing traditional procedural systems can be integrated in the approach to object-oriented testing. The main difference we have found so far is that each object must be treated as a unit, which means that unit testing in an object-oriented environment must begin earlier in the life cycle. Through continued collection of the class metrics and test results, we hope to gain more insight and continue to improve our object-oriented unit test efforts.

COPYRIGHT 1989 Hewlett Packard Company
COPYRIGHT 2004 Gale Group