Evaluating performance of the purchasing department using data envelopment analysis

The purchasing department is one of the most difficult functional areas to evaluate. A recent approach to the evaluation of purchasing is to compare that department's performance with the performance of other firms in the industry. In the past, purchasing executives could not hope to do that since aggregate performance data had not been available and, if it had been, current methodologies do not provide an effective comparison of overall performance.

The purchasing benchmark project of the Center for Advanced Purchasing Studies (CAPS) has begun to gather the required data from the leading firms in selected industry groups. Senior purchasing executives have met as an ad hoc committee in order to determine what specific data should be collected as a basis for benchmarking purchasing performance in that industry. For each of the industry groups that have been benchmarked, the measurement criteria have been established. CAPS has collected the aggregate data, and firms can compare performance against a standard for each purchasing performance measure.

But the measurement and evaluation is incomplete. The firm cannot tell where it stands in relation to the other firms, but only where it stands with respect to a composite standard for each individual purchasing performance measure. In addition, there is no aggregate measure of overall purchasing performance with which a firm can compare performance with other industry members. Overall performance indices have been developed. As an example, the Six Sigma Barometer methodology,1 developed by the Digital Equipment Corporation using benchmarking data from the semiconductor industry, constructs a composite index of overall performance. The drawbacks to such weighted point methods stem from the fact that weights are based upon perceptions rather than objective data, and that each firm must use the same weighting factor for each of the performance measures. The weight is a compromise of the perceived importance of each measure by the companies involved and the evaluators. One weighting scheme is derived that must account for the whole range of possible organizational goals, functional responsibilities, and types of purchases. These factors have been shown to affect the relative importance of performance measures.2

As these organizational characteristics vary among companies, the resulting performance index will inaccurately reflect true efficiency in purchasing performance. As a result, little insight is gained into the magnitude of possible gains in efficiency at each of the purchasing units. To account for these deficiencies and to allow for truly effective performance comparisons, an improved purchasing performance evaluation model is desired.

CHARACTERISTICS OF DEA

Data Envelopment Analysis (DEA) is a linear programming model developed by Charnes, Cooper, and Rhodes3 that measures relative productive efficiency, or productivity, of each member of a set of comparable producing units. These units are termed Decision Making Units (DMUs). It is designed to measure relative efficiency in situations where there are multiple inputs and outputs, and there is no obvious objective way of aggregating either into a meaningful index of productivity efficiency. The primary advantage of DEA is that it will establish a composite index of overall performance while utilizing weights that are allowed to vary among firms. This will allow the system to account for the differences in the types of tasks and responsibilities of each purchasing department, the differences in supplies and services procured by each department, and differences in organizational goals and objectives.

A DMU can be any organization, agency, program site, or functional department such as a purchasing department. The relative efficiency of each unit is defined as the ratio of its total weighted output to its total weighted input. Problems arise when trying to assign weights to variables which have no inherent value or price. The essence of DEA is that it allows each DMU to select the weights that maximize its own efficiency. Care must be taken when interpreting these weights, since they are not economic values and do not reflect the relative importance of the various inputs and outputs.4

An important part of DEA is the identification and measurement of the inputs and outputs. One of the primary advantages of DEA over other techniques is that each input or output can be measured in its natural physical units (e.g., number of purchase orders placed or dollar value of supplies purchased). As a result, there is no need for a weighting system that reduces those units into any single unit measure.

There appear to be three ways in which the results of DEA can have significance to the manager. First, it produces the efficiency of each DMU relative to other units in the sample. This identifies the DMUs that are in greatest need of attention and also gives an indication of the extent to which improvements may be possible. This will allow managers to focus their attention on the least efficient of the group. Second, when a DMU is less than 100 percent efficient, DEA indicates a subset of perfectly efficient DMUs called an Efficiency Reference Set (ERS), and associated multipliers. DMUs are then compared against the other units that are "most like" itself. This information can be used to form management strategies.

The solutions do not generally result in input or output values that are observed in actual DMUs. However, the input and output values of the ERS and multipliers allow the analyst to construct values for a hypothetical DMU that is efficient. This unit is called a Hypothetical Comparison Unit (HCU). This HCU uses less of each input and produces at least as much output as the inefficient DMU, and would be rated perfectly efficient if it used the same weights used by the inefficient DMU. The difference between the input and output values of this hypothetical unit and the actual values of the DMU under evaluation is said to be evidence of its inefficiency.5

Finally, DEA can also have significance in the area of secondary analysis. This involves the attempt to explain variations in efficiency among the different DMUs in terms of the various statistical models that describe those variations. For example, factors such as ownership, firm size, area of operation, or age could be studied as independent variables with efficiency as the dependent variable. Regression analysis, analysis of variance, analysis of covariance, or clustering can be used to define model parameters.6

Consider the case where there are five DMUs using the same two inputs to produce the same output. To ease interpretation, input and output levels can be normalized by dividing them through by the output levels. This gives the rates at which the inputs are used to produce one unit of output. (Such normalization is not required to perform an analysis, but is done only to aid initial understanding.) Normalized values for the production function are shown in Table 1.

These normalized input levels of each DMU are plotted in Figure 1. In this figure each DMU is represented by a point, whose coordinates are the normalized input levels.

When minimizing inputs with respect to outputs, any unit that is situated lower and to the left of another unit is judged to be more efficient. The reason for this is that the more efficient unit is producing the same unit of output while using lower input levels. It can be seen that, theoretically, the origin (0,0 point on the graph) would be the ultimate goal of each DMU as it strives to improve efficiency. The solid segmented line, ABCD, is called the "unit isoquant." This represents the possible combinations of both input amounts that will produce a unit of output in an efficient manner, relative to others in the set.

This is also the efficient frontier. Every DMU is either on the frontier, or on a line from the origin to the DMU that crosses the frontier. The efficient frontier is a piece-wise linear curve that lies as far up and to the right as possible, while satisfying the condition that every DMU lies on or above this line. In other words, it is the lower envelope of all the DMUs. The efficient frontier surrounds the data points corresponding to the DMUs. It is that image that gives DEA its name.7

The efficiency of a DMU can be defined in terms of its position relative to the efficient frontier. An inefficient DMU-DMU Fr-can be used as an example. The input values of DMU E, three units of input one and three units of input two, places it to the northeast of the efficient frontier and suggests that DMU E is inefficient. As can be seen, a line from the origin to DMU E, line OE, crosses the efficient frontier.

In evaluating DMU E, DEA attempts to ascertain the minimum amount of each input that the evidence shows it should have used to produce one unit of output. DMU E's efficiency, relative to the efficient frontier, is calculated by the relative lengths of the line segments OE' and OE (i.e. OE'/OE). In this case, .78 represents the ratio of the lengths, and DMU E should produce one unit of output 78 percent or less of its current input level.

Due to the method of constructing the efficient frontier, no point can lie between the frontier and the origin. Therefore it is evident that no ratio can exceed one. Points B and C, and all points on the line between them, have an efficiency ratio of one and are 100 percent efficient. This implies that no other DMU, or combination of DMUs, can produce more output except by either using more of at least one of the inputs or (in the case of multiple outputs) producing less of one of the outputs.8 It should be noted that the 100 percent efficient units are not absolutely efficient. Improvements most likely are still possible. These units are only relatively efficient when compared to the other DMUs under consideration.

To summarize, efficiency equals one if the point lies on the line, and the unit is inefficient if the point lies above and to the right of the efficient frontier. The efficient point, E', is the Hypothetical Comparison Unit (HCU) referred to above. The input mixture ratio of this point is the same as that for DMU E. In other words, both of these units use the same production technology, and the HCU serves as a role model for the inefficient DMU. The HCU is actually a weighted average of the DMUs included in the efficiency reference set, in this case DMUs B and C.

A graphical formulation of this problem is not possible when there are more than three variables present. The previous discussion will now be extended to the case where there are n DMUs, m inputs, and s outputs. The decision variables in DEA are the unit weights to be attached to each input and output by DMU^sub j^. One fractional linear program is formulated for each DMU. The objective function is the ratio of total weighted output divided by the total weighted input. The unit being evaluated under each linear program is designated as DMU^sub o^.

DEA AND PURCHASING

This is the first application of DEA to the evaluation of purchasing performance. It used data from the petroleum industry, although since its introduction in 1978, DEA has been used or investigated for its potential use in such diverse fields as education, recreation, electricity production, law enforcement, health care, vehicle maintenance, highway maintenance, international logistics, the airline industry, and banking. A DEA evaluation model was developed and used to compare the performance of purchasing departments of major firms within that industry. Eighteen firms provided the data required to develop a DEA model for the evaluation of purchasing performance. Those eighteen firms totalled $454.6 billion in 1991 sales, for an average of $25.2 billion per firm. Sales ranged from a low of $794 million to a high of $116.9 billion. Specific data provided by the firms, and the identity of the firms participating in this study, will not be revealed.

A general model of the purchasing function in the petroleum industry (Figure 2) was used to develop the DEA model. Eleven output and seven input measures (specific measurement factors identified in the CAPS benchmark ratios) were considered to represent the major inputs and outputs of the system.

Two outputs and four inputs were selected to measure the relative efficiency of the eighteen petroleum industry purchasing departments. Outputs measured were total purchase dollars spent by the purchasing department (PURDLRS) and percent of total company purchase dollars handled by the purchasing department (PCTTOT). Total company purchase dollars is the total of purchase dollars spent by the purchasing department and purchase dollars spent by others in the firm. (Often the purchasing department does not handle purchases of raw petroleum products or chemicals.) Inputs measured were total purchasing department operating expense (OPEXP), total number of purchasing professional employees (PROF), total number of administrative purchasing employees (ADMIN), and total number of active suppliers (SUPPLIERS). A supplier is considered active if it has provided supplies or services within the past year.

In the selection process, eleven outputs and seven inputs were considered for use in the DEA model. Firms were asked to rank order the importance of potential inputs and outputs relative to their relationship to purchasing department success. Generally, the top-ranked factors were included in the model. Nine of the outputs and three of the inputs were not used. One problem with some of the measures not used was the availability of reliable data for all of the participating firms. Not all of the firms kept or reported data on several of the measures, which could have been valuable indicators of purchasing performance, thus rendering these measures as inappropriate for use as a DEA measure. Therefore, although these measures may be of value to the individual purchasing department, they have no value in comparison of performance across firms until more complete data are available.

DEA EVALUATION

The DEA analysis identified six petroleum firms that were rated as one hundred percent efficient (1.0 rating). This efficiency is relative to the other firms in the sample. These firms could be identified as "best-in-class" for the petroleum industry sample. (It is not yet known whether DEA could be used to identify "best-in-class" across different industry groups.) Ratings of other firms ranged from a low of .12 to a high of .77. DEA ratings of overall purchasing performance for each of the firms are shown in Table 2. Each firm was assigned an arbitrary three letter code in order to protect its identity.

In addition to the DEA rating of overall purchasing performance, Table 2 also identifies which firms comprise the Efficiency Reference Set (ERS) for each of the firms rated inefficient. These firms, and their associated multipliers, will form the Hypothetical Comparison Unit (HCU) that is used for comparing performance with the efficient frontier.

As with weighted point methods, DEA provides a comprehensive evaluation of overall purchasing performance. This provides management with an advantage over comparing performance only on individual measures. DEA also identifies a subset of the "best-in-class" performers. In DEA this is not an arbitrary classification. These firms actually have achieved that performance level, so it is known with certainty that it can be reached.

The reason DEA can identify "best" in this fashion is that it does not depend upon the average of performance in the group in order to derive either its ratings or its definition of "best." It uses the extreme performers in total productivity (ratio of weighted output to weighted input) to establish these values. It should be re-emphasized that it does not establish an absolute "best," but only relative to the other purchasing departments in the group.

Another difference in the information DEA provides is that aggregate ratings of performance use a flexible weighting scheme. DEA establishes the weights that each firm uses to combine the individual inputs and outputs into a single index of relative efficiency, allowing each firm to be evaluated based upon what it does best. As a consequence, as the relative importance of each of the inputs and outputs varies, the weights developed by DEA for the purchasing department may also vary. In this way, firm characteristics (such as functions, responsibilities, and goals/objectives) may be effectively accounted for.

Additionally, DEA provides insights into the magnitude of potential improvement that other techniques cannot. Management can identify which inputs and outputs that may be altered, and by how much, in order to improve overall relative efficiency. This can be done by comparing the department's input and output values with those of the DEA generated HCU. The HCU is comprised of a set of input and output values, which are a derived composite of the performance of the firms in the ERS (see Table 2). Table 3 is an example of a comparison with the HCU for one of the inefficient firms, GGG.

This analysis will enable the purchasing executive to determine exactly which input or output levels can be changed, and by how much, in order to improve relative efficiency. For example, in the case of firm GGG, if all input levels are reduced to the amounts indicated, then the relative efficiency rating will improve to 1.0.

This HCU is derived as a linear combination of the input and output values from the firms in the ERS. DEA identifies the firms within the ERS and provides multipliers (the variable value from the linear program's dual) to be used in computing the HCU. Table 4 provides an example of how the HCU is constructed for firm GGG. Recall from Table 2 that firms AAA and DDD were identified as the ERS. (Example for OPEXP: .163126 x .5 + .016874 x 4.413 = .156)

Some management judgment must be used when interpreting these hypothetical numbers. It is possible for the DEA program to produce numbers that are not feasible. For example, in some cases the HCU will indicate that an increase in the output PCTTOT will increase relative efficiency. That PCTTOT figure is sometimes greater than 100 percent and it would not be possible to increase the variable to that level. Also, it would be difficult to employ 1.665 purchasing professionals.

The DEA model provides management with additional information that may help in selecting a course of action for improving performance. DEA assumes that any point on the efficient frontier (see Figure 1) is feasible. This indicates that efficiency can be reached using combinations of inputs other than those indicated by the HCU. One approach is to simply multiply the actual input values by the DEA efficiency rating. This will give a combination of inputs that will reach maximum relative efficiency.

Other combinations of inputs and outputs can also be developed that may be more compatible with the characteristics of the business. In other words, it may be impossible for one reason or another to reduce certain of the input values. Or it may be easier to reduce some inputs than others. DEA makes it possible to explore those other combinations. The weights developed by DEA (not shown here) allow the firm to concentrate on altering specific input or output values. These weights show the marginal productivity of the individual inputs.

CONCLUSIONS

In summary, the analysis of petroleum firms has shown that there are some striking differences between DEA and existing evaluation methods. DEA provides additional management information that enhances the decision-making process. Purchasing executives can now evaluate more alternatives, and immediately see the effects of potential changes.

The primary strength of DEA lies in two areas: (1) a flexible weighting scheme that can accommodate the varying importance of evaluation factors, and (2) an ability to provide more useful information to managers for improving performance. DEA provides a single index of overall purchasing performance that is flexible enough to account for the different firm characteristics that affect the required weights. DEA does a good job of identifying inefficiencies; however, it does not provide a means for distinguishing among the firms rated efficient.

One potential problem in attempting to implement a DEA system is the availability of data. This research was possible because the benchmarking project at the Center for Advanced Purchasing Studies had begun to collect aggregate purchasing data from firms within industry groups. As a consequence, most of the petroleum firms in this study had already begun to collect the benchmarking type data on a regular basis. This type of DEA evaluation system is not possible unless a group of comparable firms collects the data consistently and on a regular basis. Most firms would be unwilling to release such performance data unless an independent party monitored the DEA system and assured anonymity.

Interpretation of the weights established by DEA must be done carefully. Management should not interpret the weights to indicate a value, or measure of importance, of the individual performance measure (input/output). DEA establishes the weights in a purely mathematical fashion. The weights allow each firm to maximize its efficiency rating subject to the constraints of the problem. Thus, DEA calculates a technical efficiency without regard to any potential social or economic value that each performance measure may have. It may be possible in future research on DEA in purchasing to establish a range of possible weights for each measure. DEA would then establish the technical weights within the established value bounds.

One final implication for management deals with implementing DEA recommendations. DEA is only a tool. It has no knowledge of the business environment of the firm. It cannot use judgment in calculating the evaluation. It may be possible that certain values derived by DEA are impossible or undesirable to achieve in reality. In addition, management may choose corporate or departmental strategies that require the purchasing department to operate in a relatively inefficient mode. For example, the firm may have embarked on a quality improvement program that will require increases in the number of purchasing professionals employed. Management has made a conscious decision on the strategy, and the DEA results indicating inefficiencies may not be as meaningful to that firm. Although DEA is not a substitute for sound management judgement, it can provide management with valuable information for resource allocation decisions and insights into improving departmental performance.

NOTES

1Henry A. Malec, John D. McClean, Timothy B. Crain, and Janet M. Goulet, Benchmarking Purchasing in the Semiconductor Industry With the Six Sigma Barometer (Tempe, Ariz.: Center for Advanced Purchasing Studies, 1991).

2Chiang-nan Chao, "Purchasing Performance-Views of Purchasing Managers, Buyers, and Internal Customers for Different Industries," (doctoral diss., Arizona State University, 1989).

3A. Charnes, W. W. Cooper, and E. Rhodes, "Measuring the Efficiency of Decision-Making Units," European Journal of Operational Research 2 (1978): 429-444.

4Thomas R. Sexton, "The Methodology of Data Envelopment Analysis," Measuring Efficiency: An Assessment of Data Envelopment Analysis (San Francisco: Jossey-Bass, 1986).

5R. Banker, A. Charnes, W. W. Cooper, John Swarts, and D. A. Thomas, "An Introduction to Data Envelopment Analysis With Some of Its Models and Their Uses," Research in Governmental and Nonprofit Accounting 5 (1989): 125-163.

6Eduardo L. Rhodes, "An Exploratory Analysis of Variations in Performance Among U.S. Parks," in Measuring Efficiency: An Assessment of Data Envelopment Analysis (San Francisco: Jossey-Bass, 1985), and same reference as Note 4.

7Same reference as Note 4.

8Taesik Ahn, Victor Arnold, A. Charnes, and W. W. Cooper, "DEA and Ratio Efficiency Analysis for Public Institutions of Higher Learning in Texas," Research in Governmental and Nonprofit Accounting 5 (1989): 165-185.

ABOUT THE AUTHORS

David J. Murphy is assistant to the commandant at the Air Force Institute of Technology, advising on quality and strategic planning matters. He received his Ph.D. from Arizona State University. His research interests include quality management, strategic planning, and performance measurement.

John N. Pearson is associate professor of purchasing and logistics management at Arizona State University. He holds a Ph.D. in business administration from Georgia State University. His research interests include purchasing strategy, materials management, strategic planning, and entrepreneurial firms.

Sue P. Siferd is assistant professor of supply chain management at Arizona State University. She holds a Ph.D. from Ohio State University. She is a CPA and CMA. Her research interests include quality management, purchasing strategy, total cost of ownership, and service operations management.

Copyright Council of Logistics Management 1996
Provided by ProQuest Information and Learning Company. All rights Reserved