Forecasting readiness - Regression Analysis Techniques - military operational readiness

Steven A. Oliver

According to many experts, the readiness of America's Armed Forces deteriorated throughout the 1990s. The Chairman of the House National Security Committee, Floyd D. Spence, stated that the readiness of the Armed Forces has been jeopardized and there is a real danger the Defense Department will return to the hollow forces of the 1970s." (1) During this time, combat readiness of the Air Force fighter aircraft has declined in varying degrees. One indicator of aircraft combat readiness, the mission capable (MC) rate, is used to identify the percentage of aircraft able to perform their primary wartime missions. The not mission capable (NMC) rate shows the converse. From fiscal year (FY) 1991 through fall 2001, the aggregate Air Force aircraft total not mission capable rate for maintenance (TNMCM) for all aircraft steadily increased from 7.6 percent to 18.1 percent while total not mission capable rate for supply (TNMCS) increased from 5.5 percent in FY86 to 13.4 percent in FY01 (Figure 1). (2) The erosion of MC ra tes appears to have stabilized, but concern still exists, and efforts to determine the reasons behind the decline continue. To illustrate the level of concern, in a 5 January 2000 memorandum to the Air Force Deputy Chief of Staff, Installations and Logistics, the Air Force Chief of Staff, General Michael Ryan, asked, "What are the main causes for increasing TNMCM rates over the last few years?" (3)

Currently, the Air Force uses the Funding/Availability Multimethod Allocator for Spares (FAMMAS) model to forecast overall MC rates for each mission design series (MDS) aircraft in its inventory. FAMMAS uses time-series forecasting techniques to predict overall MC rates for each MDS, using past, present, and future spares funding levels, along with the last 3 years of historical TNMCS and TNMCM data. (4) Numerous operational and funding decisions are made each year based, in part, on the predictions of this model.

Problem

While the FAMMAS model does an excellent job of predicting MC rates for each MDS based on funding data and planning factors, it does not adequately consider other factors that could impact MC rates. Specifically, the FAMMAS model does not incorporate any logistics or operations-related factors into its prediction computations of MC rates, other than historical TNMCM and TNMCS data that act as adjustment factors in the model. Recent studies have identified many factors related to MC rates: maintenance manning and experience, retention, break and fix rates, operations tempo (OPSTEMPO), spare parts issues, and reliability and maintainability (R&M) of aircraft systems, among others. (5) A review of aircraft readiness literature of the 1970s, 1980s, and 1990s indicates that most of these factors can be grouped into one of the following categories: aircraft R&M, aircraft operations, logistics operations, personnel, environment, and funding (Table 1).

Unfortunately, there have been few attempts to include these different factors in the construction of a mathematical model that explains and forecasts MC rates. While FAMMAS is an effective tool for predicting MC rates, it does not adequately consider other significant factors besides funding. Furthermore, it does not identify potential cause-and-effect relationships that might be manipulated to affect future MC rates; it just projects trends into the future.

This research attempts to satisfy these deficiencies in forecasting capability. While time-series models like FAMMAS produce accurate forecasts by projecting trends, that is all they provide. Time-series forecasts are based on data trends, not explanatory data. Explanatory forecasts may reveal potential cause-and-effect relationships that might be manipulated to have an effect on future forecasts. Explanatory forecasting techniques, such as regression, can be used with greater success than time series methods for policy and decision making. (6) With fewer resources available to the Air Force and continued emphasis by senior leadership to use resources more efficiently, the Air Force cannot afford to use its resources indiscriminately with little knowledge about how their use will impact mission needs and goals. The Air Force needs to develop more precise analytical tools in making resource allocation decisions. These tools should assist in determining what results might arise from the allocation and use of it s resources in pursuit of mission needs and goals. Correlation analysis was used to identify key factors associated with MC rates and applied multiple linear regression analysis to help explain and forecast aircraft MC rates. Specifically, quarterly MC rates for Air Force F-16C/D aircraft, from FY93 to FY00 were analyzed. The F-16C/D was selected so an in-depth analysis could be conducted on a single aircraft type, as opposed to a superficial analysis of multiple aircraft types.

Factors Associated with TNMCM

The TNMCM rate describes the percentage of aircraft NMC due to one or more maintenance conditions. A grounding maintenance condition could be almost anything, ranging from the replacement of a leaking fuel cell to the completion of scheduled maintenance or a time compliance technical order (TCTO). As most aircraft maintenance personnel already know, the amount of TNMCM time an aircraft accumulates is related to and influenced by many different factors--some easily measured and some not. A study conducted by the Dynamics Research Corporation (DRC) for the Air Force Directorate of Supply identified factors--such as manning, experience, retention, increased inspections, modifications to aging aircraft, break rates, cannibalizations, increased man-hours, OPSTEMPO, and aircraft maintenance management policy changes--as being directly related to changes in the amount of TNMCM hours. (8) Factors identified in DRC's study, along with preliminary analysis from the Air Force Logistics Management Agency's (AELMA) TNMCM study, suggest that these and other factors can, by and large, be grouped into two categories, personnel and R&M.

Personnel. Personnel are key to the readiness equation. Many factors must be considered when addressing the relationship between personnel and TNMCM rates. Studies have indicated that changes in the maintenance area in manning levels, experience, morale, and retention are related to changes in TNMCM rates. While some of these factors are easily quantified (manning levels and number of noncommissioned officers [NCO]), others are not (experience and morale). With respect to the quantifiable factors, several studies have indicated that manning levels in the enlisted maintenance career fields (2WXXX and 2WXXX) appear to be negatively correlated to TNMCM hours. (9) As the number of people in these career fields decreased, the number of TNMCM hours increased. (10)

Not only does the number of people relate to TNMCM rates, the experience of personnel (defined as skill-level or years of service [YOS]) does as well. DRC's study found that reductions in the number of five- and seven-level technicians, as well as a reduction in the number of NCOs, were negatively correlated with TNMCM hours. (11) Furthermore, preliminary analysis from AFLMA's Cost and Valuation of Air Force Aircraft Maintenance Personnel study revealed the same relationship in terms of YOS.

As the number of trainees (1-4 YOS) for bomber crew chiefs increased, bomber MC and 8-hour fix rates decreased (Figure 2).12 Obviously, this is not a cause-and-effect relationship. Nonetheless, a mathematical, as well as intuitive, relationship exists.

R&M is another area that dramatically influences TNMCM rates. Both are defined as follows:

Reliability is the probability an item will perform its intended function under stated conditions for a specified interval or over its useful life.

Maintainability is the ability of an item to be retained (preventive) or restored (corrective/unscheduled) to a specified condition, when maintenance is performed by personnel having specific skill-levels, using prescribed procedures and resources, at each prescribed level of maintenance and repair. (13)

As a system's cumulative operating time increases, the probability of its failure tends to increase, decreasing the system's potential reliability. Reliability also decreases when the conditions under which the system was designed to operate change)4 The average Air Force aircraft is 20 years old, with 40 percent of the fleet 25 years or older. Many of these aircraft are at critical points in their life cycles. (15) For example, many F-16s have reached 2,400 hours flying time, a significant point in an 8,000-hour service life. As these aircraft age and operating conditions change, the reliability of systems and components decreases, and failures occur more often, which increases maintenance costs. Increased failures affect aircraft maintainability, requiring more maintenance and often increasing repair times when more hard breaks occur. In the case of the F-16, operational usage has been more severe than design usage (eight times more), resulting in the acceleration of its airframe service life at a rate tha t may not let it reach its expected overall service life. (16)

In spite of increased operational usage, fighter aircraft break rates have increased only slightly. However, break rates only account for grounding pilot-reported discrepancies and, therefore, cannot serve as the sole indicator of aircraft reliability. Other maintenance problems discovered during routine and special inspections and while performing maintenance also affect R&M. For example, preliminary analysis from AFLMA's TNMCM study found the number of TNMCM hours attributed to phase maintenance inspections increased 174 percent from 1995 to l999. (17) In the Air Combat Command (ACC), F-16 fuel leaks, F-15 flight control delamination problems, and cracked A-10 fuselage station 365 bulkheads, none of which are typically pilot-reported discrepancies, are just a few of the TNMCM R&M drivers that have negatively impacted these weapons systems in recent years. (18) Additionally, reduced R&M associated with high failure rates of F-15 and F-16 engine components resulted in substantial increases in the accumulatio n of TNMCM time. (19)

Declining R&M has also affected TNMCM time in another way. To improve R&M, numerous inspections and modifications have been initiated and implemented, many of which manifest themselves in the form of TCTOs and special inspections. Preliminary analysis obtained from AFLMA's TNMCM study of the F-16 block 42 aircraft revealed the total man-hours expended on TCTOs increased 120 percent from FY95 to FY99 and the man-hours per TCTO event increased 69 percent, indicating TCTOs may be becoming more manpower intensive and technically challenging. The analysis also indicated that low manning and fewer experienced technicians contributed to increases in man-hours required to complete them. (20) While modifications and inspections are necessary to increase overall R&M and maintain the long-term health of an aging fleet of aircraft, they will continue to comprise a substantial portion of reported TNMCM time.

Factors Associated with TNMCS

The TNMCS rate describes the percentage of aircraft NMC due to the unavailability of spare parts. Several factors influence the amount of TNMCS hours an aircraft accumulates. Like the factors that influence TNMCM time, some TNMCS factors can be quantified, while others cannot. Some of the factors currently measured include component reliability and demand, as well as logistics operation factors such as proper mix and level of inventory, component repair times, and order and ship time (O&ST). Other factors, just as important but not easily quantified, are diminishing manufacturing sources, material shortages, and inventory forecasts. (21)

Reliability and Demand. Reliability affects the accumulation of TNMCS time through demand. The more unreliable a component, the more often it fails. Failures necessitate that the component be either repaired or replaced. While either requires initiation of maintenance actions that result in an aircraft accumulating TNMCM time, it also causes TNMCS time to accrue by placing a demand on the supply system. If a part has been designed with sufficient reliability or its reliability characteristics are well understood, the appropriate level of inventory or repair capacity or capability can be established to ensure demands for the part are satisfied in a manner that maximizes aircraft availability and reduces TNMCS time. (22)

In the 1990s, the reliability of many aircraft components declined. One of the main reasons attributed to reduced reliability has been aircraft (and their components) operated outside the set of conditions for which they were designed. This condition primarily manifests itself in the form of aging aircraft and increased failures brought about by the increased OPSTEMPO of weapons systems. (23) For many different reasons, aircraft designed for a certain expected service life and certain operating conditions are being operated beyond those limitations. This resulted in the premature failure of many components that had not been anticipated. (24) In a 1998 article on aging aircraft, Colonel Irving Halter, 1st Fighter Wing Operations Group Commander, stated:

In 1997, the wing sent 16 F-15s to Saudi Arabia...and over the course of 6 months, they accumulated an average of 485 hours each ... ordinarily, it would take an F-15 more than a year and a half to fly that much... we are finding things breaking on the jets that we had not predicted.... (25)

Furthermore, since these failures were not anticipated, sufficient quantities of spares and, in some cases, adequate repair capability were not established. Consequently, delays in obtaining or repairing replacement parts occurred while replacements were sought or repair capability established. In some cases, the delay in obtaining replacement parts grew even longer due to the need to establish contractual relationships with the commercial sector to obtain replacement parts or repair capability. (26)

Level and Mix of Serviceable Inventory. Inventories are used to provide organizations with increased flexibility in executing operations. They give organizations a buffer that allows them to cope with the variability that might be encountered in demand, production, price, and transportation. When inventory levels are reduced, problems once hidden by high inventories (poor reliability, inexperienced work force, or excessive repair times) reveal themselves, requiring management to correct them. (27) The impact of inventory reduction programs driven by Department of Defense (DoD) policy decisions depleted stocks of spare parts throughout the Air Force. (28) As inventory levels dropped, reliability and depot repair process problems became more evident. The lower level of serviceable inventory and the problems it revealed contributed to an escalation in TNMCS rates. (29)

Repair Tune. The time it takes a depot or contractor to repair and return a reparable item to serviceable condition also affects TNMCS time. Under two-level maintenance, most intermediate base-level repair capability was eliminated. Consequently, the majority of reparable parts are sent to depot or contractor repair facilities where they are either condemned or repaired and returned to serviceable inventory stocks. Two-level maintenance eliminated a significant portion of an operational unit's ability to manage and control its TNMC hours. Repair times vary among components and repair facilities and are influenced by factors such as repair capacity, funding, personnel levels, skill, and policy decisions. (30) One of the major policy issues affecting depot production was the announcement of the closure of two air logistics centers. According to former Secretary of the Air Force F. Whitten Peters:

Directly relevant to readiness were the closures of two of the five Air Force maintenance depots ... almost immediately upon announcement, these closures created turmoil at our depots as skilled workers started to leave the closing depots well in advance of the actual closure dates. The most serious aircraft readiness problems were caused by our inability to move depot production lines on schedule and ... our inability to hire skilled manpower at the receiving depots ... we are still hundreds of people short at two of our depots. (31)

Further illustrating the impact of repair times, a 1990 Air Force Logistics Command (AFLC) study revealed it took about 30 days to repair an item at a depot. (32) More recently, an F-16 logistics chain management study performed by KPMG found depot repair time averaged 34.9 days for ten critical F-16 avionics components. (33) Additionally, data collected by Synergy, Inc, from the Recoverable Consumption Item Requirements System (D041) and a General Accounting Office report indicate repair time at the depot is the lengthiest portion of the Air Force reparable pipeline. (34)

O&ST. Another variable that influences TNMCS time is O&ST, which begins when the customer initiates an order with a depot for a replacement part and ends when it is received. (35) O&ST is highly dependent on the availability of serviceable inventory and is significantly affected by shipping and transportation factors. Data collected by Synergy revealed that O&ST, from the third quarter of FY98 to the second quarter of FY99, averaged 7.4 days for 121,516 transactions, (36) while an earlier AFLMA assessment suggested an average O&ST of 16.4 days. (37) However, when a serviceable part is not available, O&ST could encompass the entire repair cycle time (waiting for the failed component to be repaired and shipped back to the unit that sent it), making it possible for large variances. The KPMG study focused on ten critical F-16 components and found that O&ST for these items averaged 37 days, which appears to encompass the component's entire repair cycle. (38)

Underlying Factors Simultaneously Affecting TNMCM and TNMCS

Some factors individually affect TNMCM and TNMCS rates, and some factors, when altered, affect both rates simultaneously. Three underlying factors affecting both TNMCM and TNMCS rates are funding, aircraft operations, and the environment. While none causes readiness, each can significantly affect it. Funding provides the resources used to achieve readiness, while aircraft operations and the environment provide the conditions that shape it, While the nature of some of these factors makes the degree to which they affect readiness difficult to quantify, virtually all the research shows they impact it.

Funding. Funding is one of the common denominators affecting both TNMCM and TNMCS in the MC equation. While funding cannot cause readiness, the amount of funding made available can have a significant impact upon it. If there is no funding available, there probably will be no people or equipment available either since there is a cost for having both. Furthermore, proper allocation of limited funds also needs to be made among competing requirements. Fully funding spares purchases while underfunding personnel could lead to the Air Force having plenty of spare parts but not enough people to install them. (39) A DRC study found that FY95 and FY96 funding for spare parts through the AFMC Materiel Support Division was 58 percent and 74 percent of the requirement. According to the study, this level of funding had a negative impact on MC rates. Furthermore, the study concluded that, if funding for spare parts is even marginally less than the requirement, the result would be less aircraft availability. If inadequate fu nding exists or funds are not properly allocated, MC rates may suffer. (40)

While many examples illustrating the effect of reduced funding on readiness such as fewer spare parts and manpower reductions are obvious, others are less apparent. For example, reduced funding for R&M enhancements of existing weapons systems, infrastructure maintenance, or training tends to have a more subtle impact on MC rates that is not immediately apparent. (41) Some of the literature highlights lower O&M funding, coupled with increased competition for these limited funds (primarily unplanned contingency operations), as another contributing factor to lower MC rates. When the cost of contingency operations is not fully paid for by budget or supplemental appropriations, the remaining balance may come out of other accounts such as O&M. Even temporarily shifting funds in and out of O&M accounts can have a disruptive and negative impact on training and maintenance. (42) Figure 3 depicts how the Air Force's total obligation authority (TOA) is related to MC rates over time.

Environment. The DoD environment also affects MC rates. The end of the Cold War transformed a fairly stable defense environment to a very dynamic one, causing numerous changes, both internally and externally, in both DoD and the Air Force. The changes affected almost every facet of the Air Force, from its structure and operations to its funding and personnel. For the Air Force, substantial increases in the OPSTEMPO and personnel tempo (PERSTEMPO), the frequency and size of workload on both personnel and equipment, resulted from the new defense environment. Since the early 1990s, the number of deployments and contingency operations has increased tremendously, driving up OPSTEMPO and PERSTEMPO. According to a RAND study, the number of flying hours devoted to Air Force military operations other than war soared from about zero at the end of the Cold War to a point where they consume more than 10 percent of the active duty flying hours, placing unanticipated, heavy demands on support personnel and equipment. (44)

Increases in OPSTEMPO and PERSTEMPO have negatively affected both equipment and personnel, forcing both to work longer and harder. While no sole measurement captures OPSTEMPO or PERSTEMPO in its entirety, the research does outline their effects, many of which have been discussed already and are measurable. Some of the effects are decreased aircraft R&M and spare parts inventories, increased maintenance man-hours and deployments, and reduced retention and morale. (45) The impact of some of these effects can be seen in changes in monthly F-16 MC rates from 1990-1999 (Figure 4). Coupled with reduced funding levels, the effects of OPSTEMPO and PERSTEMPO can be magnified even more. Furthermore, it is expected that the effects of OPSTEMPO and PERSTEMPO will continue to grow if they are not reduced or at least properly supported. (46)

Figure 3 not only illustrates the effects of funding and R&M but also demonstrates the effect maintenance management decisions can have on MC rates. The management techniques employed in and applied to aircraft maintenance can influence the amount of TNMCM or TNMCS time an aircraft accumulates. At unit level, poor planning and use of resources might result in an aircraft's being NMC for longer periods than necessary. Furthermore, changes to logistics policies initiated by different headquarters can also affect MC rates. While the Air Force does not identify and quantify most of these changes, it is important to note the potential effect these changes might have on MC rates.

One of the biggest changes in aircraft maintenance during the early 1990s was the implementation of two-level maintenance. For many weapons systems, the implementation of two-level maintenance eliminated intermediate-level maintenance (wing-level repair shops) through reductions of people and equipment, transferring that repair capability to the depots. Two-level maintenance achieved its goals of cost savings and reduction of the logistics footprint, saving $259M and eliminating 4,430 positions. (48) However, even with these successes, it affected MC rates by reducing both the repair capability and flexibility of operational units. When an aircraft is grounded because of a failed part and the unit cannot acquire a replacement from the supply system in time for the aircraft to fly its next scheduled mission, the unit typically cannibalizes the replacement part from another aircraft (when it is a feasible cannibalization). Cannibalizing parts doubles the time spent on maintenance and increases the probability of damaging the part. (49) While the rate of cannibalization is affected by many factors, meaning that increased cannibalizations cannot be attributed solely to the implementation of two-level maintenance, the overall rate of cannibalization has increased by 78 percent since the inception of two-level maintenance in the early 1990s. (50) Further compounding the problem were the different maintenance priorities being applied by operational wings and depots. The main priority of operational wings was acquiring the proper parts as quickly as possible to return broken aircraft to fully MC status. The depots' primary concern was repairing parts in a cost-effective manner. In many instances, this meant that depots would delay repair activities until enough parts accumulated so it was cost-effective to repair them, forcing wings to either cannibalize parts or accumulate TNMCS hours when serviceable parts inventories were depleted. (51)

Another maintenance management change involved the area of maintenance status reporting. Until FY97, ACC aircraft were returned to MC status after all maintenance was complete but before operational checks had been completed. However, in FY97, ACC changed its policy, requiring aircraft to be returned to MC status after all maintenance and operational checks were complete. This change led to an increase in the number of TNMCM hours accumulated. According to a TNMCM study conducted at Hill AFB in 1997, operational checks accounted for approximately 5 percent of the total TNMCM time. (52) While this represents only a small amount of total TNMCM time, it has been identified as one of the factors responsible for its recent increase.

In the early 1990s, the Air Force initiated an organizational change in most major commands that drastically altered maintenance and may have influenced TNMCM and TNMCS rates: implementation of the objective wing structure. The objective wing structure removed the day-to-day leadership and oversight of flight-line maintenance operations provided by each wing's senior maintenance officers and staff and transferred that responsibility to the less maintenance-savvy operations community, leaving the structure of the maintenance complex fragmented. While the senior leadership in the operations community was perfectly capable, the increased scope of their responsibilities--flying operations and flight-line maintenance--as well as lack of in-depth maintenance experience, may have led to less than optimal decisions concerning aircraft maintenance. (53) This lack of in-depth maintenance management knowledge and experience within the new structure was validated by the creation of the Deputy Operations Group Commander for Maintenance position. (54)

Forecasting and Regression

Forecasting MC Rates. General Ryan's question and the recent concern over decreased readiness were the primary reasons regression analysis was selected over time-series forecasting techniques as the methodology used for the study. Regression models not only provide a forecast but also explain the functional relationship between the dependent variable (MC rates) and numerous independent factors (personnel, component failures, and so forth). Using regression analysis to explain and forecast MC rates in this study provides two critical pieces of information--first, a forecast that allows for planning and, second, potential causes for the forecast that might be manipulated to alter the projected forecast. (55)

FAMMAS Model. The Air Force has a multitude of tools for forecasting MC rates. It has more than 30 models that can forecast MC rates; however, most are aircraft-specific and, therefore, cannot be used with different aircraft. (56) FAMMAS is the Air Force's primary forecasting tool. FAMMAS output data are primarily used in performing POM and budget assessments and weapons system assessments and are used in the sustainment executive management reporting process. Presently, DRC operates the model, validating the current version of the model (3.0.1) in September 1996. It is a time-series model that uses past, present, and future annual spares funding profiles to forecast MC rates for different weapon systems. It also uses elements such as inflation, carryover, and lead-time factors, as well as historical TNMCS and TNMCM rates (used as adjusting factors) when computing forecasts. Data come from the unit cost document, Reliability and Maintainability Information System (REMIS), and other reliable sources are used in a time-series forecasting algorithm to produce an MC rate forecast. (57)

FAMMAS has proven to be a fairly accurate forecasting model. According to the Defense Science Board Task force on Readiness, FAMMAS, in conjunction with other Air Force systems, has predicted peacetime MC rates for each aircraft in the inventory with an accuracy of +/- 2 percent over 3 years and +/- 5 percent forecasting over 6 years. (58)

Data, Sources, and Factors

REMIS. REMIS contains numerous factors (R&M and others) that relate to MC rates in varying degrees. For this analysis, status, utilization, and on/off equipment maintenance and repair data for each F-16C/D work unit code (WUC)--a five-digit alphanumeric code that identifies individual aircraft systems, components, and processes--were extracted from REMIS. This resulted in each REMIS factor (repair actions, TNMCM hours, and man-hours) having its data broken down to the five-digit WUC level, allowing links to be established between each factor and specific datasets of WUCs for specific F-16C/D systems, components, or processes.

Many REMIS factors were enhanced so a more in-depth analysis could be performed. It was believed the enhanced factors would provide greater insight into how REMIS factors for specific WUCs impact MC rates. To create these enhanced factors, a rank-ordered list of WUCs was developed for the entire 8-year period for each factor (man-hours, TNMCM hours, supply reliability, TNMCS hours, and repair hours) based on the total number of hours accumulated by each WUC each quarter. From those rank-ordered WUC lists, data pertaining to the top 50 WUCs were used to determine how each factor's top 50 ranked WUC dataset (the 50 most significant WUCs) was related to MC rates. Analyzing the REMIS data in this manner focused the analysis on specific groups of WUCs, each factor, and its relationships to MC rates.

Recoverable Consumption Item Requirements System. To determine how logistics operations factors influence F- 16 MC rates, data from FY92-FY00 pertaining to these factors were retrieved from the D041 (since replaced by D200). The D041 system is a wholesale-level supply management system used to compute reparable and consumable spare parts requirements by national stock number (NSN) for all customers, worldwide, on an aggregate basis. The system collects a wide variety of quarterly data from different systems pertaining to reparable items such as failures, lead times, base and depot repair times, and excess inventory. (60)

Personnel Data System (PDS). Personnel issues were repeatedly cited as major influences on MC rates. To assess the influence of these factors on F-16 MC rates, personnel data were obtained from the Air Force Personnel Center (AFPC) data system. Data for enlisted personnel with control AFSCs assigning them to the manned aerospace maintenance (45XXX and 2AXXX) and the munitions and weapons (46XXX and 2WXXX) career fields were retrieved as well as personnel data for officers assigned to the 21AX and 405X career areas. In an effort to include only those personnel associated with F-16 maintenance in the research, Air Force Instruction 36-2108, Airman Classification, was reviewed and ACC career-field functional managers were consulted to identify the AFSCs that would typically be assigned to provide F-16 maintenance support. All other AFSCs not associated with supporting F-16s were excluded from the data. While some of the AFSCs included in the study typically support only F-16 aircraft (crew chiefs and avionics A FSCs), other AFSCs (fuels and structures) support not only the F-16C/D but also a wide variety of aircraft. Both types of AFSCs were included in this research.

Methodology, Analysis, and Results

Correlation Analysis. Factors from only four of the six areas (personnel, R&M, aircraft operations, and logistics operations) were analyzed. Factors from the remaining two areas (environment and funding) were not analyzed because of the difficulties and complexities associated with obtaining and quantifying factors in these categories. Because of the large number of factors obtained and created for the analysis, a correlation analysis was performed to examine the strength of the relationship between each independent factor and the dependent variable (MC rate) to determine which factors should be included in the explanatory and forecasting regression models. Additionally, each factor was lagged against MC rates with respect to time (one to four quarters into the future) to analyze the relationship between a factor in one quarter and the dependent variable (MC rate) in future quarters. For example, the number of five-levels in the first quarter of a particular year may be more strongly associated with the MC r ate that occurs two quarters into the future (the third quarter) rather than the MC rate of the first quarter.

The factors demonstrating strong correlation with MC rates were also classified as to whether each could be controlled with respect to the future. Classifying the factors in this manner identified all the controllable factors to be considered for inclusion in the forecasting model. For example, several processes (recruiting, funding, cross-training, and drawdowns) are used to ensure a specific number of F-16 crew chiefs are in the Air Force at some future point in time. Furthermore, each of those processes can be manipulated to alter the specific number of F-16 crew chiefs to adjust for projected changes in future requirements. However, in the case of the F-16 cannibalization factor, there are no known specific processes or combination of processes that can be manipulated to cause a specific number of cannibalizations to occur 2 years into the future. While there may be processes that affect the number of cannibalizations (policies, component reliability improvements, and so forth) that take place, too many unknown factors will still influence the specific number of cannibalizations, making the final outcome 2 years into the future an uncertainty. The uncontrollable nature of this type of factor allows it to be used only in the explanatory model. The application of these and other criteria ensured both models contained only those factors demonstrating the strongest relationships with MC rates.

A correlation analysis was performed on 606 factors to examine the strength of the relationship between each factor and the dependent variable (MC rate) to determine which should be included in the models. However, when lagged by time (one to four quarters), the number of factors increased from 606 to 3,030. Based on the criterion used in the study, the correlation analysis revealed 1,246 variables that demonstrated either positive or negative relationships with MC rates.

A second correlation analysis was performed, and diagnostic scatter plots (as needed) were developed for these 1,246 variables to help identify cases of multicollinearity (redundancy). The analysis revealed numerous instances of multicollinearity among the maintenance, personnel, and retention factors. For example, the number of three-levels assigned to each of the AFSCs examined was highly correlated with the total number of three-levels assigned in all F-16 maintenance AFSCs. In these instances, the factor thought to best explain these relationships with MC rates was used, significantly reducing the amount of multicollinearity among the factors. For the example cited above, the number of three-levels assigned to all F-16 maintenance AFSCs was the factor used to represent the number of three levels assigned to each specific AFSC. This process reduced the number of variables from 1,246 to 87. Next, simple linear regressions and a third correlation analysis were performed on the remaining 87 factors, and by a pplying the study's criteria, the number of factors was reduced from 87 to 16. Figure 5 lists the independent factors used in the initial model and contains the full explanatory model.

Backward Stepwise Regression Analysis and the Explanatory Model. The explanatory model was developed to identify and describe the factors that demonstrate potential cause-and-effect relationships with MC rates. The specific multiple regression technique used to develop the explanatory model was backward stepwise regression in which all significant factors are included in the initial regression model. As the model is analyzed, factors that contribute minimally to the predictive or descriptive nature of the model are removed. The reduced model is then rerun and tested to verify the reduced model is statistically equivalent to the initial regression model. If the reduced model is found to be statistically equivalent, the contribution of each factor is reassessed in the reduced model, and once again, the factors found to minimally contribute are removed from the model. As long as each reduced model continued to be statistically equivalent to the initial model, the process of reassessing and removing factors is repeated until only the most significant explanatory factors remain in the model. The end result is a simpler explanatory model that is statistically equivalent to the initial model and contains only the most significant independent factors that relate to the independent variable (MC rate). (61)

From the 32 quarters of data plotted over time, 20 percent (seven quarters) were randomly selected and removed from the dataset for model validation and sensitivity analysis. The remaining 80 percent of the dataset (25 quarters) were entered into the [JMP.sub.IN] statistical analysis software package (academic version 4.0.2) to create the full explanatory model. After analyzing and reducing the full model several times, a final explanatory model (Figure 6) with an [R.sup.2] of 0.955 can be used to explain or predict F-16C/D MC rates at a 95 percent confidence level.

Sensitivity Analysis of the Explanatory Model. To test the predictive reliability of the final explanatory model, the data that were withheld (20 percent) from the original dataset were combined with the data used to build the model (80 percent). The dependent variables (MC rates) for each of the withheld quarters were excluded from this process so that, when the model was run, the software generated predicted MC rates and confidence intervals (alpha = 0.05). The resulting predicted MC rates were analyzed to determine if they fell within the bounds of the confidence intervals generated by the model. For sensitivity analysis, the total number of predicted observations (predicted MC rates) that fell within the bounds of the confidence interval was divided by the total number of observations (actual MC rates) so the predictive reliability and overall robustness of the model could be determined.

The explanatory model's predictive reliability was computed and revealed the observed MC rates for each respective quarter fell within the individual confidence intervals generated by the model six out of seven times, indicating the model's predictive reliability to be 86 percent. To compute the model's average prediction error, the widths of the confidence interval associated with each predicted MC rate at the prediction points were summed and averaged. The computation revealed the model's average prediction error as being 1.9 percent. The results of the sensitivity analysis for the model are shown graphically in Figure 7.

Multiple Linear Regression and the Forecasting Model. After the explanatory model was developed, a separate multiple linear-regression model was developed to forecast F-16 MC rates seven quarters into the future. The factors to be used to build the forecasting model were those identified as factors that could be controlled directly or indirectly with respect to the future. Consequently, many of these factors were different from those used to build the explanatory model. A series of forecasting models was built using data from the first 80 percent of the time-ordered quarters (FY93-1-FY98-4). Data from the remaining timeordered quarters (FY99-1-FY00-4) were set aside to assess the model's predictive reliability. To determine which combination of variables produced the most accurate forecast, the mean absolute percentage error (MAPE) was computed for each forecasting model's combination of factors (more than 50 different combinations were tried). The MAPE measures the percentage error (point estimate error) of a model's ability to forecast and is computed by taking the sum of the absolute percent error for each period (absolute difference between the forecast and actual MC rate) and dividing it by the total number of forecast periods. (62)

As a starting point, the model generating the smallest MAPE was the one selected to forecast F-16 MC rates. After building more than 50 models, using different combinations of variables and analyzing the MAPE of each, the following model (Figure 8) generated a MAPE of 0.824 percent, which was the lowest of all the models tested.

Forecasting Model Sensitivity Analysis. Next, the predictive reliability, which could also be described as the usefulness of the model's forecast outputs for planning the forecasting model, was analyzed. The width of the confidence interval served as the indicator of the model's robustness. The narrower the interval, the more robust the model; alternatively, as the interval widens, the model's robustness decreases. The initial model's (version 1) average prediction error was computed (average width of the confidence interval for the forecast period that the actual MC rate should fall within), along with a series of alternative models, to determine its robustness; the smaller the average prediction error, the more robust the model.

The actual MC rates were plotted over time along with the predicted MC rates and the associated confidence intervals (alpha = 0.05) generated by running the model (Figure 9). First, the average prediction error for the forecast period was analyzed in the same manner as the explanatory model and was found to have an average prediction error of 4.8 percent and an [R.sup.2] of 0.779. Next, the average prediction error of the forecasting model (version 1) was compared to the other 50 models to validate its robustness. The comparison revealed one of the alternative models (version 2, Figure 10) produced a narrower confidence interval and prediction error of 2.1 percent and narrower confidence interval than that of the initial forecasting model (version 1). Even though forecasting model version 2 generated a higher MAPE (1.25 percent and an [R.sup.2] of 0.943) than that of version 1, the criterion used to assess the robustness of the model (narrower confidence interval) makes version 2 a more robust alternative fo r use in long-range planning (Figure 11).

Conclusions

One of the questions research for this article tried to answer was, "Which factors are related to MC rates and what are the associated relationships?" From the analysis, it was quite apparent that some categories of factors were more strongly related to MC rates than others. R&M factors demonstrated the strongest relationships; however, this was not unexpected since many of these factors are used to compute MC rates. The most meaningful factors from this area were the R&M factors ranked over time. These factors were designed to link the number of hours or occurrences a specific group of F-16C/D WUCs ranked over time to F-16C/D MC rates. Analyzing the data in this manner transformed it and made it more meaningful. Instead of analyzing how accumulated hours of quarterly maintenance or supply time relate to MC rates, the ranked variables demonstrated how the cumulative quarterly maintenance hours of the 50 most problematic WUCs over the last 32 quarters related to MC rates. Although the ranked measures were more meaningful than the summed quarterly WUC data, it is important to note that both types of factors do not identify the root causes as to why the WUCs accumulated time against them. However, the factors broken down by WUC highlight those WUCs that appear to have been the most significant over time, allowing maintenance managers to perform root cause analysis and take corrective actions as needed.

When compared with factors from the other two areas, aircraft and logistics operations demonstrated the weakest relationships with MC rates. However, when these factors were linked with either personnel or R&M factors, the new factors demonstrated strong correlation with MC rates. For example, the total maintainers assigned factors (by grade, skill-level, and AFSC) were combined with the average aircraft inventory factor to create a series of maintainers assigned per aircraft interaction factors, linking the category of personnel to aircraft operations. This demonstrated stronger correlation with MC rates than either the total maintainers assigned or the average aircraft inventory factors by themselves. Consequently, aircraft and logistics operations factors were used primarily to create new variables linking system performance to either R&M or personnel.

The results of the analysis were very similar to the findings of other studies that analyzed how personnel levels relate to MC rates. The underlying factor in the personnel data appeared to be experience. Whether the data were analyzed by grade, skill-level, or percent of authorizations filled, the story was the same: as the number of inexperienced people (defined as three-levels and E1 through E-3) increased, MC rates decreased. Conversely, as experience increased (five-, seven-, and nine-levels as well as E4 through E-9), MC rates increased. To better understand these relationships in an operational environment, the ratio of three-levels to other skill-levels was thought to be a useful measure of personnel conditions (experience mix) that might exist in a typical maintenance complex. The ratios were created to model the level of responsibility that more experienced personnel are shouldered with when training and supervising new or inexperienced personnel, while simultaneously performing their normal duties . When analyzed, increases in the ratio of three-levels to either five- or seven-levels (or both combined) were negatively correlated to MC rates. A detailed analysis of these ratios for specific AFSCs was less clear. Some AFSCs, such as crew chiefs and flight-line avionics, exhibited the same trends as the top-level analysis of the ratios; however, skill-level ratios for other AFSCs, such as engines and structures, demonstrated positive correlation with MC rates. This could indicate that MC rates are more sensitive to skill-level imbalances in certain career fields than others or that AFSCs typically associated with on-equipment maintenance affect MC rates more than those typically associated with off-equipment maintenance.

Retention and separation variables were also analyzed as part of the personnel data. The data were broken down by AFSC and grade and then by category of enlistment as first-, second-, and career-term airmen to assess how retention rates for these groups of airmen related to MC rates. Instead of analyzing the raw numbers, the data were converted to percent of reenlistment-eligible personnel who reenlist or separate. These factors exhibited varying degrees of correlation with MC rates. The strongest correlation with MC rates was demonstrated by the F16 crew chief retention rate. Most of the other AFSC-specific retention correlations were very weak. However, overall, first-term, career, and total reenlistment rate correlations were strong. These three retention factors, along with F-16 crew chief retention rates, were the retention variables that appeared to be the most significant in this area of personnel data. The second-term, retention-rate variable, although not found to be strongly related to MC rates, wa s retained in the regression analysis since several sources cited lower second-term retention rates as having a negative effect on MC rates. This appeared to be the case, as the second-term retention rate factor ended up being included as a variable in the alternative forecasting model (version 2).

Other questions this research attempted to answer were, "Which model best forecasts MC rates, and how helpful are these models in demonstrating relationships between the factors and MC rates, and what are the results?" The answer to these questions is a resounding, "It depends." Many good regression and time-series models can be developed, and some models are more useful than others. However, unlike most time-series models, regression models can be used to describe relationships among factors as they relate to the independent variable (MC) and provide forecasts. Furthermore, there are many different criteria that can be used to select the best model. The real answer as to which model explains or predicts best resides with the individual or organization using the model and the context in which the model is to be used.

The focus of the explanatory regression model is on explaining how a set of significant independent variables relates to MC rates. The study's explanatory model contained seven different factors, both controllable and uncontrollable, that mathematically explain 95 percent of the what behind MC rates. Regarding the explanatory model's uncontrollable factors, further analysis could be performed to discover the root causes behind why a particular factor is affecting MC rates in the manner it is. If the root causes can be identified and prove to be controllable, then the factor could be included in the forecasting model to see how changes in that variable might affect future MC rates.

Different user needs typically require the application of different criteria when selecting the best forecasting model. In this study, if the user's focus is on forecasting a point estimate (that is, the MC rate will be 82 percent 4 quarters from now), the user should use the forecasting model that generates the smallest MAPE (version 1). However, if the user is interested in reducing the prediction error of the forecast (that is, the actual MC rate will fall within +/-2.1 percent of the predicted MC rate), then the user should select the forecasting model that generates the smallest average prediction error (version 2). Once again, the best model is the one that is most useful to the user.

Unlike the explanatory model, the purpose of the forecasting models is to predict what future F-16 MC rates might be seven quarters into the future, given a certain set of future conditions. The forecasting models allow the user to conduct what if scenarios to see how changes in the models' variables (controllable variables such as the number of five-levels, sorties, and so forth) might impact future MC rates. The initial forecasting model, version 1, provides superb point estimates of future MC rates. On average, the model's forecasted MC rate falls within [+ or -]0.82 percent (MAPE) of the actual MC rate that occurred in the forecast period. Additionally, using this model, there is a 95-percent confidence that the actual MC rate will fall within [+ or -]4.8 percent of the forecasted MC rate (within the bounds of the confidence interval), making the future planning window rather large.

The alternative forecasting model, version 2, is an excellent long-range planning model because of its small prediction error. With this model, there is a 95-percent confidence level that the actual MC rate will fall within [+ or -] 2.1 percent of the forecasted MC rate, which is a significantly smaller future planning window than the window generated by version 1. However, unlike version 1, the forecasted MC rate generated by version 2 normally falls within [+ or -]1.25 percent of the actual MC rate (MAPE), making its point-estimate predictions less accurate, which is why this model wasn't initially selected as the best model.

The study illustrates what most logisticians already know- MC rates are determined by a variety of logistics-type factors (logistics operations, R&M, and personnel), as well as operations type factors (aircraft operations, funding, and environment) and their interaction. Another conclusion regarding MC rates, although not a primary focus of this study, was reached. In addition to measuring readiness and aircraft availability, the MC rate indirectly assesses how a weapon system was operated and how its logistics support structure performed. A weapon system's operating conditions and its logistics support structure help determine the availability and readiness of that weapon system. To determine how a weapon system might be affected by changes in these areas, its specific logistics and operations factors, as identified above, should be isolated, then simultaneously analyzed. The key is simultaneous analysis. Because each of the factors affect the MC rate in different ways at the same time, they need to be revi ewed together (a holistic approach) to understand how they affect the MC rate. As a result, the most significant factors (like experience, cannibalizations, and funding levels) could be identified, monitored, and improved as needed. Simultaneous assessment should determine a more comprehensive set of key indicators which could provide a better picture of how significant logistics and operations factors affect the way the logistics support structure sustains a weapons system and shapes its future availability--its readiness--its MC rate.

[Graph omitted]

[Graph omitted]

[Graph omitted]

[Graph omitted]

[Graph omitted]

[Graph omitted]

[Graph omitted]

Table 1.

Potential Factors Affecting MC Rates

     Personnel                 Environment       Reliability &
                                                Maintainability

Personnel assigned or        OPSTEMPO          TNMCM
authorized                   factors           hours
Personnel in each skill-     PERSTEMPO         Maintenance
level (1, 3, 5, 7, 9 and 0)  factors           downtime/reliability
Personnel in each grade      Number of         Mean time between
(E1-E9)                      deployments       failures/mean time to
                                               repair
F-16 maintenance             Policy            Code 3
personnel in various Air     changes           breaks
Force specialty codes
(AFSC)
F-16 maintenance             Contingencies     8-hour fix rate
personnel by skill-level
per AFSC
F-16 maintenance             Vanishing         Reparable item
personnel by grade per       Vendors           failures
AFSC
Retention rates for F-16     Weather           Cannibalization
maintenance personnel                          hours/actions

Personnel per aircraft       Aircraft          Repair actions/hours
ratios                       age
Maintenance officers         Aircraft mission  Maintenance
assigned or authorized       (training, test,  man-hours
                             combat)

     Personnel                 Funding            Aircraft
                                                 Operations

Personnel assigned or        Replenishment      Aircraft
authorized                   spares funding     utilization rates
Personnel in each skill-     Repair             Possessed
level (1, 3, 5, 7, 9 and 0)  funding            hours
Personnel in each grade      General support    Average sortie
(E1-E9)                      funding            duration

F-16 maintenance             Contractor         Flying
personnel in various Air     logistics support  hours
Force specialty codes        funding
(AFSC)
F-16 maintenance             Mission support    Sorties
personnel by skill-level     funding
per AFSC
F-16 maintenance             O&M funding        Flying
personnel by grade per                          scheduling
AFSC                                            effectiveness
Retention rates for F-16     Initial spares     Type mission
maintenance personnel        funding            (DACT, CAP,
                                                and so forth)
Personnel per aircraft       Acquisition        Over-Gs
ratios                       logistics funding
Maintenance officers                            Airframe
assigned or authorized                          hours


     Personnel                 Logistics
                               Operations

Personnel assigned or        TNMCS
authorized                   hours
Personnel in each skill-     Base repair
level (1, 3, 5, 7, 9 and 0)  cycle time
Personnel in each grade      Order and
(E1-E9)                      ship time

F-16 maintenance             Level of
personnel in various Air     serviceable
Force specialty codes        inventory
(AFSC)
F-16 maintenance             Level of
personnel by skill-level     unserviceable
per AFSC                     inventory
F-16 maintenance             Supply
personnel by grade per       reliability
AFSC
Retention rates for F-16     Supply downtime
maintenance personnel

Personnel per aircraft       Depot repair
ratios                       cycle time
Maintenance officers         Maintenance
assigned or authorized       scheduling
                             effectiveness
Figure 5.

Full Explanatory Model

Y = [[beta].sub.0]+[[beta].sub.1] [X.sub.1]+[[beta].sub.2][X.sub.2]+
[[beta].sub.3][X.sub.3]+[[beta].sub.4]
[X.sub.4]+[[beta].sub.5][X.sub.5]+
[[beta].sub.6][X.sub.6]+[[beta].sub.7]
[X.sub.7]+[[beta].sub.8][X.sub.8]+
[[beta].sub.9][X.sub.9]+[[beta].sub.10]
[X.sub.10]+[[beta].sub.11][X.sub.11]+ [[beta].sub.12][X.sub.12]+
[[beta].sub.13]([X.sub.10]/{[X.sub.11]+
[X.sub.12]})+[[beta].sub.14][X.sub.10]/
[X.sub.12]+[[beta].sub.15][X.sub.9]/ [X.sub.5][X.sub.15]+[[beta].sub.16]
[X.sub.3]/[X.sub.8]+[epsilon]


Prediction:        Y = Predicted F-16C/D
                   Mission Capable Rate
Original Effects:  [X.sub.1] = TNMCM Hours
(Factors)          of Top 50 Ranked WUCs
                   [X.sub.2] =
                   Cannibalization Hours of
                   Top 50 Ranked WUC
                   [X.sub.3] = Total F-16
                   Maint Personnel Assgnd
                   (Lagged 3 Qtrs)
                   [X.sub.4] = Maint
                   Reliability of Top 50
                   Ranked WUCs
                   [X.sub.5] = Average
                   Aircraft Inventory
                   [X.sub.6] = 8-Hour Fix
                   Rate (ACC)
                   [X.sub.7] = Total F-16
                   Crew Chiefs Assigned
                   [X.sub.8] = Total, O-3,
                   Maint Officers Assgnd
                   (Lagged 3 Qtrs)
                   [X.sub.9] = Total F-16
                   Maint Personnel Assigned
                   [X.sub.10] = Total 3-
                   Levels Assigned
                   [X.sub.11] = Total 5-
                   Levels Assigned
                   [X.sub.12] = Total 7-
                   Levels Assigned

Interactions:      [X.sub.10]/([X.sub.11]+
(Factors)          [X.sub.12]) = 3-Levels
                   per 5- and 7- Levels
                   [X.sub.10]/[X.sub.12] =
                   3-Levels per 7-Level
                   [X.sub.9]/[X.sub.5] = F-
                   16 Maint Personnel per
                   Aircraft
                   [X.sub.3]/[X.sub.8] =
                   Total F-16 Maint
                   Personnel Assigned per
                   Total, O-3, Maintenance
                   Officer (4024/21A3)

Higher Order:      No significant higher
(Factors)          order terms were
                   revealed.
Figure 6.

Final Explanatory Model

Y = [[beta].sub.0]+[[beta].sub.1] [X.sub.1]+[[beta].sub.2][X.sub.2]+
[[beta].sub.3][X.sub.3]+[[beta].sub.4]
[X.sub.4]+[[beta].sub.5][X.sub.5]+
[[beta].sub.6][X.sub.6]+[[beta].sub.7] [X.sub.10]/[X.sub.12]+[epsilon]


Forecast:          Y = Predicted F-16C/D
                   Mission Capable Rate

Original Effects:  [X.sub.1] = TNMCM Hours
(Factors)          of Top 50 Ranked WUCs
                   [X.sub.2] =
                   Cannibalization Hrs of
                   Top 50 Ranked WUCs
                   [X.sub.3] = Average
                   Aircraft Inventory
                   [X.sub.4] = Total F-16
                   Maint Personnel Assigned
                   [X.sub.5] = Total 3-
                   Levels Assigned
                   [X.sub.6] = Total 7-
                   Levels Assigned

Interactions:      [X.sub.10]/[X.sub.12] =
(Factors)          3-Levels per 7-Level

Higher Order:      No significant higher
(Factors)          order terms were
                   revealed.

The original X variables were renumbered to simplify the final
explanatory model.
Figure 8.

Forecasting Model--Version 1

Y = [[beta].sub.0] + [[beta].sub.1][X.sub.1] + [[beta].sub.2][X.sub.2] +
[[beta].sub.3][X.sub.3] + [[beta].sub.4][X.sub.4] +
[[beta].sub.5][X.sub.4]/ [X.sub.3] + [euro]


Forecast:          Y = Predicted F-16C/D
                   Mission Capable Rate

Original Effects:  [X.sub.1] = Sorties
(Factors)          [X.sub.2] = Flying Hours
                   [X.sub.3] = Average Aircraft
                   Inventory
                   [X.sub.4] = Total F-16 Maint
                   Personnel Assigned

Interactions:      [X.sub.4]/[X.sub.3] = Total
(Factors)          F-16 Maint Personnel per Acft

Higher Order:      No significant higher order
(Factors)          terms were revealed.
Figure 10.

Forecasting Model--Version 2

Y = [[beta].sub.0] + [[beta].sub.1] [X.sub.1] + [[beta].sub.2][X.sub.2]
+ [[beta].sub.3][X.sub.3] + [[beta].sub.4][X.sub.4] +
[[beta].sub.5][X.sub.4]/[X.sub.3] + [epsilon]


Forecast:          Y = Predicted F-16C/D Mission
                   Capable Rate

Original Effects:  [X.sub.1] = Sorties
(Factors)          [X.sub.2] = Average Aircraft
                   Inventory
                   [X.sub.3] = Total 5- and 7-levels
                   Assigned (Lagged 4 Qtrs)
                   [X.sub.4] = Total, O-3 Maint Officer
                   Assigned (4024/21A3) (Lagged 3 Qtrs)
                   [X.sub.5] = Total 9-levels Assigned
                   [X.sub.6] = Percentage of [2.sup.d]
                   Term Eligibles Reenlisting

Interactions:      No significant interactions were
(Factors)          revealed.

Higher Order:      No significant higher order
(Factors)          terms were revealed.

Notes

(1.) Robert H. Williams, "Readiness Pledge by Pentagon Prompts Challenge from Congressional Leader," Notional Defense, Jul-Aug 97, 82. 24-25.

(2.) Dick Stocchetti, ACC Readiness Trends and Outlook, Senior Leaders Maintenance Course, Mar 01 [Online] Available: https://lg.acc.af.mil/lag/lag.htm, Mar 01.

(3.) Russell P. Hall, HQ USAF/IL, Total Not Mission Capable for Maintenance Project Description Memorandum, 5 Jan 00.

(4.) Process Guide/Software Users Manual and Functional Description for FAMMAS. Massachusetts: Dynamics Research Corporation, Jul 97.

(5.) Mark Humphrey, "NMCM Escalation and Erosion of Mission Capable Rates," DRC Contract #GS-35F-477SG, Sep 99. and Carl J. Dahlman and David E. Thaler, Assessing Unit Readiness: Case Study of an Air Force Fighter Wing, Rand Corporation. Report #RAND/DB-296-AF, 2000.

(6.) Andy Sherbo, "Operation and Maintenance Funding and the Art of Readiness." The Air Force Comptroller, Apr 98, 32, 10-14.

(7.) Spyros Makridakis, Steven C. Wheelwright, and Rob J. Hyndman, Forecasting: Methods and Applications, [3.sup.d] ed. New York, New York: John Wiley and Sons, Inc, 1998.

(8.) Humphrey.

(9.) Dahlman and Thaler; Humphrey; and Leonard R. Gauthier, 1st Fighter Wing Aircraft Abort Study, 1 OSS/OSMA, ACC, 1998 [Online] Available: https://lg.acc.af.mil/lgp/lgpp/study.shtm.

(11.) Ibid.

(10.) Humphrey.

(12.) Steven A. Oliver, "Cost and Valuation of Air Force Aircraft Maintenance Personnel Study," AELMA Project #LM200107900, Aug 01.

(13.) Patrick Bresnahan, Course Lecture, LOGM 203, Reliability and Maintainability, Air Force Institute of Technology. Wright-Patterson AFB, Ohio, 17 Mar 98.

(14.) Ibid.

(15.) William Matthews, "Midlife Crisis--Aging Planes Are a Maintenance Challenge." Air Force Times, 58, 12-15, 20 Apr 98.

(16.) Andy Bouck, "F-16 Service Life Issues Briefing" 2000 F-16 Worldwide Maintainer's Conference, ACC/DR, [Online] Available: https://www.mil.acc.af.mil/dr/staff/dra/f16/conf/day1.shtm, 13 Sep 00, and Mike Paddock, "FALCON STAR (Structural Augmentation Roadmap Briefing)," 2000 F-16 Worldwide Maintainer's Conference, OO-ALC/ YPVS [Online] Available: https://www.mil.acc.af.mil/dr/staff/dra/f16/conf/day1.shtm, 13 Sep 00.

(17.) John E. Bell, "Total Not Mission Capable for Maintenance Study," AFLMA Project #LM199934800, Oct 00.

(18.) Telephone Interview with Edward Merry, Chief, Assessments Division, ACC/LGP, 4 May 00.

(19.) Humphrey and Bell.

(20.) Bell.

(21.) E-mail with Marti Hamm, Logistics Analyst, Directorate of Supply, Deputy Chief of Staff Installations and Logistics, 24 May 99.

(22.) Jay Heizer and Barry Render, Principles of Operations Management, [3.sup.d] ed, Englewood Cliffs, New Jersey: Prentice-Hall Inc, 1999.

(23.) Bailey.

(24.) Humphrey.

(25.) Matthews, 12-15.

(26.) Stanley Sieg, AFMC/LG, "Improving Customer Support Through Supply Chain Management and Constraints Analysis Briefing," 16 Jun 00.

(27.) Heizer and Render.

(28.) A. J. Bosker, "Air Force Continues to Reverse Spare Parts Shortfall," Air Force Print News, 001121, 27 Jul 00, and F. Whitten Peters, "Readiness Challenges of Today's Air Force," Remarks to the Air Force Association National Convention, Washington DC, 13 Sep 00.

(29.) "Management Actions Create Spare Parts Shortages and Operational Problems," Report #NSIAD/AIMD-99-77, Washington DC: Government Printing Office, Apr 99.

(30.) Guy R. Vanderman, "Time to Tweak the AF's Approach to 2LM?" The Exceptional Release, Summer 1998, 70, 10-12.

(31.) Peters.

(32.) Capt Charles Porter, et al, "Reduction of the Recoverable Pipeline," HQ AFLC/MM/XP/MAIDS study, Wright-Patterson AFB, Ohio, Nov 90.

(33.) KPMG Peat Marwick LLP, "F-1b Avionics Logistics Chain Management Study," HQ AFMC, Lean Logistics Program Office, Wright-Patterson AFB, Ohio, 30 Jan 98.

(34.) E-mail with Synergy, Inc. "Pipeline Times for the USAF," 1 Jul 99, and Government Accounting Office.

(35.) Marvin A. Arostegui, Jr. Class Lecture, LOGM 628, "Reparable Inventory Management," Graduate School of Engineering and Management, Air Force Institute of Technology (AU), Wright- Patterson AFB, Ohio, May 00.

(36.) Synergy.

(37.) Bradley M. Kettner and William M. Wheatley, "A Conceptual Model and Analysis of the Air Force Depot Supply and Maintenance Pipeline for Reparable Assets," MS thesis, AFIT/GLM/LSM/91S-37, School of Systems and Logistics, Air Force Institute of Technology (AU), Wright-Patterson AEB, Ohio, Sep 91.

(38.) KPMG.

(39.) Sherbo.

(40.) Sherbo and Humphrey.

(41.) Defense Science Board Task Force on Readiness, "Report of the Defense Science Board Task Force on Readiness," DTIC Report #ADA286412, Jun 94.

(42.) Defense Science Board Task Force on Readiness; Humphrey; and John Pulley, "Bigger AF Budget to Ease, Not Cure, Problems," Air Force Times, 59, 10 Feb 99.

(43.) Sieg.

(44.) Alan Vick, David T. Orletsky, Abram N. Shulsky, and John Still ion, "Preparing the US Air Force for Military Operations Other Than War, "Rand Corporation, Report #RAND/RB/48, 1997.

(45.) Defense Science Board Task Force on Readiness and Humphrey.

(46.) Julie Bird, "Stretched to the Breaking Point, Air Force Times, 57, 3-4, 21 Apr 97, and Rick Maze, "Lawmakers Seek Report on Eroding Readiness," Air Force Times, 59, 4, 4 Aug 98.

(47.) Charlie Krueger, Lockheed Martin Tactical Aircraft Systems, Fort Worth, Texas, 1999.

(48.) William P. Hallin, "The Challenge of Sustaining Older Aircraft," Air Force Journal of Logistics, Vol 22, No 2, Summer 1998, 1-2.

(49.) Matthews.

(50.) Michael E. Ryan, "Testimony Before the US Senate Armed Services Committee," Washington DC, 5 Jan 99.

(51.) Humphrey.

(52.) Bell.

(53.) Gregory P. Bernitt, "Maintenance Chief's View: The Objective Wing," The Exceptional Release, 59, 6, Fall 1995; Timothy A. Kinnan, Memorandum, "Aircraft Maintenance Assessment," 347 WG/CC to ACC/CC, Aug 95; and Joseph W. Ralston, HQ ACC/CC Memorandum, "Aircraft Maintenance Memorandum for ACC Units Down to and Including Wing/CC," 21 Jul 95.

(54.) ACC Message 071421Z, Deputy Operations Group Commander for Maintenance, Mar 97.

(54.) Makridakis, et al.

(55.) Personal interview with Greg Dierker, Logistics Analyst, Aeronautical Systems Center, Engineering Directorate, Wright-Patterson AFB, Ohio, 20 Jul 00.

(56.) Interview with Greg Dierker, Logistics Analyst, Aeronautical Systems Center, Engineering Directorate, Wright-Patterson AFB, Ohio, 20 Jul 00.

(57.) DRC and interview with Kurt Reynolds, Logistics Analyst, DRC, 10 Apr 00.

(58.) Defense Science Board Task Force on Readiness.

(59.) Process Guide/Software Users Manual and Functional Description for FAMMAS.

(60.) AFMC Manual 23-1, Recoverable Consumption Item Requirements System D041), 4 Dec 97.

(61.) John Neter, Michael H. Kutner, Christopher J. Nachtsheim, and William Wasserman, Applied Liner Statistical Models, 4th ed, Chicago: 1996, and Edward D. White, Class Lecture, STAT 535, Managerial Statistics Il, Graduate School of Engineering and Management, Air Force Institute of Technology (AU), Wright-Patterson AFB, Ohio, Feb 00.

(62.) Makridakis.

Captain Oliver is Chief Maintenance Plans and Programs Branch, Maintenance Division, Air Force Logistics Management Agency, Maxwell AFB, Gunter Annex, Alabama. Lieutenant Colonel Johnson, Assistant Professor Logistics Management; Major White, Assistant Professor Statistics; and Major Arostegui, Assistant Professor Logistics Management, are all faculty members at the Air Force Institute of Technology, Wright-Patterson AFB, Ohio.

COPYRIGHT 2001 U.S. Air Force, Logistics Management Agency
COPYRIGHT 2004 Gale Group