The use of consultants in law enforcement

Police departments sometimes turn to outside consultants to treat specific departmental problems. There are valid reasons for hiring a consultant to obtain an organizational evaluation of a police department. The time and talents needed for this particular task often are not available internally. Additionally, the built-in biases and subjective beliefs held by officers within the department may make an outside evaluator necessary to assess the true state of the department's operations or to determine the true problem when a conflict exists among agency managers.

However, while private consultants can be extremely beneficial to law enforcement agencies, administrators should enter into such arrangements with caution. First, they must know how to choose a suitable consultant. Second, they must educate themselves in research methods so that they can analyze effectively the results of any study produced by the consultant.

This article discusses the experience of the Pasadena, Texas, Police Department, which hired a consulting firm to conduct a study of the department. It also discusses how to select a consulting firm, how to work cooperatively with the consultant, and how to interpret study results.

THE PASADENA EXPERIENCE

The Pasadena Police Department retained the services of a private management group to evaluate the department. This group produced a report that recommended a revamping of the management structure and operations of the department in order to improve its overall performance. The group maintained that implementing the recommendations would enhance the effectiveness and efficiency of departmental services. It further asserted that while service quality would be maintained or improved by implementing the recommendations, departmental costs would be reduced by some $1.4 million, a substantial savings to the department's $13 million annual budget.

Upon close review, however, the study appeared flawed. Department managers found the study data skewed and many of the suggestions to be unworkable. The managers' familiarity with the inner workings of the department enabled them to recognize these flaws immediately. Obviously, the consulting group's evaluation achieved desired cost reductions but failed to provide practical recommendations for improvement.

How, then, should police executives select a consultant in order to avoid such pitfalls? There are several critical considerations for administrators who decide to employ an outside consultant.

SELECTING A CONSULTANT

Reference Sources

Once administrators decide on the type of evaluation needed, they can contact several police professional associations, such as the International Association of Chiefs of Police, the National Organization of Black Law Enforcement Executives, the Police Executive Research Forum, and the Police Foundation, to request a list of potential consultants. Professional publications and journals and the National Criminal Justice Reference Service are also excellent sources for information on consultants. However, networking with other police executives may be the best way to find a consultant, because they most likely will refer one with whom they have worked and one familiar with the police subculture.

The next step is for administrators to contact prospective consultants to request a list of past clients. Administrators then should contact these clients for an assessment of the consultant's performance.

Expertise

Consultants who evaluate law enforcement agencies need to possess certain critical skills, including both substantive and methodology expertise. Methodology expertise includes knowledge of research design, data collection, and statistical analysis procedures. Substantive expertise includes knowledge of the problem and any law, rule, and regulation relating to it.

Consultants hired by law enforcement agencies also should have experience in working with law enforcement professionals. A knowledge of current Federal, State, and local laws; familiarity with standard, acceptable police procedures; and a thorough understanding of the police subculture are essential if evaluations and recommendations are to be accepted in the police environment. If consultants lack this knowledge and understanding, they most likely will make illogical recommendations.

For example, the Pasadena study recommended several hierarchical changes that conflicted with civil service laws governing the department. In addition, the study called for several operational changes that restricted officers' discretion in making arrests. It further suggested that officers no longer investigate minor traffic accidents and that officers issue misdemeanor citations more often.

Department members believed these operational changes to be incongruent with the desires of local citizens. Several mid- and upper-level managers expressed disappointment with the study results because the recommendations called for immense changes in the operations, organization, and culture of the department. In fact, one patrol commander stated that the study was plagued with both errors and contradictions and that the recommendations were out of touch with the realities of police service in the community.

As another consideration, police executives could seek out consultants with a background in the specific problems being experienced by the department. Some experts believe that the firm's experience does not need to be in the same field. However, as the Pasadena experience proved, a lack of knowledge of police techniques severely hampers the effectiveness of a consulting firm working for a law enforcement agency.

WORKING WITH CONSULTANTS

Once a consultant is chosen, both the consultant and the police executive should work to establish mutual expectations and open communication. Clearly defined roles facilitate clear communication. Police executives must delineate both their perceptions of the problem and their expectations of the consultant. In turn, the consultants must detail their projected services to the department.

To delineate the department's problem effectively, police executives should avail themselves to critical resources. They can identify and contact knowledgeable community leaders and experts, listen to attendees at open forums, and check existing statistics, records, and data from surveys and censuses.

When executives clearly delineate problems, not simply the symptoms of an overall problem, the consultant can offer workable solutions to them. Unfortunately, many administrators focus on whatever flame is burning the brightest at the time. In 6 to 8 months, however, when the consultant's study is completed, that flame may not be the brightest one, and the offered solutions may no longer be critical to the organization.(1)

With a clear understanding of both the problem and their expected role, consultants have the necessary foundation to evaluate and to make a diagnosis, thereby avoiding helter-skelter research efforts. To facilitate this effort, administrators should assign a team or representative to act as liaison with the consultant. The liaison provides necessary information to the consultant and implements any new program after the consultant's assignment ends.

Liaison is critical to a successful experience with consultants. Had the Pasadena Police Department appointed a liaison team, the final product might have been less flawed. This, in turn, may have resulted in more credibility being placed on the valid recommendations offered by the study.

REVIEWING THE STUDY

When a consultant's final report recommends changes in operational procedures, an overhaul of the organizational structure, or a reduction in the work force, should the department automatically implement all of the changes? The answer to this question is an unequivocal "no." Police executives must review the study and interpret its results before deciding what recommendations to implement. Organizational costs could be enormous if police executives implement inappropriate changes based on faulty research.

In order to determine whether any problem exists with the study, administrators first need to examine the specifics of how it was conducted. By doing this, they often can ferret out problem areas and adjust final recommendations accordingly.

To begin, administrators must attempt to separate good research from bad research. This means that today's police administrators must understand research methods. An understanding of these methods allows them to review the study to determine how the consultant reached certain conclusions and whether these conclusions are valid.

Research Techniques

Applied research involves collecting and analyzing data regarding a specific problem. This type of research assists police managers in making decisions.(2) Police executives use applied research when they allocate personnel, decide what type of police equipment to purchase, study crime trends, and make other administrative decisions.

There are two types of applied research: Descriptive and evaluative.(3) Descriptive research identifies the size of the problem, its causes, and who is affected. For example, this type of research involves looking at calls-for-service data to determine the number of calls, the purpose for the calls, and the effect of the calls on officer workload.

Evaluative research, on the other hand, compares one program to an alternative program. Examples of this type of research include determining the effectiveness of having a traffic accident investigations unit or determining what model car would best suit the needs of the patrol unit. Evaluative research ascertains whether a program, policy, procedure, or purchase is doing what it should do; how the program might be deficient; how the program can be improved; and whether the program should be eliminated.

Police executives who understand basic research techniques can review a consultant's study to determine whether any problem exists. This critical step ensures that the department does not make changes based on faulty research techniques.

Problem Research and Research Design

When reviewing a study, the administrator must determine whether the consultant had a clear problem formation and research design. It is the administrator's responsibility to delineate the purpose of the study--exploration, description, explanation, or a combination of these. For the most part, law enforcement agencies require a combination of descriptive and evaluative research.

Individuals Involved in the Study

Administrators also must consider who conducted the study to ensure that individuals hired by the consultant understood police problems and practices. Individuals working on the study also should have research methodology expertise and the ability to interpret the findings and report them accurately. In addition, they should be familiar with the law, the police subculture, and the community's needs.

Variables

It is important for administrators to determine whether researchers delineated different dimensions of the variables being studied and whether they operationalized these variables correctly. For example, if researchers report that crime in the city is up, administrators should determine how they arrived at this conclusion. Did the researchers define crime? Perhaps, they based their conclusion on the fact that the total number of police calls for service increased during the previous year, and this is an operationalized definition used by them to make the increased crime rate claim.

Administrators then must decide whether the calls-for-police-service variable is a valid measure of crime. For example, a community-based, nonpolice assistance program implemented in the previous year may have increased the number of calls for service to the police without affecting the crime rate.

Police executives also should consider whether the crime rate for certain crimes increased, while the number of more serious crimes decreased. For example, thefts in the jurisdiction may have increased while the number of murders, sexual assaults, and robberies decreased. Or, perhaps the previous year was the first year that theft of items valued under $20 was reported along with all other theft charges, causing an inflated theft statistic.

In the case of the Pasadena Department, the study asserted that the department had a slow response rate--an average of 15 minutes per call. However, individuals conducting the study arrived at this figure by averaging critical calls, such as shootings and felony crimes in progress, with complaints of loud music and a barking dog. Clearly, the consultants' data skewed the findings. Administrators must always question whether the data obtained actually provide accurate information about what supposedly is being measured. Accurate results require accurate data.

Sampling Methods

Administrators then must look at the sampling methods employed by the consultant. Did the consultant select a sample of employees or were all employees contacted?

When assessing a policy or a procedure in a large agency, sampling is probably appropriate. However, a modest sample from a small- or medium-sized department may not reflect the true beliefs of the organization.

If a sample is used, it is crucial for the consultant to describe the type of sampling on which the information is based--convenience, simple random, systematic, or cluster sampling. How well a sample represents the entire agency depends on the sampling frame and the specific design of selection procedures. Did the consultant select a sample out of convenience, or does the sample truly represent the organization? For example, did the study conveniently sample a particular segment of employees, such as detectives or day-shift officers, to the exclusion of others?

Statistically speaking, a sample can be representative of only the population included in the sampling frame. Each person in the organization must have a known chance of being selected by the sampling procedure used. Sampling and analyzing data from a sample can be fairly straightforward if an accurate employee list is used as a sampling frame, if a simple random or systematic sampling scheme is used, and if all respondents are selected at the same rate.

Also critical is whether the consultant selected an appropriate sample size and what percentage of the sample responded by completing the instrument (survey questionnaire). The sample should be representative of the organization and should reflect the gender, age, rank, ethnicity, and education level of the entire organization. The researcher must estimate the size of the sample in order to provide adequate representation of these organizational subgroups. Most decisions about sample size do not focus on estimates for the total population. Rather, they concentrate on the minimum sample sizes that can be tolerated for the smallest subgroups of importance.

Gathering Data

Administrators need to understand how the consultants gathered data. Did they use survey or field research, or did they analyze existing data or experimental research? Typically, consultants rely on several types of research methodology. However, several internal and external validity questions arise with each type of research method.

Survey Instruments

Research can be gathered using written or orally administered survey instruments. If researchers used a written survey, police administrators need to determine the validity of the questions. Additionally, if researchers administered an oral survey, police executives should ascertain whether interviewers asked the survey questions differently or perhaps even asked different questions.

Four types of questions can appear on surveys: Demographic, behavior-oriented, knowledge, and attitude.(4) Researchers must construct the questions properly to obtain internal and external validity. They also must construct the questions in a manner that allows the respondent to give clear, unambiguous answers. Questions containing negative terms could cause the respondent to misunderstand what is being asked.

There should be no compound questions, and the respondents should be capable of answering all questions. Survey questions should be applicable to all who are asked to answer them.

Field Research

Field research is appropriate to police research topics that defy simple quantification. This type of research may identify the nuances of specific attitudes or behaviors that might escape researchers using other methods. Researchers who employ this method go directly to the problem under study and observe it as completely as possible, which allows them to develop a deeper understanding of the problem.

However, field research has potential problems that can affect the validity of the study. For example, the researchers' own cultural identities or backgrounds could color their interpretation of what they observe. Not all researchers who observe the same events would classify things in the same manner.

Existing Statistics

Administrators need to determine whether researchers relied on existing statistics. For instance, to study traffic accidents, they may have relied, in part, on traffic flow statistics supplied by the local traffic or transportation department. In such cases, administrators should ascertain who originally collected the data, when the data were collected, and whether the findings were appropriate to current concerns.

Experimental Design

Another way to gather information is through the experimental design method. In these cases, administrators need to establish what other variables may have influenced the findings. For example, did the researcher use a proper control group? Were the experimental subjects properly assigned to experimental and control groups? Did the testing process affect the results? Did the fact that an interest was shown in a particular group affect its performance irrespective of the independent variable's effects? Above all else, the findings of the experiment must be applicable to real-world situations.

CONCLUSION

Whenever possible, police executives should rely on their employees to collect and evaluate organizational data. However, when necessary, private consultants can be of immense assistance to law enforcement agencies. They can provide fresh ideas and an objective evaluation of the department.

Consultants who work cooperatively with departmental personnel--especially the chief executive--can produce studies that educate administrators on how they can improve greatly the effectiveness of their departments. In the end, this benefits not only the department but also the community it serves.

Endnotes

1 John H. Sheridan, "Where Bench-markers Go Wrong," Industry Week, March 1993, 28-31.

2 J. Eck, Using Research: A Primer for Law Enforcement Managers (Washington, DC: Police Executive Research Forum/National Institute of Justice, 1974), 3.

3 Ibid.

4 F. Leavitt, Research Methods for Behavioral Scientists (Dubuque, Iowa: William C. Brown, 1991).

COPYRIGHT 1994 Federal Bureau of Investigation
COPYRIGHT 2004 Gale Group