Pattern recognition software and dramas of deception: new challenges in electronic financial services

Patterns of deception are built on drama. Like magicians at a circus, those practicing the deception must divert attention from what is actually taking place by creating another story that masks reality. We may like to refer to these people as "bad actors," but the successful ones are very good actors indeed. Six tips are provided to help the banker make these actors take their final bow.

Pattern recognition software is one of the most promising growth trends in financial services in this decade. Today's reality is that pattern recognition software is everywhere. A species of artificial intelligence, it influences business and consumer behavior in ways that most people do not the take time to recognize, much less analyze. amazon.com uses software to identify the buying habits of its customers and present them with items they are likely to want based on their buying history. Credit-scoring software has become so sophisticated that risky customers can be spotted based on behaviors, like poor driving records, that superficially have nothing to do with credit management. Credit card issuers, engaged in a war of attrition for good customers, fling computer-generated, attractively priced loan offers at customers whose payment histories show large monthly payoffs. So where does all this end? Is it a net benefit to financial services companies and their customers? What are the pitfalls for the unwary?

Foundations of Growth

The proliferation of pattern recognition software (PRS) is rooted in several continuing developments. Most fundamentally, continuous advances in computer technology make it possible to write and execute algorithms that identify important patterns across large populations of data and that respond to the observed patterns in ways that promote desired behaviors and discourage undesirable ones. Customer relationship management software is a leading example.

Supporting the development of this technology are recent legal changes that encourage and protect it. From the 1960s to the late 1970s, the U.S. Supreme Court said software could not be protected by patent. Software, the reasoning went, was merely the expression of mathematical formulas. As such, anyone could create or recreate them, and patent protection should not be available. Beginning in the early 1990s, the U.S. Court of Appeals for the Federal Circuit began to take a different tack. "The proper inquiry in dealing with the so-called mathematical subject matter exception is to see whether the claimed subject matter as a whole is a disembodied mathematical concept ...," the court wrote in a 1994 case. (1) Four years later, in the State Street Bank case, the court made clear its intention to reverse decades of settled case law. The court reasoned that the software in question no longer represented only mathematical processes, but constituted a machine when operated in tandem with associated hardware. As a machine, the software was an invention under federal patent law. (2)

The result of these developments has been an explosion in the number of patents granted in patent class 705--financial, business practice, management, or cost/price determination (data-processing) patents. During the early 1980s, the U.S. Patent and Trademark Office issued an annual average of 60 patents in this class. During the early 1990s, the annual average rose to 208. By 2001, the last year for which data was published, the annual average reached 980, equivalent to all the patents of this class issued during the 1980s! Equally significant, leading financial services companies now join technology companies as recipients of the largest numbers of these patents.

PRS Classified

As the class of patented software has expanded, so have its uses. Pattern recognition software, in particular, can be classified into at least four categories, according to the pattern that the software works to deconstruct.

Patterns of obligation uses software to measure how well or poorly people meet their obligations. Credit-scoring software is the leading example. Patterns of occupation automate banking processes previously performed by humans. This class holds huge potential for transforming the industry. Examples of processes being automated include the entire lending process, from application through administration, the process of syndicating loans, the process of selling securities to the public, and any number of other labor-intensive endeavors associated with financial services.

Patterns of desire are the focus of customer relationship management software that retailers like Amazon.com use. Rather than ask the open-ended question embodied in Microsoft Corporation's trademarked phrase, "Where would you like to go today?" this software builds on the enterprise's knowledge of the customer's buying habits and drives the customer to repeat transactions in an area that the software recognizes as a sweet spot for that customer. If a customer's data file shows that he or she typically finances a European vacation every three years with a $10,000 installment loan, why should the software not invite the customer to increase the frequency of vacations to every two years if the monthly payment to retire the associated debt can be kept nearly the same? Conversely, if a cohort of customers can be statistically proven to be indifferent to fluctuation in deposit account interest rates, why should their bank not use that information in its asset/liability model to support the purchase of longer-bond maturities in the bank's investment portfolio?

Patterns of deception software is in some ways the mirror image of software that focuses on patterns of desire. In each case, the purpose of the software is to cause the recipient to do something the recipient likely would not otherwise do. With patterns of deception, the software's focus can be either offensive or defensive.

PRS and Patterns of Deception

The recent rash of "phishing" offers a case of both offensive and defensive software deployed to practice or defeat patterns of deception. To perpetrate an attack of phishing, the actor deploys software that queries consumers under false pretenses in order to obtain confidential financial information, such as credit card account information, social security numbers, or other valuable data. Financial institutions and major Internet service providers have deployed software countermeasures in an effort to mitigate losses that result from misappropriation of consumer financial data; nevertheless, the AntiPhishing Working Group reports that up to 5% of consumers respond to phishing--the cumulative number of attacks having quintupled to nearly 5,000 from April through June 2004. (3) Unreported is the number of attacks made against financial institution computers in more sophisticated efforts to steal money, information, or both.

On a larger scale, computers and computer software have served as tools of bad actors involved in several significant bank failures. False computer reports of loans being serviced were one means by which executives of the failed First National Bank of Keystone deceived the bank's independent accountants and federal bank examiners. Next Bank, N.A., a credit card bank built on providing under-30-second responses to on-line loan applications, failed when soon-to-be-unemployed consumers flocked to sign up for NextCard accounts. Most recently, the Oakwood Deposit Bank in Ohio failed after its president used the Internet to recruit high-rate deposits that he diverted to fund improper personal investments.

The Engineering Trap

As the financial services industry continues to transition to one based on electronic financial services, and as pattern recognition software grows in importance, it will be tempting to view patterns of deception as ones that can be solved with a software engineering skill set: measures and countermeasures, just as in warfare. And yet if financial services executives take that approach alone, they risk defeat. For to truly manage the risks associated with patterns of deception, they must appreciate and devote sufficient energy to the human dimensions of the problem.

Patterns of deception, at bottom, are built on drama, lake magicians at a circus, those practicing the deception must divert attention from what is actually taking place by creating another story that masks reality. The most successful Allied deceptions of World War II were those that the German or Japanese high commands accepted because they wanted to believe them. Hitler persisted in his belief that the Normandy invasion was a diversion, despite mounting evidence to the contrary, because he wanted to believe he had deciphered the Allies' battle plan. The false intelligence that diverted bombs from targets around London after the British broke the Germans' Enigma code succeeded because the German high command wanted to believe its code could not be broken. The Allies' success at preventing the Nazis' bombing of Suez Canal came because the British recruited the leading magician in England at the time. He suggested a diversionary light show over a strategically insignificant area of Egypt, which the Germans' bombed, thinking it the real Suez Canal. Again, they bought the diversion because they wanted to believe their bombs had found their mark.

Although these examples may seem far removed from the emerging world of electronic financial services, they are not. To prevent pattern recognition software from being used to deceive financial institutions and their managements, executives and managers must focus on their motivations and those of their customers as much as on the technology that will be available to prevent bad outcomes. Human motivations matter most. Machines, patented or not, merely give expression to those motivations. To avoid the engineering trap, keep these guidelines in mind:

1. Recognize the centrality of the information technology function. Your institution's safety and soundness depend on it. Acknowledge that a rogue information technology employee presents maximum risk to the organization and build redundancies accordingly. One of this author's clients was prepared to change out its entire IT system ostensibly because it would not produce desired reports. Only when the IT systems provider challenged the premise that the reports could not be produced did the CEO probe further. When he did, he learned that his treasurer and IT director were preventing the reports from being produced in order to mask an embezzlement scheme they had been practicing for years. As pattern recognition software grows in importance to institutional mission, risks like these will grow commensurately.

2. Beware the IT know-it-all. Often information technology officers feel threatened by the presence of outside experts. At one level, this is an understandable, if unacceptable, manifestation of their sense of vulnerability. At another level, it may be a tip-off to their infidelity. In the current environment, no expert-inside or outside a financial institution-knows all and cannot benefit from others' knowledge and experience.

3. Minor inconsistencies can be major tips. Significant problems are often revealed by what appear to be minor inconsistencies or even contradictions. When they present themselves, they seem insignificant yet remain puzzling. Rather than dismiss them, ponder them. What do they really mean? What more is there to the story? What am I missing that would make them make sense?

4. Insecure employees are at-risk employees. With employment security in financial services companies declining, employees are increasingly likely to act against the company's interest--and their own. Although it may not be possible to increase employees' real job security, any measures that can be taken to boost their morale will pay dividends in reduced risk of dishonesty and fraud. Increased vigilance is also necessary.

5. Extraordinary performance may conceal malfeasance. This axiom of banking has been well understood for decades. Banking law codified it with a requirement that bankers be out of the bank for a period of weeks each year. In the realm of technology, directors and executive officers have been reluctant to impose similar requirements because of the scarcity of talent and the (accurate) perception that certain IT professionals are mission critical. None of these reasons, however, excuses the need to build in appropriate redundancies and backup staffing.

6. Linear thinking, can be dangerous. Programmers, accountants, bankers, lawyers, engineers, and others with highly specialized training they apply to solve very specific problems can become too rigid in their thinking as a result. Identifying risks of deception and fraud requires a way of thinking more characteristic of the clergy, theorists in mathematics and physics, or classically trained scholars. These were the occupations of those who broke Germany's Enigma code. Men and women like them are those best suited to deciphering patterns of deception, real or potential. Linear thinking has its place. So does thinking in terms of alternative realities.

These tips are far from an exhaustive catalog of measures that can be taken to prevent patterns of deception, embodied in software or otherwise, from making victims of you and your institution. Yet they do provide reference points from which financial institution executives can formulate an approach that will help them avoid being victimized in the emerging world of electronic financial services.

James Bauerle may be reached by e-mail at jbauerle@dkwlaw.com or jbauerle@ficap.com.

Notes

(1) In re Alappat, 33 F.3d 1526, 1544 (Fed. Cir. 1994).

(2) State St. Bank & Trust Co. v. Signature Fin. Group, 149 F.3d 1368, 47 U.S.RQ.2d 1596 (Fed. Cir. 1998), reversing State St Bank & Trust Co. v. Signature Fin. Group, 927 E Supp. 502 (D. Mass 1996).

(3) See http://www.antiphishing.org.

James Bauerle is director of Legal and Business Services at DKW Law Group, LLC, a business lace, firm with offices in Cleveland, Detroit, Harrisburg, Pittsburgh, and Washington, PC. He has served as an executive officer or director of publicly traded and privately held financial services companies and is a founding principal in FiCap Strategic Partners, LLC, a consulting firm that positions financial institutions for the future.

COPYRIGHT 2004 The Risk Management Association
COPYRIGHT 2005 Gale Group