Spero Consulting

Update: What tim­ing! Moments after we pub­lished this, we saw this col­umn, Intel­li­gence Is a Ter­ri­ble Thing to Waste, by L. Gor­don Crovitz at The Wall Street Journal’s web site. It nicely com­ple­ments our post and val­i­dates a few of our spec­u­la­tions – although we must admit that his col­umn has a catch­ier title.

In this rather long post we spec­u­late about a pos­si­ble under­ly­ing cause of the “intel­li­gence fail­ure” involv­ing Umar Farouk Abdul­mu­tal­lab, the Niger­ian accused of try­ing to blow-​up North­west Flight 253 on Christ­mas Day. Of inter­est is how he was cleared to fly despite his father noti­fy­ing U.S. author­i­ties of his – the son’s – extrem­ism and poten­tial for terrorism.

Note that we have absolutely no pri­vate infor­ma­tion regard­ing either the inci­dent or gov­ern­ment infor­ma­tion sys­tems; so, we spec­u­late based upon our knowl­edge of other large, bureau­cratic orga­ni­za­tions with rigid, poorly-​designed systems.

We real­ize that incen­tive prob­lems – which result in the unwill­ing­ness of agen­cies and indi­vid­u­als to share data and infor­ma­tion across juris­dic­tions – and our free­doms and rights con­strain the effec­tive­ness of inves­tiga­tive efforts, but for the most part, we’ll ignore those issues to focus on infor­ma­tion systems.

Com­mon MIS Issues & Problems

A few weeks ago we wrote Inex­pen­sive but Valu­able Web-​based MIS. Besides describ­ing those ben­e­fi­cial sys­tems, we men­tioned that many so-​called “man­age­ment infor­ma­tion sys­tems” are, in fact, merely data-​processing and record-​keeping sys­tems (for trans­ac­tions and events).

Such sys­tems rarely pro­vide infor­ma­tion – decision-​altering con­tent – for the types of strate­gic deci­sions made by senior man­agers, and unfor­tu­nately, they may not be well-​designed to pro­vide use­ful tac­ti­cal infor­ma­tion, either. That’s the case if the systems:

1. Pro­duce use­less stan­dard­ized out­put (reports);
2. Are dif­fi­cult to fully access or query; or
3. Don’t adapt quickly or well to changes in the envi­ron­ment, oper­a­tions or insti­tu­tional knowledge.

In Umar Farouk Abdulmutallab’s case, we sus­pect that it is the inher­ent rigid­ity of the data­base appli­ca­tion and/​or the rigid­ity of the design­ers’ thought processes that are to blame. (Note that for new infor­ma­tion sys­tems, use­less stan­dard­ized reports result when sys­tems design­ers don’t ask users the cor­rect ques­tions or do ask the right ques­tions, but don’t really under­stand the replies. See Details Are Not Infor­ma­tion for more on this topic. One of our MIS friends often remarks that her key func­tion is to serve as a trans­la­tor between sys­tem users and sys­tem devel­op­ers, and that role is crit­i­cal but too often ignored. For older sys­tems, irrel­e­vance and obso­les­cence usu­ally result when the sys­tem isn’t easy to change.)

What Went Wrong on Christmas?

When bad things hap­pen, i.e., when some­one like Umar Farouk Abdul­mu­tal­lab squeezes through the detec­tion sieve, it is pos­si­ble that noth­ing failed. One must con­sider that the detec­tion sys­tem – the net, the fil­ter, the web – may not have designed to catch every­thing and that the designer or owner con­sid­ered a cer­tain level of error or mis­clas­si­fi­ca­tion to be accept­able. The designer may have con­cluded that a per­fect, error-​free sys­tem is too expen­sive to develop and main­tain.¹

How­ever, the fail­ure in the Abdul­mu­tal­lab case was so egre­gious that it seems far more likely that either the detec­tion sys­tem was either incom­pe­tently designed or administered.

Now, it is quite pos­si­ble that a gov­ern­ment sen­try or sen­tinel fell asleep or neglected his or her respon­si­bil­ity. In that case, it is both a human error – because a per­son failed – but also a sys­temic error because there was no redun­dancy or backup mit­i­gate such error. How­ever, rather than crit­i­cize gov­ern­ment employ­ees involved with the nation’s secu­rity, we’ll assume that they are earnest, capa­ble, and hard-​working as we believe that is true.

In that case, it must be that despite their best efforts, the detec­tion sys­tem failed, and one rea­son for the fail­ure could be the improper design of the government’s infor­ma­tion system.

One obvi­ous weak­ness in the ter­ror­ist detec­tion sys­tem – and it is by design – is the government’s unwill­ing­ness to use con­di­tional prob­a­bil­i­ties to assess the like­li­hood that some­one is a ter­ror­ist, espe­cially if the per­son is a for­eigner and is not pro­tected by our Con­sti­tu­tion and Bill of Rights. As we wrote in The Absur­dity of Has­sling Grandma but not Nidal Hasan, we do blame the gov­ern­ment (and Pres­i­dent Obama) for main­tain­ing poli­cies and pro­ce­dures that ignore infor­ma­tion, i.e., prior and pos­te­rior (con­di­tional) prob­a­bil­i­ties that some­one fits the well-​defined pro­file of a terrorist.

How­ever, other than crit­i­ciz­ing his unwill­ing­ness to “pro­file,” we don’t blame Pres­i­dent Obama for the fail­ure on Christ­mas, and we think that it is silly for oth­ers to blame him.

We do think that his pref­er­ences and mind­set for large, cen­tral­ized, mech­a­nisms – e.g., nation­al­ized health-​care, bail-​outs, etc – are sim­i­lar to the prob­lem we dis­cuss below, but in all like­li­hood, the sys­tem pre­dates his tenure.²

So, despite the sys­tem hand­i­capped by the unwill­ing­ness to pro­file, if the intel­li­gence fail­ure was not Pres­i­dent Obama’s fault (and not for­mer Pres­i­dent Bush’s fault) and it is not the fault of those man­ning the sys­tems, than who or what is to blame? We sug­gest that the reader con­sider a poorly-​designed, overly-​rigid database/​information system.

Too Rigid

By def­i­n­i­tion, in an overly-​rigid infor­ma­tion sys­tem, both the input and out­put func­tions may be less flex­i­ble and user-​friendly than required. Given the fed­eral government’s pen­chant for large, cen­tral­ized, stan­dard­ized solu­tions, it is easy for us to believe that such an infor­ma­tion sys­tem (or sys­tems) has (have) been employed in the war against ter­ror­ism and that such sys­tems increase the like­li­hood of “intel­li­gence fail­ures” and ter­ror­ists evad­ing detection.

Rigid Input: Round Holes, Square Pegs and Worse

Con­sider the idiom of “putting a square peg in a round hole.” For data­bases that means that cer­tain facts that should be recorded may not be eas­ily cat­e­go­rized into avail­able fields because proper, descrip­tive fields do not exist (and can­not be eas­ily added). For exam­ple, con­sider cen­sus or EEOC forms where there is no appro­pri­ate box to check: where it is required to select a sin­gle “nation­al­ity” or “race” when you are ¹⁄₁₆ of this and ¹⁄₈ of that, et. al.

If such metaphor­i­cal “square pegs” could con­sis­tently be jammed into “round holes,” there would not be an issue because users would likely have devel­oped heuris­tics (rules-​of-​thumb) to cre­ate well-​formed sub­sti­tu­tions and work-​arounds. In all like­li­hood, those rules or map­pings would not be for­mal­ized in any offi­cial man­ual or doc­u­men­ta­tion, but they would be well-​known and trans­mit­ted dur­ing both for­mal and infor­mal train­ing sessions.

Unfor­tu­nately, real-​life is often not so sim­ple, because the so-​called “square pegs” may not be of, say, uni­form size, color, and shape.

In fact, other than cer­tain fields like names and addresses, we sus­pect that many of the facts that should be recorded can’t be eas­ily or suc­cinctly described in a word or two – that they are more nuanced and qual­i­ta­tive and grad­u­ated and require length­ier, usu­ally sub­jec­tive descrip­tions. Actu­ally, they may not be very dif­fer­ent than blog posts, and we would hope that writ­ers and recorders of those posts would have the flex­i­bil­ity to cre­ate new fields and cat­e­gories on-​the-​fly – like we do every time we add a new tag or category.

Unfor­tu­nately, we sus­pect that leads to many “cod­ing” errors and incon­sis­ten­cies and extremely long descrip­tions of fields (to pre­vent such “errors”.) We also sus­pect that it leads to too much over­sight; many lay­ers of approval by supe­ri­ors (and there­fore much edit­ing and chang­ing); and overly-​restrictive input poli­cies, e.g., “he doesn’t have the per­mis­sion or author­ity to write that.”

More­over, we also sus­pect that these prob­lems are exac­er­bated when inves­ti­ga­tors and field agents aren’t involved in the infor­ma­tion sys­tem design process.

Rigid Out­put

Other prob­lems with rigid, poorly-​designed sys­tems include (1) not pro­vid­ing use­ful, stan­dard­ized out­put or (2) not hav­ing the capac­ity for users to eas­ily search and access stored data for ad hoc queries.

Note again that we have no knowl­edge of actual, rou­tine TSA, FBI, CIA, and Home­land Secu­rity reports, and if we did, we prob­a­bly couldn’t write anything.

1. Too Cen­tral­ized and Uniform

That being said, we could imag­ine that there are dif­fer­ent lev­els of secu­rity clear­ance, and that access to the data could be overly-​restricted based upon those clear­ances. In par­tic­u­lar, we could imag­ine that unver­i­fied and unsub­stan­ti­ated reports are among the least generally-​accessible data – until they are ver­i­fied, reviewed or accepted by the bureau­cracy, regard­less of whether that involves a sin­gle agency or an over-​seeing umbrella group.

BUT those unsub­stan­ti­ated reports are the ones that are most likely to pro­vide infor­ma­tion about new ter­ror­ists like Abdul­mu­tal­lab, (and that is the prob­lem with treat­ing for­eign­ers who are threats to our national secu­rity as crim­i­nals rather than enemy com­bat­ants.) If our hunch is cor­rect, then one should expect future “intel­li­gence fail­ures” to arise in sim­i­lar situations.

More­over, if our hunch is cor­rect, then a cen­tral­ized, data­base administrator’s (rather arbi­trary) rules – or worse, some lawyer’s rules – sub­sti­tute for the indi­vid­ual knowl­edge and dis­cre­tion of var­i­ous field agents and super­vi­sors.³ As such, fields agents may not have the oppor­tu­nity to syn­the­size the infor­ma­tion until it is too late. (It’s a case of the per­fect being the enemy of the good.)

Unfor­tu­nately, that prob­lem is exac­er­bated once those rules and poli­cies are set. Later admin­is­tra­tors may be unwill­ing to “rock the boat” and ini­ti­ate worth­while changes because there is a chance of being blamed for sub­se­quent fail­ures but lit­tle chance of being rewarded for success. (Those acco­lades would most likely go to the “eagle-​eyed” agent who noticed some­thing was wrong.) By the way, as we often argue, it is dif­fi­cult to cat­e­go­rize such a choice – not to act – as irre­spon­si­ble behav­ior, espe­cially when it is induced by poorly-​designed poli­cies and a lack of man­age­r­ial dis­ci­pline. That’s why it is a bureau­cracy, after all.

So, rigid poli­cies self-​perpetuate and infor­ma­tion, hunches, and rumors are not passed along.

2. Search­able? We Doubt It.

As we have repeat­edly men­tioned, much of this post is mere spec­u­la­tion. A few of our con­jec­tures are pro­jec­tions based upon our own expe­ri­ences. Given that, we could imag­ine that inves­ti­ga­tors, ana­lysts, and agents can­not query or search the entire data­base (if it exists in one place).

Most likely, they receive exported sub­sets of the data, and those sub­set do not arrive imme­di­ately upon request. (The deci­sion to grant the request is prob­a­bly made by a data­base man­ager or admin­is­tra­tor and may require detailed spec­i­fi­ca­tions and pos­si­bly mul­ti­ple approvals – a whole process. Again, that’s why it is a bureaucracy.)

Now, we’re not sure of the ben­e­fits of such a bureau­cracy and sus­pect that such processes con­tinue to exist because “that’s how we’ve always done it,” which could be trans­lated as “we don’t know any better.”

Regard­less, there are costs to such pro­ce­dures. Besides the pos­si­ble lack of time­li­ness, there is a reduced oppor­tu­nity of dis­cov­er­ing any­thing – pat­terns, what not – acci­den­tally or serendip­i­tously. When a sub­set or export is requested and jus­ti­fied it must be com­pletely spec­i­fied; so, the requester needs to know exactly what he or she plans to inves­ti­gate before com­plet­ing a request and there is lit­tle chance of expand­ing or redi­rect­ing the inves­ti­ga­tion with­out re-​submitting requests for addi­tional fields.

In addi­tion, if the entire data­base is not fully-​searchable, then inves­ti­ga­tors are less likely to find matches and pat­terns across fields. Recall our crit­i­cism above: with rigid input fields, and vary­ing “square pegs,” agents in dif­fer­ent loca­tions and depart­ments may input sim­i­lar facts in dif­fer­ent fields. If some of those fields are not avail­able and search­able, then inves­ti­ga­tors will get fewer hits and matches and that will reduce the chance of mak­ing con­nec­tions and discoveries.

Our Rec­om­men­da­tion

So. the dili­gent reader, who has made it to this point, may ask: if your hypothe­ses and spec­u­la­tions are cor­rect, then what’s your solu­tion? (Alter­na­tively, they may note that the sell­ers of ham­mers tend to see a lot of nails.)

We reply with a rhetor­i­cal ques­tion: why can’t such sys­tems or con­glom­er­a­tions of sys­tems be more like the web and blo­gos­phere? By that we mean why can’t they be unfet­tered, completely-​searchable, accept respon­si­ble com­ments and ques­tions, and even per­mit writ­ers with vary­ing degrees of cred­i­bil­ity to post entries. (If the gov­ern­ment already has such a sys­tem, then kudos to it.)

Why not decen­tral­ize the process and empower secu­rity inves­ti­ga­tors, ana­lysts, and agents to use their idio­syn­cratic beliefs, opin­ions, infor­ma­tion, expe­ri­ences, posi­tions, and knowl­edge to iden­tify prob­lems and to adapt the data­base as threats and knowl­edge change?⁴

We imag­ine a mini-​version of the inter­net (with the abil­ity to search the entire inter­net, too), where indi­vid­ual agen­cies pub­lish blogs and news reports for them­selves and other agen­cies. (Geez, they could even sign-​up for each oth­ers’ feeds.

Of course, such a sys­tem would need to be at least as secure as on-​line bank­ing, but more pri­vate, but all such sys­tems must be.

Note, also, that noth­ing pre­cludes the run­ning or har­vest­ing of rou­tine reports from such sites. That’s what search engines and their bots and a host of sites already do. They stan­dard­ize the out­put of many dis­parate sys­tems. In fact, our rec­om­men­da­tion does not require any new or advanced tech­nol­ogy – just the appli­ca­tion of exist­ing plat­forms that are freely and read­ily avail­able to any­one with a few bucks and an inter­net connection.

Granted, it’s on a much larger scale than our blog, but it need not be expen­sive.⁵ More­over, we sus­pect that access to exist­ing sys­tems could be incor­po­rated eas­ier via web apps than through cus­tom pro­gram­ming for­ays that attempt to merge or con­sol­i­date exist­ing data­bases. For exam­ple, every Google query searches mil­lions of MySQL and MSSQL data­bases all with slightly dif­fer­ent struc­tures and fields.

Maybe we’re wrong, maybe we’re right. How­ever, even if our diag­no­sis is cor­rect, we doubt that the gov­ern­ment would act on our rec­om­men­da­tion. It would most likely try a cen­tral­ized “fix” of the iden­ti­fied prob­lems or would try a pilot-​program that (due to its lim­ited nature) would be des­tined to fail. In that case, hop­ing for con­tin­ued good luck might be the most rea­son­able and viable strategy.

In clos­ing, note that we are not dis­parag­ing the efforts of our fel­low cit­i­zens or the nation’s allies in their defense of our coun­try and way of life. Instead, if our spec­u­la­tions are cor­rect (or nearly so) we are rec­om­mend­ing a change in strat­egy and tac­tics so that their earnest effort yields more pro­duc­tive results.

As usual with long posts, we’ll likely make cor­rec­tions and edits that clar­ify our prose dur­ing the next few days.

Copy­right © 2010 Spero Consulting.

---

Foot­notes: