Uncertainty Management
Or, Ignoramus et ignorabimus
Or How Trading is Like Playing in a Culvert on a Hot, Sunny, Summer Day
This is the first in a series of essays on risk management, but we don’t like that phrase, particularly that “risk” word. So, we’ll try to convince those readers with an interest in the field that they shouldn’t like it either. We hope to persuade based upon reason, rather than just our natural charm, by emphasizing the broader concept of uncertainty over the narrower concept of risk. For completeness, we’ll do this in a couple of stages and show the dangers of focusing on risk rather than uncertainty. Our main criticism: whether out of naiveté or cynicism, many parties ignore the difference and treat all uncertainty as simple, estimable risk. We think this has to do with the poor training provided by universities (in the following sense). The sterile, artificial research environment permits idle and inconsequential speculation by the faculty. Now, we’re all for idle speculation as research, but not for teaching because the same approach — particularly the same level of abstraction and elegance — applied in the real world can be costly — even deadly.
Update: readers interested in a short description of the difference between risk and uncertainty should see this aptly-named post: The Difference Between Risk and Uncertainty. Readers interested in our view of the narrowness of many risk management departments should see: Learning the Difference Between Risk and Uncertainty, or not. Of course, we spent the time to write this longer essay; so, we think that it is worthwhile, too.
Introduction
We hope to convince the reader of the similarities between certain financial activities — like investing and trading — and playing in a culvert on a hot, summer day. Many readers with an interest in financial activities may have heard of another analogy that evokes a more immediate sense of danger: picking up nickels (or dimes) in front of a steam-roller. However, in the most important ways, the two analogies are quite different because while the danger of standing in front of a steam-roller is immediate, it is rather certain and estimable and can (usually) easily be avoided. We are interested in those dangers that are less visible and less certain and less measurable. To differentiate between these notions (and degrees of ignorance) we must first consider the nature of uncertainty.
Uncertainty
There are various notions of uncertainty, and we are interested in a couple of them. We call the first type scientific uncertainty, and define it as the lack of assuredness about something in the future — an outcome — which at some point in time we might know with a degree of surety. This is the same type that many scientists face when conducting experiments or analyzing theories of natural or social phenomena. Such scientific investigation usually involves modeling or abstracting away from reality to focus on understanding a few relationships better. When this behavior is mimicked by traders or investment or “risk” managers, it seems to permit the pretense that their approaches and methods are “scientific” and therefore not arbitrary. We argue that such an approach is problematic when there are real, non-academic implications at stake because in the real world, more than just modeled variables can hurt.
We think these scientific pretensions develop partially because scientific uncertainty is usually the only kind discussed in many academic settings, particularly in probability and statistics courses (and “science” courses, too). Introductory statistics courses usually attempt to provide “tools” for other areas of investigation while more advanced courses tend to cover the mathematical foundations underlying these tools, but usually ignore any philosophical foundation. Thus, many college-educated risk managers, analysts, investors, and traders have limited exposure to notions of uncertainty that can’t be easily quantified. So, if they do what was taught in school, then it can’t be their fault, can it?
Moreover, it seems that still fewer appreciate that what they have learned in these classes, while detailed and sometimes complicated or mathematically challenging, are actually very stylized and idealized notions of uncertainty with few real-world analogues — outside of games of chance and perhaps a few, rare cases. To us, this similar to the old saw that if one only possesses a hammer, lots of things look like nails; so, these techniques and methods are applied with less consideration than necessary to understand the potential costs of misapplication or misspecification.
Reasonable Uncertainty
Besides scientific uncertainty, there is also uncertainty about what we are uncertain about, i.e., there is a lack of assuredness about whether we possess a clear and complete understanding of a phenomenon or phenomena. We call this second-order uncertainty reasonable uncertainty and do so for two reasons. First, it is reasonable to believe that we don’t know everything, and possessing that knowledge should change our behavior. In this way, our outlook is similar to the Hippocratic Oath of “first, do no harm.” Second, it is through reason (and maybe luck, too) that potential and unknown influences can be found or considered or possibly immunized (even without their discovery). This (reasonable) realization that one does not know everything that can happen to one’s self, one’s assets, or one’s trades is at the very heart of reasonable uncertainty. For us this realization does not lead to pessimism that investigation and analysis is futile because nothing can be known for certain — far from it. Rather, such awareness permits a broader perspective and shows the need for imagination and conjecture when attempting to mitigate potential future losses.
Because reasonable uncertainty is can’t be measured, it seems that it is often ignored, and that is unfortunate because ignoring such uncertainty and denying the existence of the unknown is foolish and is often dangerous, especially when mitigation tactics are expensive to employ.
Reasonable uncertainty complements scientific uncertainty and is closely related to epistemology and dates back to at least the ancient Greek philosophers Plato and Socrates. Where epistemology deals with knowing what we know (and how we know it), reasonably uncertainty deals with knowing that there are things that we don’t know about, and knowing that it is unlikely for one to know anything with absolute certainty. For typical “risk management” considerations, we can think of this as the uncertainty about whether (1) all possible outcomes can be specified, or (2) if they can, whether their true likelihoods (and all of the potential factors that can affect them) can be known. We personally attribute this lack of knowledge in about outcomes and likelihoods in social settings — like trading — to man’s freewill.
Recall former Secretary of Defense Donald Rumsfeld’s ideas of the knowable and the unknowable that became well-known from his press conferences after the start of war in Iraq. His famous quote is: “There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don’t know. But there are also unknown unknowns. There are things we do not know we don’t know.” (Italics added.) While the third and fourth sentences of the quote deal with scientific uncertainty, the last two italicized sentences refer to reasonable uncertainty — as in a reasonable and reasoning adult should expect that he or she has not discovered every possible pitfall — that he knows that he is ignorant about the level of his ignorance.
Almost 2,000 years before Mr. Rumsfeld, our notion of reasonable uncertainty was beautifully expressed by St. James (the Lesser) in a chapter from his only epistle (4: 13 – 17):
Come now, you who say, “Today or tomorrow we shall go into such and such a town, spend a year there doing business, and make a profit
–you have no idea what your life will be like tomorrow. You are a puff of smoke that appears briefly and then disappears.
Instead you should say, “If the Lord wills it, we shall live to do this or that.”
But now you are boasting in your arrogance. All such boasting is evil.
So for one who knows the right thing to do and does not do it, it is a sin.
We included the final two verses for a couple of reasons: first, to show that the third verse isn’t about predestination or fate, and second, because the sentiments they express are so close to our own heart in their criticism of hubris and thoughtlessness — our own excluded, of course — particularly when other people’s money is involved.
The empathetic reader should find it quite easy to imagine that St. James was reflecting on his own life when he wrote, “you have no idea what your life will be like tomorrow.” Remember, this is a man who was recruited to be a disciple by Jesus and witnessed several miracles prior to his own martyrdom.
What Do Rummy and St. James Have To Do With Finance?
While it might surprise a few readers, unknown unknowns and unconsidered knowns do exist in trading and investment activities. Such unforeseen factors have been the cause of many large financial losses, especially when investors or traders or modelers or managers lack the requisite humility and knowledge — wisdom or reasonableness might be betters words — to thoughtfully execute their responsibilities, particularly when they protect against only known destructive events. In this way, they remind us of the French and their Maginot line prior to World War II, where their large artillery guns only pointed to the east.
Such events may also result in specification and modeling errors, which we dryly note may be more important to some than the financial losses of others. Consider various now-defunct firms, hedge funds, and traders with magic black boxes and program trades. They were always right, except when they were wrong, and who could blame them? Who could have thought that unconsidered or unknown bad things could happen? Yeah, who indeed? 1
The unconsidered and the unknown are two separate notions that at times may be difficult to distinguish: (1) errors of convenience occur when one ignores known factors that might not neatly fit into existing models, and (2) errors of ignorance which occur when one does not consider or reason the potential existence of unknown factors that might be harmful. Rumsfeld and St. James have the latter in mind — as do we in this section.
Because of the existence of both types of errors, we like George Box’s line, “…all models are wrong, but some are useful” as a guiding principle for developing and analyzing abstractions like financial or “risk” models. We also like using other tools like scenario analysis and stress testing — both the kind based upon developing an economic setting that informs combinations of market parameters, and the kind that stresses everything in the wrong way (against positions) despite the direction and magnitude of historical linear correlations.
So far, we’ve mentioned losses and discussed uncertainty without defining risk. Now, it is necessary to define it.
Risk
We follow Frank Knight’s 1921 definition of risk as “measurable uncertainty.” With that definition, the reader can see that risk is a subset of scientific uncertainty. Scientific uncertainty is broader and includes settings where certain characteristics may not be measurable, i.e., not expressible as a finite number, e.g., when the variance or the mean (and therefore the variance, too) are not well-defined. Such things can happen if their characteristic’s value gets either (1) infinitely big or (2) is indeterminate. For example, indeterminacy can occur when the mean or average of a probability distribution is the sum of positive infinity and negative infinity. Unfortunately the sum need not equal zero. Search for “Cauchy” or, more generally, for “stable” distributions to investigate these “abnormal” types of distributions.)
Note also that our notion of reasonable uncertainty is different than immeasurable scientific uncertainty. The former has to do with the existence or absence of a “correct” model as well as the analyst’s ability and luck to discover it (if it exists). The latter is concerned with cases where supposing a right model exists — say, a(n impossibly) perfect representation of reality — certain aspects of the random environment cannot be quantified because of the true distribution’s properties.
In sum, we categorize the unknown or uncertain future as:
We emphasize these differences between reasonable uncertainty, scientific uncertainty, and the much narrower notion of risk because their implications go beyond the merely theoretical or philosophical or pedantic. Ignorance of reasonable uncertainty can be deadly and the fat tails of distributions without means or variances can permit enormous losses at rates millions of times greater than their associated rates under normal or lognormal distributions.
For example, consider the analysis of a trading program or trading strategy with associated and proposed loss mitigation strategy, i.e., its hedging strategy. Like St. James, knowing that outside of contrived experiments and games, true distributions are almost never known, one can then assess the degree to which the viability and profitability of a proposed program or strategy depends upon its probabilistic assumptions, like the assumed parametric levels and its assumed distribution family and on possible events outside of the model.
So, if a plan’s proposed profitability is not robust enough to survive tweaks or changes to the assumed distribution function — either by varying parameter values in the same distribution or using a different family — then the analyst should have strong doubts about the program’s viability. (And the advertised “arbitrage” opportunity may not exist.) Outside of any such distributional sensitivity analyses, scenario analyses and stress-testing provide estimates of losses under various combinations of environmental (market) variables.
For another illustration, reconsider our diagram and note that the uncertain future may include reasonable uncertainty, immeasurable scientific uncertainty, and risk, and both types of scientific uncertainty may or may not include stationary and non-stationary processes. (Briefly and roughly, a process is something that generates uncertainty or randomness through time, and in a non-stationary process the nature of the uncertainty or the level of risk can change through time.) Even with completely measurable processes and values (risky processes), the parameters may change smoothly or discretely (unlike most steam-rollers). 2 What this means is that if we are surprised by an outcome it may be for several reasons: (1) we don’t understand the process, (2) we do understand, but can’t calculate it, or (3) we understand and can calculate it, but the levels of risk can change or jump without warning so the process can go from low risk to high risk very, very quickly. (Fortunately, financial and commodity markets don’t behave like that! Ha!)
Of course, surprises aren’t always easy to explain. For example, the harms suffered from well-understood, non-stationary processes may be the same ones suffered in poorly understood environments — as with reasonable uncertainty — or in well-defined but not fully measurable environments, as when a distribution has fat tails.
Culverts and Conduits
After the introduction and six pages of prose, the patient reader may wonder: “So, how exactly are financial activities like playing in a culvert on a bright, sunny, summer day?” We explain below.
As a very young boy, we recall hearing about two older boys who decided to play in a nearby culvert one hot, sunny, summer day. The water level of creek that flowed through the culvert was low and presumably the shade was inviting. The culvert was big; it was long, and high enough for grown-ups to walk through it — at least eight feet tall as we recall. (We doubt that the boys had ever wondered why such a large culvert was necessary.)
What the boys didn’t know was on that day it wasn’t sunny everywhere. In fact, upstream, less than ten miles away — there were heavy storms — heavy enough to cause a severe flash flood. When the wall of water reached the culvert, the boys had no hope of escape; sadly, they drowned; and their lifeless bodies were found downstream later that afternoon near the highway that leads to the city. Whether (1) the boys lacked the knowledge that events could turn disastrous or (2) whether they realized the full range of possible outcomes but could not calculate conditional expectations (because the expected value of such tail-events were infinitely expensive) or (3) whether they had a true and correct understanding of the non-stationary nature of the creek’s depth but couldn’t predict when the régime or values would change, the tragic consequences were the same.
Now some might argue that with more data, their deaths could have been avoided, but we ask: how many boys are going to stop playing in a culvert on a bright sunny day with no danger in sight? Analogously, we ask: how many CDO structurers will stop filling their conduits despite the metaphorical sound of distant thunder. From what we have seen from the past two 18 months, the answer seems to be “not many.”
We argue that unless the analyst is careful and thoughtful, he or she is unlikely to realize the limits of his or her knowledge, particularly when all effort is focused on some immediate task, like investigating, say, a historical time series of price observations (or tadpoles and fish in a June stream). Moreover, consideration of our limits of knowledge, measurement, and foresight should provide thoughtful analysts with a better perspective from which to analyze proposed strategies and tactics, including hedging strategies that depend upon idealized probabilities, idealized costs, and an idealized — and possibly infinite — number of trades and adjustments. Our guess is that such analysis would call into doubt the efficiency of many proprietary, i.e., non-customer driven, trading activities and many hedging strategies (just as using a true, risk-adjusted cost of capital would negate any positive net present value based generated from dividing by a lower rate like LIBOR. Please see our archived blog posts on nedges and sledges.
Uncertainty Management
Given what we consider to be the necessity to recognize these differences between the more general (and more common) reasonable uncertainty and narrower, idealized, measurable risk, we prefer the broader phrase “uncertainty management” to “risk management.” It is a way to constantly remind interested parties and ourselves of the possibly unknowable and somewhat immeasurable nature of the environment and world we face.
From our small philosophical foundation we can now discuss preliminary concepts one is likely to encounter in risk management. So long as we recall the limits of these tools and our knowledge.
Preliminary Concepts
In this section, we discuss a few basic notions with which many readers are familiar. We’ll build upon these ideas in subsequent essays, and but we’ll get philosophical again at the end of this one.
Random Variables
When one talks about risk one usually talks about random variables, which technically aren’t variables at all; they are mathematical functions. Random variables assign unique numbers to things, and that is what math functions do. (Yeah, we know “things” is not a technical term.) However, for our purposes it is okay to think of a random variable less formally as a number that we don’t know, yet.
To say more about the realization of a particular number, including the likelihood of it or other possible numbers arising, we need to introduce probability distribution and density functions, which we’ve already alluded to several times. These functions will also help describe other properties of interest.
A random variable has to be able to take at least two possible values (with some positive probability for each); otherwise, it is not random at all. Some variables may take only a finite number of values, while others may take uncountably-many values: like all values between negative and positive infinity (-¥, +¥); all non-negative values, [0, +¥); or all values between [0, 1]. These latter three ranges are used quite frequently in financial risk management; prices or interest rates are assumed to be non-negative and continuous, and simulated values are often found by assuming that probabilities themselves are random variables between zero and one like in this Excel spreadsheet.
We will define terms under the assumption that our functions are continuous (and differentiable), which we don’t define here. Neither discrete random variables, like the six possible values of a die, nor piecewise continuous ones, like jump-diffusion processes, are defined.
Probability Distribution & Density Functions (Continuous)
Ignoring technicalities, we can think of the range of possible values, X, for a random variable, x, as the domain (or inputs) of its probability distribution function (PDF). A continuous probability distribution function shows the cumulative probabilities as they accumulate from the smallest possible value, say xℓ, which could be -¥, to the value of interest, say, xo. For emphasis we sometimes write “cumulative probability distribution function,” but given the following definition, the cumulative is superfluous.
For F to be a probability distribution function it can’t have a value less than zero or greater than one, and it must be non-decreasing across the range of possible values. That means that as the interval gets bigger, the probability of realizing a number in that interval can’t get smaller. So, using symbols this means, F(x) Î [0, 1] for every possible x, and if xs ≤ xb, then 0 ≤ F(xs) ≤ F(xb) ≤ 1. Here is an illustration of a distribution function for a random variable defined on [0, +¥) but truncated at 100 in the graph.
To find the probability of any number being realized between two points, say, xs ≤ xb, we subtract the difference between F(xb) and F(xs):
It is worth noting that with a continuous probability distribution function, the probability of any single number being realized is zero. Only intervals of numbers can have positive probability. For a single, particular number to have a positive probability the distribution must be discrete or be a mixture of continuous and discrete characteristics and we’re avoiding those types in this essay.
We illustrate the probability of an interval by continuing our example for xs = 35 and xb = 45.
The area under the density function between any two points then represents the probability that the random variable will fall into that interval. So, we can rewrite the expression above:
as we show in the following graph that area is equal to 26.35% as in Graph 2.
There are a few very common density functions in financial risk management, including the normal density (the bell curve) and the log-normal density. Interested parties may look for a separate essay on those functions to be posted in the near future.
But, Which Function Is the Best for Risk Management?
There are other functions to choose from when modeling financial risk besides the normal and log-normal densities; however, there is usually no single correct choice. However, there are usually many wrong choices.
Except for games of chance or other contrived experiments, a perfect model almost never exists. Restated, we know of no phenomenon in the financial markets, other than investing in lottery tickets, where there is a single correct choice of a particular distribution function. Restated again, we know of no choice of distribution functions where the choice is not an assumption regardless of the existence or abundance of (historical) empirical evidence, i.e., whether cloaked in the scientific method or not.
Before continuing with this line of reasoning, note that this does not mean that we recommend doing nothing or that nothing can be known — that philosophy, scientific, or financial investigation and research are worthless. Rather we argue for the investigator or analyst to retain a degree of humility and understand or internalize that their abstractions ignore potentially vital considerations of the future that neither they nor anyone else may possess. We recommend using simple, well-understood models and the near continuous communication of their flaws and missing components. Our recommendation is based upon behavioral considerations and for both efficiency and effectiveness reasons.
Abstractions (or models) are almost always necessary to understand anything, but that does not mean that by understanding a model one has captured and understands all salient features of reality. For example, paper maps are models of terrain and roadways and show the shortest route between two places but rarely show the amount of traffic or the probability of being robbed or assaulted along with the street names. So, supplementing the model with other knowledge of the environment is essential to survival and quality of life. Just as one is unlikely to buy a home in a neighborhood based upon a graphic artist’s or mapmaker’s color choice for that neighborhood, one should be equally suspect of a “quant’s” analysis if the person has no relevant economic experience or intuition to justify his or her implicit assumptions.
The Problem of Induction: The use of a parameter or distribution estimate based upon a historical, empirical analysis for the analysis of an unknown, future random variable like a market price, requires several assumptions. The most important one is that the past has informed the analyst about every prospective possible outcome and each outcome’s correct likelihood.
We claim that regardless of the task, one almost always faces reasonable uncertainty, especially in social settings like markets where individuals exhibit free will. To the question, “can one ever really infer the way that natural phenomena or complicated societal interactions generate outcomes or random variables?” we reply we think not, and that question gets to the very heart of empiricism (and science for that matter) and risk management, particularly the consideration that no number of historical observations guarantees the future.
This is the problem of induction, and it dates back to Aristotle through David Hume and Karl Popper. In recent years, Nassim Nicholas Taleb has re-popularized the notion through his criticisms in Fooled by Randomness and The Black Swan. In fact, the title of the latter book comes from the fact that for centuries Europeans thought that all swans were white because all they saw were white swans. As he writes, they were so convinced of the veracity of this statement that the phrase “black swan” became euphemism for the oxymoronic like “the funny, late-night, talk-show monologue.” Needless to say, black swans were subsequently discovered in Australia and now are relatively common (so much so that a few live down the hill from us). Thus, the received wisdom from hundreds, if not thousands, of years of empirical observations and experiences was wiped out with a single observation because each mistaken party took the absence of evidence to be evidence of absence.
For such reasons, commentators like Taleb consider such assumptions of a future like the past to be foolish. In fact, some commentators abjure the use of relatively simple distribution functions with measurable parameters and moments (e.g., means and variances, etc.). No normal or lognormal distributions for them! They argue that especially in financial markets, the uses of such functions only mislead and that the expected costs associated with such misconceptions are too high.
We are less dogmatic and believe that simple statistical examples and models can be quite appropriate for analysis and communication (1) depending upon the context and (2) contingent upon one retaining a certain degree of humility and skepticism, which are too often lacking. In fact, we prefer simpler models to more complex, Rube Goldbergian contraptions that provide no additional explanatory power of future events, but often provide false confidence that either naively or cynically is attributed to “thoroughness” or “hard-work” or “complication” rather than thoughtfulness. In such situations, less-informed observers like senior managers may ask, “why would anyone spend so much time working on something if they didn’t think it was correct?” We reply: that’s not evidence correctness anymore than the hours and lives spent by alchemists provide evidence that lead can be turned into gold.
In subsequent essays, we will define common risk management terms and illustrate basic concepts, but always within our framework of uncertainty, rather than the narrower risk.
As always, let us know if you think that we are right or wrong or if you agree or disagree with our comments. We like to believe that we are reasonable and thus holdout a slim probability that we are mistaken.
Copyright © 2008 Spero Consulting.
Footnotes: