by john siegfried and
john siegfriedis a professor emeritus at Vanderbilt University who specializes in economics education. david colander is a professor of economics at Middlebury College and the author of an introductory economics textbook, Economics. This article adapts portions of the authors’ essay in The Journal of Economic Education. Used by permission of Taylor & Francis Group.
Published October 31, 2022
Critical thinking is far too often conceived of as a vague, feel-good platitude. Every discipline claims critical thinking — CT for short – as a central part of its methodology. A look at academic analyses of the concept suggests that the essence is generally agreed upon. It involves approaching questions with an open mind and basing conclusions on deductive reasoning and evidence, while recognizing our limitations in terms of objectivity. Critical thinkers possess the virtues of intellectual integrity, humility, civility and a sense of justice. Who can argue with that?
Unfortunately, as soon as one moves from generalities to specifics, the substance of CT becomes more elusive, and one person’s CT is another person’s systemically biased thinking. For example, is an economist who points out the unintended, sometimes negative, consequences of an anti-discrimination policy exhibiting CT, or are they demonstrating a racist bias that is embedded in economic methodology? Our “CT” answer is that they are doing both at once.
Open-minded critical thinkers need to be comfortable with contradictions, and, as Walt Whitman famously put it in another context, “to contain multitudes.”
Inside-the-Box vs. Outside-the-Box
To capture these multitudes, we find it useful to distinguish “inside-the-box” from “outside- the-box” critical thinking. The inside the-box sort is what most economists have in mind when they refer to “thinking like an economist.” It is the thought process of an expert in their field. For an economist, that means using economists’ standard state-ofthe- art tools, models and methods.
Much of economists’ CT involves tools common to other STEM-oriented disciplines — logical deduction, induction, statistical modeling and testing. But economists also favor some particular aspects of CT. For example, they are inclined to look skeptically for the missing piece, such as when demand alone is identified as a cause of a price increase or decrease, or when benefits alone are identified as a reason to adopt a new public policy.
Economists also place emphasis on identifying potential unintended consequences of a policy. And they tend to think “on the margin,” focusing on changes moving forward. Indeed, they are trained to ignore costs that have been irretrievably incurred, or “sunk.” Above all, economists search for incentives, both positive and negative, that could stimulate behavioral reactions to environmental changes. Incentives may be financial. But many do not involve money, and economists’ version of CT involves a serious search for them.
When conducting empirical analysis, economists are particularly sensitive to the distinction between correlation (X happens when Y happens) and causation (X happens because Y happens). And in recent decades, they have developed numerous statistical methods to distinguish between them.
“Selection bias” is an important example. Selection bias occurs when individuals or groups in a study differ systematically from the population of interest, leading to a systematic error in statistical analysis.
“Selection bias” is one important example that has attracted a lot of attention among economists in recent decades. Selection bias occurs when individuals or groups in a study differ systematically from the population of interest, leading to a systematic error in statistical analysis. It often occurs when individuals self-select into either a treatment or control group for purposes of comparison.
For example, in testing a newly developed vaccine for safety and efficacy, if test subjects are allowed to decide for themselves whether to participate in the treatment or control group, those who are really motivated to use the therapy — say, the elderly or those with chronic health problems — may disproportionately opt for the treatment group. As a result, the treatment will seem less effective than it really is because those receiving it are more likely to get sick in the first place. Accordingly, economists have worked hard to ensure that participants in controlled experiments are randomly assigned to treatment and control groups.
Formal empirical analysis has become increasingly important in economics as data availability has expanded and new techniques of analysis have been developed. Those developments have made clever variations of the experimental method routinely used in the hard sciences far more central to critically thinking like an economist. Today, sophisticated economic analysis is deeply grounded in the worldview of the great British physicist Lord Kelvin, who opined that “when you can measure what you are speaking about, and express it in numbers, you know something about it, but … when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.”
Whereas inside-the-box CT deals with analysis in an information-rich environment, outside- the-box CT deals with analysis when information is scarce. Here, formal empirical analysis cannot provide definitive answers, and policy analysis must necessarily involve subjective judgement. One must rely on what we call philosophical methodologies, including careful informal analysis of a question from many perspectives. The goal is to arrive at what might be considered a defensible conclusion that can be used until data are available — not a scientific truth.
Outside-the-box CT is highly skeptical of all allegedly definitive analysis, including the best current inside-the-box analysis. When one’s models’ links to the real world are tenuous, any conclusions drawn from them are equally tenuous. So, a critically thinking outside- the-box economist would add to Lord Kelvin’s statement: unless your measurement truly corresponds to the concept being discussed, your “precise scientific knowledge” might be not only meager, it might be dead wrong.
Outside-the-box CT is grounded in the methodology of the humanities rather than that of science. It offers no scientifically agreed-upon test of what should be accepted as true, but instead relies on reflection and instinct. While this approach to critical thought was very much a part of 18th-century “classical” economics — John Stuart Mill’s work is a good example — it is alien to many economists today. It requires everything to be challenged and open for discussion. Thus, it recognizes the potential fallibility of all experts. Economists’ version of outside-the-box CT explores differences in what economists say they do and what they really do.
All sciences have powerful inside-the-box models and methods at hand. But they, too, have limitations — and all sciences encourage experts to be aware of them. It is a balancing act. Inside-the-box CT is what separates an expert from an armchair ruminator. Outsidethe- box critical thought is what keeps experts honest, either through self-criticism or reminders by qualified critics. To put it another way, outside-the-box CT can be seen as playing the role that inspectors-general play in keeping public agencies honest.
Outside-the-box CT encourages experts to avoid claiming more expertise than they actually have. It emphasizes that the standard economic model is a model, not the model, that economists’ method is just a method, not the method — and that many conclusions supported by deductive models involve assumptions that do not match reality and thus may not be relevant to solving real-world problems. It reminds experts that there is no getting around subjectivity in analyzing policy, and that science alone cannot be a definitive guide to action.
Understanding the value of this “neo-classical” model requires more than an Adam Smith necktie and a mantra about the virtues of free markets. It demands a deep knowledge of tools and models that many non-economist critics of economics lack.
Economists’ Failure To Convey Outside-the-Box CT
In economics, inside-the-box CT has been embodied in models that amount to modern distillations of the thinking of the great 18thcentury economists. These models contain important inside-the-box insights, including that people make choices involving tradeoffs between perceived costs and benefits, and that (under certain assumptions) prices in competitive markets create incentives that align private interests with the interests of society.
But understanding the value of this “neoclassical” model requires more than an Adam Smith necktie and a mantra about the virtues of free markets. It demands a deep knowledge of tools and models that many non-economist critics of economics lack. That means criticisms of neoclassical economics are often dismissed by even those economists who agree about the limitations of their field. Modern economics outgrew traditional neoclassical economics a half-century ago, and criticism of contemporary economic thinking doesn’t acknowledge that change.
Economists are much more aware of the limitations of economic models than most non-economists understand. And in large part that misunderstanding follows from the reality that introductory courses generally focus on the bare bones of inside-the-box CT. This makes sense: in economics, as in any field, there is a natural progression in learning, starting from fundamentals and later incorporating insights that add to the power of the analysis.
Introductory economics classes should not, in our view, present formal outside-the-box CT. But we believe the sort of informal outside- the-box CT that economists are doing all the time could and should be better conveyed.
As an example, consider the way the concept of a demand curve is introduced. It is important to not ignore the reality that demand depends on the distribution of income and wealth, and that a shift in either will alter the curves. Failing to make this clear leaves students with the impression that the existing distributions of income and wealth are “appropriate,” without much thought as to whether all stakeholders agree.
Another example is the way textbooks generally assume that maximizing the welfare of society is the ultimate goal of economic policy, with little discussion of the enormous complexity that a specification of society’s goals entails. If introductory economics did more to show that thinking like an economist included such outside-the-box considerations, students would come away with a more nuanced — and more empathetic — view of economics and the importance of outsidethe- box CT.
Too often, intro courses fail their students in this regard. Consider Nobel Prize-winning economist Esther Duflo’s description of what she took away from her introductory class:
I thought of economics as an elaborate hoax (or at best a Panglossian illusion) aimed at justifying the world and keeping it exactly as it was; using simple mathematics to describe some very rudimentary version of it, and “proving” that any attempt to intervene against the smooth functions of the market would wreak havoc.
This view of economists as closed-minded automatons is shared by many outside critics of the discipline. But, truth be told, almost any criticism of the economics profession offered by outside critics is shopworn. The profession has long recognized there are difficulties with its models and methods, but has concluded that the benefits of using them generally outweigh the costs of starting from scratch.
The problem is that this recognition is often not transparent. For example, most economics courses introduce students to the idea of economic efficiency as a natural product of market outcomes with little consideration that it applies only to perfectly competitive markets in which distributional outcomes are not important. Little attention is paid to the numerous alternative methods to allocate resources that economists themselves often prefer to competitive markets for some goods and services. Consider, for example, choosing political leaders by voting, choosing the household dinner menu by command of the cook, assigning property rights to the Donbas region of Ukraine by physical violence or allocating concert tickets by who gets in line first.
Introducing non-economists to the debate within economics about the usefulness and limitations of economists’ models and methods encourages them to see that outside-thebox CT is not a substitute for inside-the-box CT, but a complement. Outside-the-box CT exists not because economics is in crisis, nor because the models taught in the introductory course are not the best way to understand some issues. It exists because there are limitations of even the best tools, and outside- the-box CT reminds researchers about those limitations.
Our point is that if a rational choice model is used with appropriate caveats, then whether the assumptions are always true is not an issue of debate. The sophisticated view of rational choice is that it has problems. It is used, despite those problems, because it is manageable and often yields definitive results.
Enlightened Inside-the-Box CT
How do the alternative types of CT fit into thinking like an economist? We believe the best description is “enlightened” inside-thebox CT. This translates as inside-the-box CT that has passed through an outside-the-box filter. It views the models as useful starting points, not as unchallengeable truths. The real debate about the usefulness of economic models concerns the nature of the filter.
For example, consider the rational choice model built around the assumption that individuals act to maximize their self-interest. The appropriate question is not “are people rational?” but whether the assumption of rationality advances useful analysis. Digging a little deeper, what if the rational choice model distributes resources in a fashion that market participants believe is not fair? People tend to respond to perceived fair treatment with fair treatment in return. But if they believe they are treated unfairly, they may retaliate even when such reactions do not maximize their material wealth.
Our point is that if a rational choice model is used with appropriate caveats kept in the back of one’s mind, then whether the assumptions are always true is not an issue of debate. The sophisticated view of rational choice is that it has problems. It is used, despite those problems, because it also has advantages—it is manageable and often yields definitive results. This makes it a useful tool that provides insight into how the economy works and how various policies might affect it.
Beginning students should be taught that economists are aware of the limitations of the standard models, and that they are working on expanding the toolbox to adapt to different assumptions. For example, researchers such as Marianne Bertrand of the University of Chicago are offering a richer study of discrimination, incorporating feedback effects into an analysis rather than assuming norms and tastes are fixed. Outside-the-box CT would emphasize to students that, as economics evolves, its models must change. Doing so would make students less likely to draw strong policy implications from the rudimentary models and more likely to see them as tools for thinking about problems.
Nobel Prize-winning economist Abhijit Banerjee of MIT conveyed the modern economist’s view of models when he wrote that
it may be more useful to build models with ingredients tailored to the particular context: specific types of behavioral assumptions that go beyond the “standard” model, specific assumptions about market failures, all based, as best as possible, on results from past research in similar settings. This is in effect what a lot of empirical researchers already do, but mostly on an ad hoc basis, with the result that we continue to default to the standard model whenever our results are not in direct conflict with it.
Economists are not doing a good job in conveying what thinking like an economist means. Too many people see it as involving only inside-the-box CT and are oblivious to outside-the-box CT. That leads them to associate economic thinking with a narrow class of policy approaches that amount to garbagein- garbage-out. Modern economics can no longer be described as neoclassical economics. It is not tied to any particular model, but rather to analysis that includes both insideand outside-the-box critical thought.