Mario Tama/Getty Images

Why Is It So Hard to Measure Poverty?

 

ed dolan is a senior fellow at the Washington-based Niskanen Center.

Published October 23, 2024

 

Measuring poverty is one of the most important and most controversial tasks in social policy. What makes it so hard? Just assembling the data is tough enough. To know whether someone is poor, you need to measure both their resources and their needs. Some of the data can be found in public sources, like tax returns. But those are incomplete since many poor people don’t file taxes or their filings are inaccurate.

Other data, like the prices consumers pay for goods and services, may be available for the entire population, but the poor sometimes end up paying higher prices than the better off. Some data, like patterns of spending on food, rent and health care, come from surveys, but surveys suffer from sampling error, incomplete answers and willful underreporting. It is thus hardly surprising that different ways of dealing with these issues lead to widely varying estimates of poverty rates.

To further complicate matters, we ask our chosen poverty measures to serve two different purposes. Like tests for Covid-19, we use poverty measures both for tracking and for treatment. Tracking is a matter of learning what share of people in the community are poor (or in the case of Covid-19, infected). Treatment means asking whether a specific person qualifies for a program such as Medicaid (or in the case of Covid-19, for a treatment such as the antiviral Paxlovid). A measure that works for one purpose may not be the best for another. In the case of Covid-19, we used PCR or antigen tests for eligibility for treatment, while wastewater testing proved more useful for tracking the spread through a community.

Finally, politics and ideology add to the difficulty of measuring poverty. Ideally, as Senator Marco Rubio wrote in a recent letter to the Census Bureau, a poverty measure would “signal a national consensus about the goals of our economy and system of government.” But, as the senator’s letter makes clear, there is no such consensus. Progressives and conservatives approach the issue of poverty with very different prior beliefs.

Priors on the progressive left include the belief that capitalism favors the rich, in the sense that for those near the bottom, bootstraps are an inadequate substitute for a helping hand. Even the center left, which sees a legitimate place in a liberal democracy for free markets, views the American version as systematically tilted against the interests of minorities, notably the unskilled and the less-educated. So it’s not surprising that confirmation bias – the inherent human tendency to focus on evidence that fits with prior beliefs – leads progressives to favor tracking measures that show higher poverty rates.

On the other side, conservative priors include the beliefs that talent and hard work are the keys to prosperity, and that a rising tide lifts all boats. The quest for confirmation leads conservatives to favor measures that show low and declining levels of poverty.

Dolan Ed Hard to Measure Poverty 2
Tim Boyle/Getty Images

Turning from tracking to treatment, we find another political bias. When we try to determine whether specific people qualify for public assistance, we inevitably make mistakes. Those come in two forms: a “false negative” occurs when we wrongly declare someone who truly needs assistance to be ineligible. A “false positive” occurs when we give assistance to someone who in reality is not needy, even though the numbers seem to fit the official definition.

On the whole, progressives tend to be more sensitive to false negatives – cases where someone falls through a gap in the social safety net. Conservatives tend to be more sensitive to false positives – people who are better off than they appear to be, along with those who would be capable of helping themselves if they tried harder.

As a result of these biases, conservatives like eligibility tests that require lots of information from applicants, careful review by authorities and frequent reauthorization. Progressives, in contrast, favor simple qualification and continuous enrollment once qualified.

How Many Poor? Three Very Different Answers

As a tracking tool, a poverty measure tells us how many people there are whose resources fall short of a threshold high enough to cover their basic needs. There are many such measures because there are many possible meanings for “needs,” “resources,” “threshold” – and even “people.” Three examples will show why estimates of the poverty rate vary so widely.

Washington’s official poverty measure (OPM) is the obvious place to start. The OPM dates back to Lyndon Johnson’s War on Poverty in the 1960s. Its definition of needs starts with the cost of a “thrifty food plan” from the U.S. Department of Agriculture – consumers’ cost of buying a healthy diet – which is multiplied by three to cover necessities other than food. The OPM forms the basis for a set of thresholds for families of various sizes, defining a family as a group related by kinship, marriage, adoption or other legal ties. Resources consist of before-tax regular cash income including wages, salaries, interest and retirement income; cash benefits from programs such as Temporary Assistance for Needy Families and Supplemental Security Income; and a few other items. Note that the OPM’s resource measure does not include benefits delivered through the tax system or the cash-equivalent value of in-kind benefits such as health care.

 
Today, nearly everyone views the OPM as functionally obsolete. Some see it as overstating the poverty rate, in that its measure of resources ignores some of the biggest benefit programs including the Supplemental Nutrition Assistance Program (aka food stamps) and the Earned Income Tax Credit, which mostly provides cash for the families of low-wage workers.
 

At the beginning of the War on Poverty, the OPM stood at about 20 percent of the population. Since then, it has seen annual adjustments to reflect changes in the cost of living (using the consumer price index). As economic growth raised average incomes and as Congress and the states increased cash benefits, the official poverty rate fell, reaching 12 percent by the early 1970s. Since then, aside from cyclical ups and downs, it has changed little.

Today, nearly everyone views the OPM as functionally obsolete. Some see it as overstating the poverty rate, in that its measure of resources ignores some of the biggest benefit programs including the Supplemental Nutrition Assistance Program (SNAP, aka food stamps) and the Earned Income Tax Credit, which mostly provides cash for the families of low-wage workers. Others see it as understating poverty in that three times the cost of food is no longer enough to meet minimal needs for shelter, health care, childcare, transportation and modern communication services. Almost no one sees the OPM as just right.

In a recent paper, economists Richard Burkhauser, Kevin Corinth, James Elwell and Jeff Larrimore provide an alternative measure that reflects their view that the OPM overstates poverty. The question they ask is what percentage of American households today lack the resources they need to reach the original three-times-food threshold. Simply put, was Lyndon Johnson’s War on Poverty, on its own terms, a success or failure?

To answer that question, Burkhauser et al. develop what they call an absolute full-income poverty measure (FPM). It is “absolute” in that the basic basket of goods on which the threshold is based never changes. Their basket of basic needs includes more items than food, but they calibrate it to make the FPM poverty rate for 1963 equal to the 20 percent OPM rate for that year. The “full income” part means that they adjust their measure of resources to include in-kind transfer programs, like school lunches and Head Start, and the effects of taxes and tax credits, which have grown greatly since the 1960s.

Finally, they use households rather than families as their basic unit of analysis. By household, they mean one person living alone or a group of people who live together and share resources, regardless of kinship or legal relationships. These changes alone are enough to cut the FPM poverty rate for 2019 to just 4 percent, far below the 2019 OPM’s 10.6 percent.

Dolan Ed Hard to Measure Poverty 3
Spencer Platt/Getty Images

Going further, Burkhauser et al. maintain, as do many other economists, that the widely used consumer price index overstates the true rate of inflation. Instead, they prefer the personal consumption expenditure index, which the Federal Reserve uses as a guide to monetary policy. Replacing the CPI with the PCE reduces the FPM poverty rate for 2019 to a mere 1.6 percent.

The third poverty indicator frequently considered is the Supplemental Poverty Measure, published by the Census Bureau since 2011. The SPM reflects the view that the OPM understates poverty, in that it fails to recognize the way minimum household needs change as average incomes grow. The SPM enlarges the OPM’s basic bundle of essential goods to include not only food, but also shelter, utilities, telephone and internet service. On the resource side, the SPM adds many cash and in-kind benefits (although not as many as the FPM). It further adjusts resources by deducting certain necessary expenses, such as childcare and out-of-pocket health care costs. Like the FPM, the SPM moves away from the family toward the household as its unit of analysis.

There is another big difference here: unlike the OPM and FPM, the SPM provides a relative rather than an absolute poverty standard. Instead of updating poverty thresholds only for changes in price levels, it further adjusts them to reflect growth in real median household income. In that regard, the SPM is closer to the poverty measures used by the European Union and most other high-income countries, which define “at risk of poverty” as income at or below a given percentage of the median.

The figure on the next page shows how these three measures of poverty track from the 1960s up to the onset of the Covid-19 pandemic.

Which of the three measures is best? None of them is “best” in any meaningful sense. Nor are any of the many other proposed measures, some of which track even higher or even lower than the three shown in the chart, unambiguously superior. Rather, each answers a different question about the distribution of income. Do we want to know how many people today lack the resources to meet even the narrow definition of needs that was accepted in the 1960s? Then the FPM is a good choice. Do we want to know how many people are below, but within shouting distance, of the rising standards of today’s middle class? Then the SPM gets the nod.

From Tracking to Treatment

For tracking purposes we do not need to limit ourselves to a single measure of poverty, but a new set of problems arise when we shift the focus to treatment. To determine whether someone makes the cut for a specific antipoverty program such as SNAP or EITC, we have to select one unique eligibility standard.

The official poverty measure is not a good choice, as an official federal directive first issued in 1969 makes clear. The directive notes that the OPM is only “a rough statistical measure” valued to count the number of people in poverty and was “not developed for administrative use.” Nonetheless, dozens of federal programs, including Medicaid, SNAP, school lunches and grants for college students, use the OPM to decide who qualifies for benefits and how much aid they get.

MR104 web dolan poverty v01

Because the OPM omits so many categories on both the need and resource sides of the ledger, it is open both to false positives and false negatives. Households whose needs it underestimates may fail to qualify for help even though they meet most peoples’ criteria for poverty. Others whose resources are in fact greater may sometimes end up with more aid than the neediest.

The issue here is not whether the government is spending too much or too little on aid to the poor. That is a separate question. The share of the government’s budget that goes to poverty programs could be reduced by cutting the threshold for eligibility to, say, 60 percent of the OPM, or increased by raising it to, say, 130 percent of the OPM, as is done in setting eligibility thresholds for some programs. But whether spending overall is generous or parsimonious, as long as the OPM remains the baseline, some of the wrong people will be excluded while some of the less needy will get more.

False positives and false negatives are only one category of problems with existing eligibility standards for social programs. Work disincentives are another. Some programs, such as Medicaid, are subject to “benefit cliffs” where earning more than a specified percentage of the OPM results in a sudden loss of benefits. Other programs reduce benefits by a percentage of each dollar earned.

For example, a family with children might lose 24 cents in SNAP benefits for each dollar earned – and also 21 cents in EITC benefits. If you include the 7.65 percent payroll tax deduction, the family gets to keep less than half of each added dollar earned even before considering employment-related expenses like commuting, childcare or work clothing that might justifiably be subtracted from the resource side.

The disincentives vary by income. The households that face the most severe work disincentives are the near-poor – those who have just made it past the OPM threshold and are struggling to achieve full self-sufficiency.

Marriage penalties are a further problem with the existing OPM-based system. They arise when, for example, a single mother with one child and a single father with one child receive more aid in total than the same couple would get if they married. Apart from the simple inequity, marriage penalties discourage the formation of the kind of stable, two-parent families that are associated with less poverty and less welfare dependence.

Dolan Ed Hard to Measure Poverty 4
Spencer Platt/Getty Images
Can We Do Better?

The OPM, with its high error rates, work disincentives and marriage penalties is nonetheless deeply embedded in the American system for treating poverty. Changing it would not be easy. Just adopting one of the alternative poverty measures discussed above, such as the FPM or SPM, would not be enough since they, too, are designed primarily for tracking. What we need is a poverty measure designed from the ground up for the administration of existing and future poverty programs. Here are six steps that together would move us toward a measure suitable for treating, not just tracking, poverty.

Maintain balance. To have any hope of making it through the political process, an eligibility standard would have to maintain a reasonable political balance. Conservatives like the American Enterprise Institute’s Kevin Corinth worry that anything like the SPM would lead to runaway growth of welfare spending. Progressives could hardly be expected to endorse something like the FPM, which would drastically reduce the number of people eligible for assistance. But whatever overall level of spending is finally decided on, it should be possible to get broad agreement on changes that increased the accuracy of poverty measurement. Even without huge cuts or expansions of total poverty spending, it ought to be possible to reduce both false positives and false negatives in determining which households qualify for aid.

Redefine basic needs. A recent report on updating the measurement of poverty from the National Academies of Sciences, Engineering and Medicine recommends a list of needs that include food, clothing, telephone, internet, housing based on fair market rents, basic health insurance and childcare. The NASEM report recommends calibrating costs based on a percentage of median expenditures. But at least conceptually, a new measure of needs could use an absolute standard with baseline costs averaged over a period leading up to the date of implementation. Such a basket of needs could then serve as the basis for defining specific program thresholds that would vary according to the size and composition of household units – and perhaps also by place of residence. (It’s cheaper to live in Little Rock than in Los Angeles.)

Redefine resources. The new measure should take a full-income approach to determining the resources available to a given household for covering basic needs. Those would include cash income plus non-cash benefits such as SNAP or vouchers that
subsidize rental housing. The NASEM report suggests that they should also include refundable tax credits such as the EITC, childcare subsidies, health insurance subsidies, rent subsidies and (for homeowners) imputed rental income – that is, the rental value of their house. Total resources could then be adjusted downward to reflect taxes paid, work expenses, out-of-pocket medical expenses, homeowner costs (if applicable) and child support payments.

Centralize resource data. Determining eligibility for a given household for any given program requires assembling data from many sources. To avoid placing an undue burden on applicants, it would make sense to rely as much as possible on data that are already being collected for administrative purposes, and to centralize reporting as much as possible. Ideally, all necessary information on cash income, public benefits, taxes paid, tax credits and the rest should be available in a central database, so that applicants would not have to assemble the numbers separately for each program.

Dolan Ed Hard to Measure Poverty 5
AP Photo/Rich Pedroncelli

The IRS could provide data on cash income other than public benefits (wages, salaries, interest, dividends, etc.) along with payments made to households through refundable tax credits such as the EITC. The federal or state agencies that administer SNAP, Medicaid and other programs could provide household-by-household data directly to the database. Employers could report the cash-equivalent value of employer-provided health benefits along with earnings and taxable benefits.

Unify reporting of deductions. Eligibility for any specific program would depend on a household’s net resources – that is, resources reported from the central database less applicable deductions that vary by household, such as work expenses, out-of-pocket medical expenses, homeowner costs and so on. Households would have to submit that data themselves when applying for a program. However, there should be a uniform format for reporting the aforementioned deductions along with uniform standards for documenting them, so that the information can be submitted to multiple benefit programs with the least possible burden.

Harmonize program phase-outs. As noted above, many anti-poverty programs are subject to phase-ins and phase-outs. Once a uniform household-by-household measure of net resources is available, it will be possible to compute benefit phaseouts for all programs in a harmonized fashion. Replacing separate phase-outs for individual programs with a
single harmonized-benefit reduction rate
would significantly mitigate work disincentives. A harmonized system would also go a long way toward reducing marriage penalties.

* * *

Implementing all of these steps would clearly be a major undertaking. However, the result would be a social safety net that is simpler, less prone to error, and less administratively burdensome, both for state and federal agencies and for poor households themselves.

Taken together, these steps could move us toward a poverty measure that could be used both for tracking and for treating poverty. The steps I outlined imply the need for many changes in definitions, procedures and data collection.

But importantly, what the steps do not specify is just where to draw the line between the poor and the nonpoor – the line that would determine whether the new system would, on balance, be more or less generous than the current system. As Senator Rubio rightly notes, “specifying a poverty line is the most judgmental of all aspects of a poverty measure.” In my view, he is also correct that where to draw the line is a question “that can only be answered by elected representatives.”

The six steps in the preceding section explicitly recognize those truths. They allow ample room for elected officials to calibrate measures of needs, resources and benefits at levels that are high or low – but to do so in a way that is simpler, fairer and more accurate than what we have now.

Who could object to that? Well, everyone who believes in a more generous – or less generous – safety net. But surely, this way is better than muddling through with the opaque, inconsistent programs on the books today.