There’s a scene early in the film version of The Lord of the Rings in which Aragorn confronts Frodo about the evil that pursues him. “Are you frightened?” Aragorn asks, to which Frodo answers in the affirmative. “Not nearly frightened enough,” Aragorn replies. That scene comes to mind when I see the collective shrug that seems to greet each new study showing the rise of antibiotic-resistant infectious diseases.

Particularly striking here is the dichotomy between private R&D for new antibiotics and R&D for a host of drugs targeting chronic diseases. Companies pursuing, say, immuno-oncology drugs are attracting cash by the bucketful, while those working on new therapies for bacterial infections struggle to fund development. Oncology accounted for 24 percent of all medical venture-capital dollars in the last decade, with over $9 billion in total dollars invested, according to the Biotechnology Industry Organization. Funding for all infectious disease drugs totaled $4.6 billion, with antibiotics just a third of that.

This disparity is understandable in market terms. For one thing, when treatments for infectious diseases work, they generally work rapidly. Thus producers must recover their fixed R&D costs over relatively few doses — and the public is outraged at the prospect of paying tens of thousands of dollars for a handful of pills or a few months of injections. For another, many infectious diseases are most prevalent in populations that can least afford to pay for them.

Oh, wait; we’ve missed what is arguably the most important market-linked explanation for how we’ve gotten to this perilous position. As has been widely reported, resistance is, in part, the product of overuse of antibiotics, both in people and in livestock. Physicians prescribe them to patients who present with a sore throat or a nagging cough, even when they are highly unlikely to make a difference. Ranchers habitually dose cattle, pigs and chickens with massive quantities of antibiotics both as prophylaxis against bacterial diseases and because they encourage rapid animal growth.

The consequence of all this is what economists would diagnose as a large gap between the private returns to antibiotic development and their value to society. What’s more, the gap is growing rapidly. In May, the UK released a two-year study estimating that about 700,000 people are already dying each year from infections resistant to the widely available antibiotics. Absent major intervention soon, by 2050 some ten million will die each year — more than currently die from cancer. Jim O'Neill, the former Goldman Sachs economist who led the study, estimates that drug-resistant infections will drain $100 trillion from the global economy in that period, about $10,000 for every human alive today.

Were that not sufficiently apocalyptic, barely a week after the May 16^th release, researchers found a 49-year old Pennsylvania woman whose urine sample contained bacteria resistant to colistin, a half-century old antibiotic which has been rarely used in recent decades because it has serious side-effects. Ironically, colistin is now an antibiotic of last resort precisely because today’s “superbugs” have had little exposure to it, and thus (until now) little resistance.

While that patient was saved with other antibiotics, the case “heralds the emergence of a truly pan-drug resistant bacteria,” the Multidrug Resistant Organism Repository and Surveillance Network at the Walter Reed Army Institute of Research researchers wrote. It’s probably just a matter of time until the colistin-resistance gene, mcr-1, leaps to other bacteria that are already resistant to multiple antibiotics.

So are you frightened? Somehow fright has yet to translate into decisive action. The report also found that funding for antibiotic development actually declined 19 percent in the five-year window ending in 2013, compared with the previous five years.  

One part of the solution is to eliminate the profligate use of antibiotics, and there has been movement on that front. Physicians are somewhat less likely to write scripts for runny noses and ear infections these days, and a growing (though still modest) number of restaurants and grocery store meat counters are insisting on animals raised without antibiotics. Last year California passed a law that bans the use of medically important antibiotics to promote growth in animals raised for food. Producers will only be able to administer the drugs with the approval of a veterinarian when animals are sick, or to prevent infections when there's an “elevated risk.”

But new types of antibiotics, with new mechanisms of action, are still needed to treat the resistant bacterial strains that are already established, and here both the free market and public policy are failing. Like Willie Sutton, investors and pharmas go where the money is, and the money is in drugs for widespread chronic conditions — type II diabetes, cancer, cardiovascular diseases — not antibiotics.

“The people who pay for cancer drugs and the people who pay for antibiotics are completely different companies with completely different incentives,” says Roger Longman, chief executive of Real Endpoints, a firm that assesses the value of drugs. “On the cancer side, you’re largely talking about a reimbursement world, which effectively has long abdicated its responsibility for managing costs. Moreover, in Medicare, oncology is one of six so-called protected classes. That means that any Medicare plan … has to effectively include all approved oncology drugs in their formularies. This gives oncology companies an extraordinary amount of leverage.”

It gets worse. More than half of all oncology drugs are “medical benefit” drugs, meaning they are sold by healthcare providers, like physicians, not by pharmacists. Medicare pays a percentage fee on top of the cost of the drug, creating an incentive to use more expensive therapies. By contrast, most late-generation antibiotics are hospital products, meaning they are paid for under the DRG (diagnostic related group) system used by insurers, which sets a fixed fee for the treatment of specific conditions. That means hospitals do have incentives to economize on pharmaceuticals.

“Hospital pharmacists, who are extraordinarily powerful people, can limit the use of expensive hospital drugs, even very valuable drugs,” says Longman. “That’s why you rarely see very expensive hospital drugs, the way you see very expensive drugs for cancer, hepatitis C or rheumatoid arthritis,” which are administered in outpatient settings, he says. “The market looks at this, and says not only do I have a straightforward path to approval with the oncology drugs, but I can charge whatever I want,” Longman adds. With antibiotics, there a more torturous path to profits because new anti-infectives must deal with the bean-counters in hospitals and prove superior to existing (usually cheaper) therapy.

“Economically, if you could price out a drug that cured an 18-year-old of a life-threatening disease, think what you could charge for a drug like that,” says James Knighton, president of AvidBiotics, a biotech startup based in South San Francisco. “Culturally, it’s just the opposite. People are just not used to paying big dollars for antibiotics.”

Knighton says his company struggles to make the case to investors for its continued pursuit of antibiotics, while its immuno-oncology program is an easy sell. “Investors are not a complicated group; a lot of them follow the lead of others who have made a fortune,” he says. “Eighty percent are in immuno-oncology, very few in antibiotics. People have made a lot of money in oncology.”

The market for pharmaceuticals is an economist’s horror show, one rife with inefficiencies and inconsistencies, that in some cases promotes profligate use while in others chokes off innovation even when the societal return is high. To a degree, this is inevitable. Health care, after all, is driven by an uneasy mix of technocratic public policy, political lobbying and private decision-making — decisions often made with limited knowledge by parties who won’t suffer much if they're wrong.

But this much is clear: If we don’t get our act together in facilitating the development of new antibiotics and husbanding the ones we already have, bacterial diseases that have almost disappeared in developed countries will again stalk the land.