Clinicians at Valley-Wide Health Systems never know who will appear at their clinic in San Luis, a town of about 600 people in southern Colorado.

“If someone's in labor, they'll show up. If someone has a laceration, they'll show up,” said nurse practitioner Emelin Martinez, the chief medical officer for the health care system serving 13 rural Colorado counties.

But she struggled to find a full-time medical provider for that clinic, the only one in Costilla County. Born and raised in the area, Martinez filled some of the gap by driving about 45 minutes from Alamosa, the nearest city, once a week for months. A physician assistant from another town chipped in, too.

As one of the nation’s more than 1,000 federally designated primary care shortage areas, Costilla County has many carrots to dangle in front of medical providers willing to practice there, including federal student loan repayments, bonus Medicare payments, and expedited visas for foreign clinicians. Still, Martinez said, its latest opening remained unfilled for more than a year. Not a single physician applied.

Policymakers have long tried to lure more primary care providers to the areas of the nation that have fewer than one physician for every 3,500 residents. Recent examples include the Biden administration boosting funding in 2022 to address shortages and Sen. Bernie Sanders (I-Vt.) pushing sweeping primary care legislation in 2023.

But researchers steeped in the issue have a persistent frustration: It’s hard to know if any policy is working given that the data the federal government collects on primary care shortage areas has been flawed for a long time. One of the biggest gaps is that the system counts only physicians, not the myriad other health care professionals who now provide much of our nation’s primary care.

Additionally, a Health Affairs study shows the federal designations, which help allocate an estimated $1 billion in annual funding through at least 20 federal programs aimed at boosting primary care capacity, haven’t helped much.

In fact, Costilla County is among more than 180 federally designated areas that have remained stuck on the primary care shortage list for at least 40 years, according to a KFF Health News analysis. That’s even as the overall number of licensed U.S. physicians more than doubled from 1990 to 2022 to over 1 million, according to the Federation of State Medical Boards, outpacing overall population growth.

No one disputes that much of the nation is starved for primary care clinicians, with patients having to wait weeks to get appointments or travel long distances for basic preventive care. Many doctors decide against primary care career paths, let alone practicing in isolated communities, because those jobs entail heavy workloads and earn less money and respect than specialists. But how does the nation solve the problem without knowing exactly where it is? And what tools must be used? Does a physician need to be the one providing the care?

Whitney Zahnd, president of the board of the Iowa Rural Health Association, said the fact that some rural areas have had such federal shortage designations for decades doesn’t prove they are ineffective. “Had the program not been there, would it have been even worse?” she said.

Federal funding supports 18,000 primary care doctors, nurse practitioners, and physician assistants to provide care to more than 18 million patients in the highest-need urban and rural communities across the country, said David Bowman, a spokesperson for the Health Resources and Services Administration, which manages the shortage designations. He said more than 80% of clinicians who get such scholarships or loan repayments continue to practice in shortage areas beyond their obligation of several years.

But that doesn’t mean they stick around forever.

Justin Markowski, a Yale School of Public Health doctoral student, co-authored the Health Affairs study that found the federal shortage designation makes no difference in upping physician density long-term. He is skeptical of policy ideas that promise big primary care fixes. That includes the Biden administration’s investment in more scholarships and loan repayments through the National Health Service Corps.

“You're just throwing more money at a set of programs that don't really seem to work,” he said. “We'll see in a few years, but I'll be shocked if it actually moved any physicians or any other advanced practice providers.”

One possible explanation for the persistence of shortage areas is that such incentives are too small or too fleeting.

But another issue is how shortages are measured. The government considers geographic shortage areas, now numbering just over 1,000, but also population groups such as migrant farmworkers and individual facilities such as prisons that lack enough providers. Yet it’s up to state offices to identify populations and locations that might qualify as shortage areas and submit them to HRSA, which then scores the extent of any shortages. The funding and staffing for those state offices vary, creating an uneven foundation from which to map actual shortages.

“Some states became very adept at the equivalent of gerrymandering, where they were piecing together census blocks or census tracts in odd shapes in order to maximize the areas that are eligible,” said Stephen Petterson, a senior scholar at the Robert Graham Center, a policy think tank in Washington, D.C., that focuses on primary care.

The federal Government Accountability Office has highlighted such issues since at least 1995, when it released a report identifying widespread data problems with the shortage area system and concluding it had “little assurance that federal funds are used where most needed.” The report noted one of the persistent shortcomings is that the system counts only physicians, not other key primary care providers.

Since 1998, federal officials have made three attempts to update the 1970s-era rules that define what counts as a shortage area. The authors of the Affordable Care Act tried most recently, tasking a committee of experts to decide on an update.

Among other things, the committee concluded in its 2011 report that nurse practitioners, physician assistants, and certified nurse midwives should be counted as primary care providers. But the recommendations fell short by just a handful of votes.

“We failed and the committee as a whole failed and HRSA failed by not moving the process forward,” said Petterson, who presented to the committee on how to comprehensively measure primary care needs.

Steve Holloway, who directs the Colorado health department’s Primary Care Office, served on the committee. Without action at the federal level, he then led a team to create Colorado’s own health professional shortage area designations that factor in nurse practitioners and physician assistants, not just doctors.

He said it’s taken about six years to create a tool and map of Colorado to answer a deceptively simple question: “How many actual flesh-and-blood, live clinicians are seeing patients?”

Will the doctor see you now?Maybe not amid the changing health care landscape in the US

Health care deficitA third of Americans don't have a primary care provider, report finds

Ed Salsberg, who was the lead federal government representative on that committee and who headed HRSA’s National Center for Health Workforce Analysis, said the rest of the nation needs more precise data, too.

“It’s so important for the nation to target its resources to the highest-need communities,” he said. “It's time again to try one more time to develop an improved methodology.”

In the past few years, more readily available data from insurance claims has allowed researchers to distinguish the medical providers who are practicing primary care from those who have specialized or retired.

Candice Chen, an associate professor of health policy and management at George Washington University’s Fitzhugh Mullan Institute for Health Workforce Equity, used claims data that reflects one large slice of the American population — about 66 million Medicaid beneficiaries — to map the primary care workforce.

Meanwhile, Monica O’Reilly-Jacob, a nurse-scientist who recently moved from Boston College to Columbia University’s School of Nursing, studied Medicare claims to conclude that fewer than 70% of physicians typically considered primary care providers were actually providing primary care. The rest, she said, often find more lucrative positions, such as subspecializing or working in hospitals. By contrast, nurse practitioners are likely undercounted. Her study found that close to half are providing primary care.

But such publicly available data leaves out much of the country, given that fewer than 40% of Americans are insured through Medicaid or Medicare.

“There's no government organization that's tracking: Who trained in what, where, and where are they now, and what are they practicing,” said Alison Huffstetler, medical director of the Robert Graham Center. “And if we don't know who is doing what kind of care — and where — then there is no way for us to equitably manage the patient-to-clinician ratio across every state."

In Costilla County, Martinez finally found someone to provide primary care: an experienced physician assistant who moved from Texas in December.

The physician assistant’s presence should bump the county out of its dire shortage, according to Colorado’s measure. But since he isn’t a physician, he’ll remain invisible in the national data and Costilla County will likely remain on the books as a federal shortage area.

Data reporter Hannah Recht, data editor Holly K. Hacker, and rural editor/correspondent Tony Leys contributed to this report.

KFF Health News, formerly known as Kaiser Health News (KHN), is a national newsroom that produces in-depth journalism about health issues and is one of the core operating programs atKFF — an independent source for health policy research, polling, and journalism.