ProPublica Explains How DOGE’s AI Cut Support for Veterans Care
Even among ProPublica’s exceptional work exposing DOGE’s failures, this is notable.
ProPublica used the opportunity of the disgruntled departure of an engineer named Sahil Lavingia from DOGE as an opportunity to unpack a specific task he took on, and botched. It provides valuable insight to the source of errors as Elon unleashed a bunch of coders on federal bureaucracy without the context to understand what they were doing.
Lavingia joined DOGE — after previously attempting to get a job with DOGE’s nonpartisan predecessor, US Digital Services — with a genuine wish to improve the way government works. He was assigned to review VA contracts to decide which could be “munched” — canceled. He claims that after his AI review of the contracts, people with some actual knowledge of the VA services should have reviewed the contracts he flagged to prevent obvious errors. It appears that didn’t happen, and as happened so often elsewhere, pretty critical contracts were cut.
Lavingia’s ouster and his willingness to speak up provides a glimpse of what has led to such stupid decisions from DOGE.
Back in March, after asking Elon at the sole all-hands DOGE meeting he ever attended if he could open source his code, he published it to GitHub. Months later he did an interview with FastCompany, which led to his firing.
Since his firing, in addition to telling multiple media outlets that there really wasn’t the kind of waste he’d expected, he walked ProPublica through the specifics of a task he was assigned, reviewing VA contracts for DEI and waste, which has led to key contracts getting canceled.
VA officials have said they’ve killed nearly 600 contracts overall. Congressional Democrats have been pressing VA leaders for specific details of what’s been canceled without success.
We identified at least two dozen on the DOGE list that have been canceled so far. Among the canceled contracts was one to maintain a gene sequencing device used to develop better cancer treatments. Another was for blood sample analysis in support of a VA research project. Another was to provide additional tools to measure and improve the care nurses provide.
[snip]
Sahil Lavingia, the programmer enlisted by DOGE, which was then run by Elon Musk, acknowledged flaws in the code.
“I think that mistakes were made,” said Lavingia, who worked at DOGE for nearly two months. “I’m sure mistakes were made. Mistakes are always made. I would never recommend someone run my code and do what it says. It’s like that ‘Office’ episode where Steve Carell drives into the lake because Google Maps says drive into the lake. Do not drive into the lake.”
But the really great thing ProPublica did was to have experts, including Waldo Jaquith, who used to do IT at Treasury, review Lavingia’s code to explain how it went wrong.
You should read both stories, but here’s where things went wrong.
First, rather than simply consulting USA Spending to learn what contracts were doing and how much they were spending, Lavingia instead used AI to review the contracts themselves, which often had outdated information.
This portion of the prompt instructs the AI to extract the contract number and other key details of a contract, such as the “total contract value.”
This was error-prone and not necessary, as accurate contract information can already be found in publicly available databases like USASpending. In some cases, this led to the AI system being given an outdated version of a contract, which led to it reporting a misleadingly large contract amount. In other cases, the model mistakenly pulled an irrelevant number from the page instead of the contract value.
When he did that, though, Lavingia only asked AI to review the first 10,000 characters of the contracts, which isn’t where some of the most important information (not to mention information on whether a contract included a DEI component) would be found.
Analyze the following contract text and extract the basic information below. If you can’t find specific information, write “Not found”.
CONTRACT TEXT:
{text[:10000]} # Using first 10000 chars to stay within token limitsThe models were only shown the first 10,000 characters from each document, or approximately 2,500 words. Experts were confused by this, noting that OpenAI models support inputs over 50 times that size. Lavingia said that he had to use an older AI model that the VA had already signed a contract for.
He did that, he explained, because the VA only had dated AI that could only handle 10,000 characters.
Then the script prompted to assess whether contracts provided “direct patient care,” defined first by including “medical procedures,” then excluding “psychosocial support” of the sort that keeps Veterans alive, measuring how many layers removed from actual care a contract was, then finally running it through a list of things like audits (including “Nuclear physics and radiation safety audits for medical equipment” !!) that could not be “munched,” or canceled.
These two lines — which experts say were poorly defined — carried the most weight in the DOGE analysis. The response from the AI frequently cited these reasons as the justification for munchability. Nearly every justification included a form of the phrase “direct patient care,” and in a third of cases the model flagged contracts because it stated the services could be handled in-house.
But the exclusion of audits didn’t work.
The article provided one example of the kind of obvious (literal) patient support that got targeted for cancelation: the maintenance contracts for ceiling lifts used to reposition patients during their care.
The emphasis on “direct patient care” is reflected in how often the AI cited it in its recommendations, even when the model did not have any information about a contract. In one instance where it labeled every field “not found,” it still decided the contract was munchable. It gave this reason:
Without evidence that it involves essential medical procedures or direct clinical support, and assuming the contract is for administrative or related support services, it meets the criteria for being classified as munchable.
In reality, this contract was for the preventative maintenance of important safety devices known as ceiling lifts at VA medical centers, including three sites in Maryland. The contract itself stated:
Ceiling Lifts are used by employees to reposition patients during their care. They are critical safety devices for employees and patients, and must be maintained and inspected appropriately.
Back in February, Doug Collins bragged about the work DOGE was doing reviewing contracts.
This was, he said, the work DOGE was supposed to be doing.
I guess Doug Collins believed his job running the VA involved eliminating critical care based on shoddy code.
I realize this comment will not further the post all that much if at all. But as a former IT guy I CANNOT resist.
Here it is. I am sure others can do better.
So, Trump and Musk begin to munch on each other.
So there.
But the more I learn about these AI based deconstruction efforts the more I…puke in my mouth.
Even non-shoddy code needs review and more review. Our son is an Android developer and the codes, even when good need to be checked and rechecked. Seems like Musk likes people, like himself, who shoot from the hip. Another case of: Fire, Ready, Aim!
Musk is a “move fast and break things” guy. He doesn’t understand systems, or engineering, or, really, anything else but money and power – and I’m not sure he understands those.
The genesis – for me – of this (SW/HW) design philosophy dates back to the 1990s where a little known UNIX mini computer vendor in SE Florida engaged with a company that had a rather weird UNIX based emulator for IBM disk controllers and DASD for IBM 360/370 environments.
As you might expect – if you are as old as me or just plain “normal” – companies who consider(ed) their business computer systems to be mission critical six-sigma uptime environments – and thus paid big money for IBM mainframes (yeah it’s a bit different now) – would not knowingly buy a product from a “move fast and break things” vendor.
So, yeah this otherwise very interesting and if engineered with a different mindset product might have really had a place – think EMC of the time – didn’t last long. No one was gonna replace mainframes very easily at the time, but DASD was very expensive so if you could get more for less and maintain reliability you could be a winner.; Again, think EMC who even they got absorbed into DELL of all companies in 2016. Computer tech was a wild ride – for me – 1990s through 2010s..
And so I moved on as well as a product of IBM mainframe business systems who transitioned to UNIX based Open Systems and was – at this time – a sales support systems engineer – who could talk IBM as well as UNIX (a bit rare at the time).
So, even I knew the technical interesting product was not gonna fly when it fell on its face under stress in a conservative IBM mainframe environment.
reply to PJ Evans:
And mainly he doesn’t care. Those ceiling lifts? They save the backs of nurses, to say nothing of the lives of vets. But so what? This man blithely dismissed the lives of African children as if they were gnats–not even gnats. As if they were nothing, which is what they are to a sociopath like him, or Trump, incapable of imagining the life of anyone except himself.
“He was assigned to review VA contracts to decide which could be “munched” — canceled. He claims that after his AI review of the contracts, people with some actual knowledge of the VA services should have reviewed the contracts he flagged to prevent obvious errors.”
If you’re using AI to review something, what exactly are you doing? Wouldn’t the people with “actual knowledge” be doing your work for you? Seems to me that Lavingia himself was the waste, fraud, and abuse.
Josh Marshall has written imho the most thorough article about AI that I’ve read in a long time.
Gift link
https://talkingpointsmemo.com/edblog/artificial-intelligence-and-the-posture-of-skepticism/sharetoken/0afc0c51-4bc0-4d13-90f8-63be155c3be8
I finally got around to reading Marshall’s essay. I don’t find anything I can argue with in it. And it does provoke thought. Computers can sort through and find information at lightning speed, but I balk at them creating information. You would think that a programmer could direct AI to use actual citations rather than made up ones. I’m not talking about ethics but ones and zeros. But maybe I’m simplifying things. Still, it seems that AI is still very much in the garbage in garbage out phase. Or, perhaps worse, it is trapped in an ideological framework that it doesn’t question just as we “freethinking” humans usually are.
Doug Collins might have let the cat out of the bag. Disabling the VA probably is his job, not making it work better.
How can anyone audit a company without knowing what it does or why, and not have anyone check their work before disabling critical govt infrastructure? Next thing you know, Trump will appoint a 22-yr-old intern to direct DHS’s anti-terrorism efforts.
Marcy: Since his firing, in addition to telling multiple media outlets that there really wasn’t the kind of waste he’d expected [NPR link]
I know it’s not the point of this post,
but because I’ve been silently obsessing about this exact point for months…
here’s Lavingia‘s answer in full:
If it isn’t on paper, it isn’t real. Electronic records can be faked; we know that. So can photos. Paper records are harder to fake. Letters to taxpayers/clients are safer than phone calls and emails, and harder to fake.
So…maybe not *elimination* of paper, but perhaps cutting down on the faxing? I confess to being surprised that faxes are still (apparently) in regular use in government. I’d love to know how prevalent that use really is.
And yes, PJ, I see your point about the fungibility of cyber records. Would not trust this administration with my own life’s historical interface with Big Record Keeping.
I watched his interview with my significant other and I just had to laugh when he said those words as I have uttered them many times myself. Is there waste in the US Government, absolutely. Are employees in each Agency purposely trying to waste money, absolutely not. They are stuck in their departments working on the things that their managers have been tasked to achieve so they go about doing their work with the tools and managers they have been given. Since many of the Agencies and Departments have similar mandates, you end up with different ways and software to achieve the goals in your Agency. If I were running DOGE, I would have looked across all those Agencies and Departments to find like processes and software, then picked the best in class to implement across all the activities.
As an example, almost all Federal Agencies utilize protocols developed by the United States Construction Engineering Research Laboratory to evaluate their compliance with environmental, health and safety requirements. The protocols are developed in a specific MS Word format then those documents are run through a piece of software that outputs an Access Database.
That core Database is then distributed out to all the Agencies and Departments to conduct Environmental Compliance Audits (ECAs). However, each one of these Agencies uses a different and proprietary piece of software to conduct these ECAs. Essentially each Agency or Department is using the same core database to evaluate its operations, spit out reports and track compliance using a different piece of software depending on the Agency. The Forest Service software is different from the US Army’s that is different from the National Guard, that is different from the Park Service, that is different from the VA’s, that is different from the BLM’s and on it goes, with each one of these systems costing the Agencies a wide range of costs. For instance, if I recall correctly the Army’s software costs about 3 million per year, while the Forest Service’s cost nothing.
DOGE or Congress should be looking at those types of efficiencies; however, until Musk came along the gate keeping contractors like BAH or the Techno overlords at VOLPE prevented any kind of efficiency across agencies. So instead of fixing stuff, they broke it. Just like the man said.
The whole concept of slaying “fraud waste and abuse” (FWA) as if it was a mythical dragon is a non-sequitur. FWA is an unavoidable by-product of performing mission. There is a spectrum between regulation and oversight and mission. On one end is a highly regulated environment which doesn’t perform the mission well because of bureaucratic roadblocks but will have little to no FWA. Think of a car engine with no oil. The other end of the spectrum is a very loosely regulated environment where performing mission is the only imperative which will be rife with FWA.
Finding the right balance of regulation and oversight and mission performance will naturally limit FWA, but trying to fix individual FWA items is pointless. New ones will rapidly spring up to replace them. The way to “fix” it is to move toward the regulation and oversight end of the spectrum and accept the loss of mission effectiveness that entails.
AI is a model, which comes with assumptions needed to make the model fit the available resources (like memory storage). However, any assumption still needs to be validated and the inputs need to be accurate (we also see this in the self-driving vehicles) and representative. Lavingia’s confession reveals both aspects to be missing as well as a continuing review to ensure assumptions at time X are still valid at time Y. No wonder the results are actually inverse to the claims – DOGE has cost more than its saved.
Elno grabbed these DOGE commissars because they were cheap, stupid and not going to challenge his ‘genius’. They’re now burrowed in and I would expect a sudden resurrection of civil service norms to keep them in place as long as possible. Make the GOP own this, and the town hall reports show some of the voters are venting. I’ll be more impressed when they vote the GOP out.
LLMs don’t “know” or “understand” anything. They can’t reason, they can’t perform logic and they don’t track the provenance of information to know if it should be considered trustworthy or not.
LLMs just create an illusion of doing these things. They don’t process information in any way similar to what human thinkers do.
Their model is generated by statistically processing word frequencies in a vast training set of data from actual mostly-human activities. What LLMs excel at, is writing human-sounding gibberish drawing from bits and pieces of their training data.
Clever uses of this technology can interface it to tools, use it to generate images or even video, and so on. But calling it “artificial intelligence” is marketing spin. Humans are good at anthropomorphizing, and its important to remember that “AI” is not intelligent at all and not trustworthy. It does not “think” in any way, and it does not understand the text it spits out.
It can “hallucinate” sources and citations, it can “make up facts”, because it is just stringing words together that are most probable in the given context. Anyone who trusts its output for anything important is a fool.
GIGO, for sure, and the tap-dancing to downplay the BS health report this week just highlights how unsupervised modeling can screw things up (remember the old joke about how to really screw things up?).
However, whenever AI is raised in the media it’s in fawning terms about how new / exciting / successful / awesome it is like some new high school romcom relationship. Nothing about the flaws.
It seems to me nothing more than Unix grep on steroids. As far as intelligence, it’s on the same level as an amoeba.
“Ceiling Lifts are used by employees to reposition patients during their care.”
It is poignant and sad to be given an example of just where it is that the harm created by DOGE can ultimately accrue. Some of the most human-to-human interactions occur at the bedside of the sick. Of these interactions, to me, one stands out: a human body caring for, by moving, another human body; where those who can move, move—bear the weight of—those who cannot. Why should we, as a wealthy country, forsake “effectiveness” for some misguided notion of “efficiency”?
This is pretty confusing to me. Are they calling feeding prompts to an AI “coding”? And what was the VA doing “owning” any kind of AI at all? The explanation of how “mistakes were made” (the kid after all learns fast) makes perfect sense otherwise. Using a predictive algorithm to make factual analysis is not a great idea.
Trump’s policies are the epitome of waste, fraud, and abuse, e.g., burn more coal to take long hot showers to clean his beautiful hair.
Citizens United guaranteed that plutocracy would write the rules, most essentially the tax laws, while erasing as much regulation as possible. This is the definition of abuse. Trump simply personifies it in especially ugly and blatant ways; it’s what the rich people we don’t see or hear (enough) about want too. They are hiding behind his ill-fitting flapping pant legs with their hands out, and they’ll overlook a little slap at Leonard Leo if it means they make out all right in the end.
Which they always do.
AI is a tool. A hammer is a tool you can hold in your hands. You can analyze it. It becomes something other than itself. It hammer becomes you, a way of seeing and doing. A hammer is a kind of social institution. The hammer is mythological object in Thor’s stories where it projects strength, shelter, overcoming and protection. A myth reflects a narrative to explain the world.
Like a hammer, AI is a kind of tool. It uses factorial analysis in imitation of the heuristics of thought. DOGE is using AI to administer, giving AI the authority of an oracle, so AI has a projection as techno-utopian myth, a myth of fascism, as a kind of phenomenology, because AI provides a black box arbitrary ethos based on nothingness. Arguing with AI is like arguing with Mao or saying Comrade Stalin doesn’t know.
The myth of AI is like cryptocurrency, which is also a myth. The myths of AI and cryptocurrency teach that thought and value are imitation and so human identity is a myth too, and there can be a Department of Reality, where AI is a kind of “truth” in the sense of offering a way forward for everything that has gotten lost.
I read the information about ceiling lift maintenance contracts not being renewed with great interest. I have Guillain-Barre Syndrome: have had it since 1968. I use a wheelchair: I cannot walk. I have a portable lift (a “Hoyer” lift) that raises me out of bed and lowers me into my wheelchair. Vice versa at night.
For various reasons, I am segueing to VA care, and many (all?) of their beds have Hoyer-type lifts built into the ceilings. That enables me to do care at a VA facility, and transfer there from bed to wheelchair and back again. What an unpleasant surprise to find that DOGEBAGs have put my care and safety inside VA facilities at risk.
I request that everyone reading this post forward the article to their Congress-critter and Senators, asking them to restore the cuts for maintaining these devices. My life and my safety depend on it. Also ask them to fire the SOB in charge of the VA (a loyalist for Emperor Felon), if only due to his claim that the cuts did not/do not impair safety for VA personnel or patients; that’s an outright lie – and this article proves it beyond doubt.
Thank you.
Thank you, Dennis K Mann, for sharing your perspective. If only DOGE and its implementers had cared to seek it out *before* trashing the system based on little more than whim.
I will again add that these ceiling lifts make an invaluable contribution to the health and safety of the medical staff, especially nurses, who care for patients. Maybe Doug Collins thinks the VA will staff itself with college linebackers. That would only happen if the pay got a lot better, and God knows the Trump administration isn’t going to waste money paying people what they deserve–not when they can line their own and their donors’ pockets with obscene tax.cuts and grift.
The horror stories have been around for a while that AI may/will destroy us. Here we have examples of it’s pushers trying to do so starting with our government.
Interestingly, this destructive force isn’t because it is so smart. More so because it isn’t.
This is reassuring:
Supreme Court allows DOGE team to access Social Security systems with data on millions of Americans
[AP/Seattle Times]
“The Supreme Court handed the Trump administration two victories Friday in cases involving the Department of Government Efficiency, including giving it access to Social Security systems containing personal data on millions of Americans.
The justices also separately reined in orders seeking transparency at DOGE, the team once led by billionaire Elon Musk.
The court’s conservative majority sided with the Trump administration in the first Supreme Court appeals involving DOGE. …”
Of course they did, being able to read the Constitution’s sections written in invisible ink,
which mere mortals are not.
https://www.seattletimes.com/nation-world/nation/supreme-court-allows-doge-team-to-access-social-security-systems-with-data-on-millions-of-americans/
The whole rampant “waste and fraud” in government is directly descended from the demoguary against Reconstruction by the Redeemers intent on reestablishing white supremacy in the aftermath of the Civil War. Reagan’s whole spiel, and all the white wingnuttery today, is nothing but racist dog whistles harking back to that time, which in almost every case can be translated into “no free shit for black people”.