Tag Archive for: Sahil Lavingia

Even among ProPublica’s exceptional work exposing DOGE’s failures, this is notable.

ProPublica used the opportunity of the disgruntled departure of an engineer named Sahil Lavingia from DOGE as an opportunity to unpack a specific task he took on, and botched. It provides valuable insight to the source of errors as Elon unleashed a bunch of coders on federal bureaucracy without the context to understand what they were doing.

Lavingia joined DOGE — after previously attempting to get a job with DOGE’s nonpartisan predecessor, US Digital Services — with a genuine wish to improve the way government works. He was assigned to review VA contracts to decide which could be “munched” — canceled. He claims that after his AI review of the contracts, people with some actual knowledge of the VA services should have reviewed the contracts he flagged to prevent obvious errors. It appears that didn’t happen, and as happened so often elsewhere, pretty critical contracts were cut.

Lavingia’s ouster and his willingness to speak up provides a glimpse of what has led to such stupid decisions from DOGE.

Back in March, after asking Elon at the sole all-hands DOGE meeting he ever attended if he could open source his code, he published it to GitHub. Months later he did an interview with FastCompany, which led to his firing.

Since his firing, in addition to telling multiple media outlets that there really wasn’t the kind of waste he’d expected, he walked ProPublica through the specifics of a task he was assigned, reviewing VA contracts for DEI and waste, which has led to key contracts getting canceled.

VA officials have said they’ve killed nearly 600 contracts overall. Congressional Democrats have been pressing VA leaders for specific details of what’s been canceled without success.

We identified at least two dozen on the DOGE list that have been canceled so far. Among the canceled contracts was one to maintain a gene sequencing device used to develop better cancer treatments. Another was for blood sample analysis in support of a VA research project. Another was to provide additional tools to measure and improve the care nurses provide.

[snip]

Sahil Lavingia, the programmer enlisted by DOGE, which was then run by Elon Musk, acknowledged flaws in the code.

“I think that mistakes were made,” said Lavingia, who worked at DOGE for nearly two months. “I’m sure mistakes were made. Mistakes are always made. I would never recommend someone run my code and do what it says. It’s like that ‘Office’ episode where Steve Carell drives into the lake because Google Maps says drive into the lake. Do not drive into the lake.”

But the really great thing ProPublica did was to have experts, including Waldo Jaquith, who used to do IT at Treasury, review Lavingia’s code to explain how it went wrong.

You should read both stories, but here’s where things went wrong.

First, rather than simply consulting USA Spending to learn what contracts were doing and how much they were spending, Lavingia instead used AI to review the contracts themselves, which often had outdated information.

This portion of the prompt instructs the AI to extract the contract number and other key details of a contract, such as the “total contract value.”

This was error-prone and not necessary, as accurate contract information can already be found in publicly available databases like USASpending. In some cases, this led to the AI system being given an outdated version of a contract, which led to it reporting a misleadingly large contract amount. In other cases, the model mistakenly pulled an irrelevant number from the page instead of the contract value.

When he did that, though, Lavingia only asked AI to review the first 10,000 characters of the contracts, which isn’t where some of the most important information (not to mention information on whether a contract included a DEI component) would be found.

Analyze the following contract text and extract the basic information below. If you can’t find specific information, write “Not found”.

CONTRACT TEXT:
{text[:10000]} # Using first 10000 chars to stay within token limits

The models were only shown the first 10,000 characters from each document, or approximately 2,500 words. Experts were confused by this, noting that OpenAI models support inputs over 50 times that size. Lavingia said that he had to use an older AI model that the VA had already signed a contract for.

He did that, he explained, because the VA only had dated AI that could only handle 10,000 characters.

Then the script prompted to assess whether contracts provided “direct patient care,” defined first by including “medical procedures,” then excluding “psychosocial support” of the sort that keeps Veterans alive, measuring how many layers removed from actual care a contract was, then finally running it through a list of things like audits (including “Nuclear physics and radiation safety audits for medical equipment” !!) that could not be “munched,” or canceled.

These two lines — which experts say were poorly defined — carried the most weight in the DOGE analysis. The response from the AI frequently cited these reasons as the justification for munchability. Nearly every justification included a form of the phrase “direct patient care,” and in a third of cases the model flagged contracts because it stated the services could be handled in-house.

But the exclusion of audits didn’t work.

The article provided one example of the kind of obvious (literal) patient support that got targeted for cancelation: the maintenance contracts for ceiling lifts used to reposition patients during their care.

The emphasis on “direct patient care” is reflected in how often the AI cited it in its recommendations, even when the model did not have any information about a contract. In one instance where it labeled every field “not found,” it still decided the contract was munchable. It gave this reason:

Without evidence that it involves essential medical procedures or direct clinical support, and assuming the contract is for administrative or related support services, it meets the criteria for being classified as munchable.

In reality, this contract was for the preventative maintenance of important safety devices known as ceiling lifts at VA medical centers, including three sites in Maryland. The contract itself stated:

Ceiling Lifts are used by employees to reposition patients during their care. They are critical safety devices for employees and patients, and must be maintained and inspected appropriately.

Back in February, Doug Collins bragged about the work DOGE was doing reviewing contracts.

 

This was, he said, the work DOGE was supposed to be doing.

I guess Doug Collins believed his job running the VA involved eliminating critical care based on shoddy code.