But the rates did not move a single point and we have spent four million on the portal alone. The man from the audit team stood up and he looked at the chart on the big screen and he rubbed his face like he was trying to wipe away the bad news. I stood in the back of the room and I had the hiccups and every time my chest jumped the people in the front row turned around and glared at me.

It was the fifth year of the new reform plan and we had built the best integrity office in the state and we had three people whose only job was to check if researchers were lying on their forms. We had a portal where you had to write down every step of your plan before you even touched a flask and we had a box you had to check to say you did not have a secret pile of money from a drug company.

Everything was clean and everything was tracked and the math still came out wrong when the next lab tried to do the same work. The board looked at the slide that showed the failure rate and they did not see a reason to stop and they saw a reason to double the budget for the oversight office. They thought the researchers were just getting better at hiding their mistakes and they wanted more eyes and more forms and more rules.

The Ethics Class Dead-End

I have spent fifteen years as a trainer and I have seen this happen in three different states and it is always the same story. If the study does not work then we say the person was lazy or the person was a cheat and we send them to a class where I have to stand in front of them and talk about ethics for six hours.

I once had the hiccups for twelve minutes during a talk about data bias and the room just sat there and watched me struggle and I think that is how these scientists feel when they follow every rule and the result still dies in their hands. They fill out the forms and they check the boxes and they wait for the mail and then they run the test and the test fails and the institution tells them they need to be more honest next time. It is a loop and it is a very expensive way to go nowhere.

The problem is that we look at the person and we never look at the stuff in the vial. We assume that if a company sells you a powder then that powder is what the label says it is and we build our whole world on that trust. But the world of research supply is a dark place and it is full of middle men and white labels and labs that haven’t cleaned their machines since the last president was in office.

The Ghost Variable in the Supply Chain

We found that seven out of ten of these labs just buy the cheapest thing they can find on a website and they never check it because checking it costs time and money and they think the vendor would not lie to them. They put the powder in the water and they put the water in the cell and then they wonder why the cell behaves in a way that no one has ever seen before.

They go back to their portal and they update their logs and they swear they followed the protocol and they did follow it. They followed a perfect protocol with a ghost variable that they did not even know was there.

The Sawdust in the Flour

If you buy a bag of flour and it is half sawdust then your bread will not rise and it does not matter how many times you wash your hands or how carefully you measure the salt. You can have a priest come and bless the kitchen and you can hire a man to watch you bake and you can write a long paper about your baking plan but the bread will still be bad.

“In the world of peptides and proteins the sawdust is everywhere and it is called impurity or it is just the wrong stuff entirely.”

I have seen reports where the vial was forty percent salt and the researcher spent six months trying to figure out why his mouse died. He thought he was a bad scientist and his boss thought he was a bad scientist and the integrity office thought he was hiding something. No one thought to ask if the vial was a lie because we are trained to look at the behavior of the person and not the quality of the input.

This is where the career risk comes in for people like me. If I tell the board that their four million dollar portal is a waste of time then I am out of a job and if I tell them the problem is the supply chain then they have to do work that they do not know how to do. They know how to hire more trainers and they know how to buy software and they know how to make people take tests.

They do not know how to verify a lot number or how to read a mass spec report. They want the world to be made of forms because forms are easy to count and vials are hard to test. We have created a system where the paper is more real than the chemical and that is why the replication crisis is not going away.

The Most Careful Man in the Lab

I remember a guy named Mark who worked in a neuro lab and he was the most careful man I ever met. He color coded his pipettes and he kept a diary of the room temperature and he even timed how long he held the door open so the air would not move too much. He spent two years on a study about brain signaling and he did everything by the book and he used the portal for every single step.

When he went to publish the results a lab in California tried to do it again and they got nothing and then a lab in Germany tried and they got nothing. Mark was devastated and he thought his career was over and he came to my ethics class with his head down like a dog that had been kicked. He thought he had made a mistake that he could not find and he was looking for me to tell him where he went wrong.

I told him to send his last three vials to a real lab for a weight test and a light test and he found out the peptide was only sixty percent pure. The rest was just junk from the factory and it had been killing his cells for twenty four months.

He took that report to his dean and the dean did not want to hear it. The dean said that Mark should have known better and that the lab should have a better way to pick vendors but the school did not provide any money for testing. The school provided money for the integrity office and the school provided money for the ethics trainer but they did not provide a single dollar to make sure the chemicals were real. Mark had followed the rules but the rules were designed to catch a liar and not to catch a bad batch of dust. He ended up leaving the field because he could not stand the thought of spending another two years chasing a shadow.

The Path to Material Integrity

We need a way to bring the physical truth back into the room and we need to stop pretending that disclosure is the same thing as data. A researcher can disclose every penny they ever earned and it will not change the fact that their reagent is garbage.

That is the kind of material reform that actually fixes science because it removes the ghost variable before the clock starts. It is much harder than building a portal and it is much more useful than a six hour class on why you should not cook your numbers.

The board in that meeting did not want to hear about the vials. They wanted to talk about the dashboard and they wanted to talk about the new certificate that every student would have to hang on their wall. They liked the certificate because it was blue and it had a gold seal and it looked like progress.

I sat there with my hiccups and I thought about all the bread being made with sawdust and all the bakers being blamed for the flat loaves. We are building a cathedral of oversight on a foundation of sand and we are surprised when the walls crack.

I think we stay in this loop because it feels good to feel like we are doing something. If we admit that the inputs are the problem then we have to admit that we have no control over the global market and that we have been blaming the wrong people for a decade. It is much easier to say that researchers need more training than it is to say that the materials they buy are untrustworthy. Training is a line item and trust is a ghost. We choose the line item every time.

The Hiccup Cure

I stopped my hiccups by drinking water out of the wrong side of the cup while the audit man was still talking. It is a strange trick and it looks stupid but it works because it changes the way you move your throat. Science needs a trick like that right now.

It needs to stop looking at the person and start looking at the glass and it needs to stop thinking that a form can fix a vial. Until we start testing the dust we are just going to keep spending millions of dollars to watch the failure rate stay exactly where it is. We will have the best documented failures in the history of the world and we will have the most ethical researchers who ever produced a result that no one can repeat.

We will have the gold seals and the blue certificates and the portals and the dashboards and we will still be sitting in a room wondering why the bread is flat.

The Integrity Mandate

I told the man from the audit team that he should look at the supply logs and he just blinked at me. He said that the supply logs were not part of the integrity mandate and that his job was to ensure compliance with the reporting standards. He was right and that is the whole problem.

“We have defined integrity as the act of reporting and we have forgotten that integrity is also the quality of the thing being reported.”

We have decoupled the process from the product and we are now masters of the process. We can tell you exactly when a researcher logged in and exactly which box they checked and exactly how many hours they spent in my class but we cannot tell you if the powder in their hand was real.

We have traded the lab for the office and we are calling it progress and we are doing it with a straight face and a full heart. I walked out of that meeting and I bought a coffee and I thought about Mark and I wondered how many more people we are going to lose before we decide to look inside the vial. It is a long walk to the parking lot and it is a long way to the truth.