Re: an interesting dilemma...
Posted: April 30th, 2011, 10:35 am
The saga continues. I had to redo the data analysis yet again because Pushy PI wasn't pleased with the answer I came up with. My boss and I came up with a very solid way to handle the data interpretation, and my final answer (obtained a couple of days ago) is somewhere between the minion's answer and the one I came up with the first time. The Pushy PI is ticked because I'm not including two pieces of data that are questionable from my perspective but give a necessary answer from their perspective. To put it in layman's terms as best I can: I know that the instrument's measurements were off from known values by a certain amount (10 ppm, let's say). I also know the standard deviation (how close together all of the measurements are). I'm including everything within 2x standard deviation from the mean, which is perfectly reasonable. Again, just to throw in some imaginary numbers, that means that I'm including everything from 2 ppm to 18 ppm from the theoretical values. Now, their data point falls at -4 ppm. My answer is that this is clearly wrong. Their answer is that I'm being too stringent and it hurts their paper. Fortunately, I THINK my boss is with me on the fact that we have no justification whatsoever for including this particular data point. In fact, I spent the majority of the day yesterday trying to find a way to include it. Every time the answer was "no."
We got to this point in the first place because The Minion has historically presented data to her boss as correct that I have told her as incorrect, and then she presents the data in our meetings with my boss as being correct. Every time I try to speak up in meetings to say this I've been shut down, and I'm sick and tired of it. The Minion had taken the two bad pieces of data to Pushy PI and told him I was being mean and wouldn't include them. Fortunately I knew what she wanted to do and came to the meeting prepared, for once. How did I know? Well, she happened to send an e-mail message to her boss and my boss (WITHOUT copying me) about some data that she wanted to show them. I happened to see it because she sent the message while working on a computer in our lab. And, since I had told her about 5X the previous day why we could include the data she wanted to include, I knew exactly what she was planning to present. So, for once, I beat her to the punch. Before The Minion could present the "good" data that I didn't want to include, I went through the justification for including what I did and showed why the "bad" data couldn't be included. Amazingly, at least in the meeting, the boss backed me. I had already gone over my side of the argument with the boss, so he knew what we were up against. That probably helped.
The issue involves an understanding of precision vs accuracy. Pushy PI and his minion are saying that the data point is good because it's so close to the theoretical value for their unknown sample. My stand is that, although this is true, we know from measurements of KNOWN samples that the instrument's accuracy was not at 0 ppm - it was at 10 ppm (precise, but not accurate). The problem is, they're treating their unknown sample as a known sample, and it's not. In my mind it's like traveling with the flow of traffic at 70 mph and saying you're only going 60 mph because that's what your broken speedometer says.
We got to this point in the first place because The Minion has historically presented data to her boss as correct that I have told her as incorrect, and then she presents the data in our meetings with my boss as being correct. Every time I try to speak up in meetings to say this I've been shut down, and I'm sick and tired of it. The Minion had taken the two bad pieces of data to Pushy PI and told him I was being mean and wouldn't include them. Fortunately I knew what she wanted to do and came to the meeting prepared, for once. How did I know? Well, she happened to send an e-mail message to her boss and my boss (WITHOUT copying me) about some data that she wanted to show them. I happened to see it because she sent the message while working on a computer in our lab. And, since I had told her about 5X the previous day why we could include the data she wanted to include, I knew exactly what she was planning to present. So, for once, I beat her to the punch. Before The Minion could present the "good" data that I didn't want to include, I went through the justification for including what I did and showed why the "bad" data couldn't be included. Amazingly, at least in the meeting, the boss backed me. I had already gone over my side of the argument with the boss, so he knew what we were up against. That probably helped.
The issue involves an understanding of precision vs accuracy. Pushy PI and his minion are saying that the data point is good because it's so close to the theoretical value for their unknown sample. My stand is that, although this is true, we know from measurements of KNOWN samples that the instrument's accuracy was not at 0 ppm - it was at 10 ppm (precise, but not accurate). The problem is, they're treating their unknown sample as a known sample, and it's not. In my mind it's like traveling with the flow of traffic at 70 mph and saying you're only going 60 mph because that's what your broken speedometer says.