When reading a paper, how do you evaluate other peoples data? Do you look at an image and say, yes what you’re saying is obvious? Or do you look at the image and say, what you say appears obvious, but the quality of your image is not very good, so I don’t really trust it?
I am reading a paper that is quite important in my particular subfield. As I read the paper, I examined the microscopy images in detail as they were the first to demonstrate “colocalization” of my proteins of interest. IMHO, the images are not very good. As the authors used confocal microscopy, they either had too much ab, the detector pinhole set to > 1 airy disc or the pmt gain and offset settings were too high. They also stated that the 2 proteins colocalized without providing information on the voxel (a 3 dimensional pixel) size used to collect the images. Yes the 2 signals overlap, but is that because you used a 100 nm X 100 nm X 250 nm or because you used something greater than that? because if you used something bigger, I don’t care that signal from 2 proteins share the same space it doesn’t mean they interact. Heck signal from 2 proteins can occupy the same 100nm x 100 nm x 250 nm and that doesn’t mean they interact. In another set of images, as expected the images collected using GFP-tagged protein is much cleaner then images collected wheere an antibody was used to detect the protein. For those of you who don’t know, there will be additional noise with an antibody because of non-specific binding. It is for this reason, that is why you always take images of your cells, tissue, etc with no antibody (account for auto fluorescence), secondary only (non-specific ab binding) and / or an isotype control. I say and/or as I’ve heard legitimate arguments for an and against it – I just haven’t decided which side of the fence I sit on. These control images are used to remove “noise” or background (which should be stated in the methods section!!!)
As I explained my critique to a fellow lab member, they became somewhat defensive** and stated that when critiquing someone else’s data I should ask if it passes the bloody obvious question before slamming the quality of it. I don’t agree with this, because I know if MSc advisor had seen those images, he never would have let me publish them. Plus the images coming out of our lab do not look like that. Which brings me to my question for you dear readers? How do you evaluate data?
**although this individual was strongly annoyed with me (probably because they think I was being overly critical and they know how hard the first author worked), they were perfectly normal with me 10 minutes later. I love that we can be strongly opinionated on a subject and then be totally normal on the next topic.
Posts
10 months ago
5 comments:
I've only dabbled in confocal, so take this with a grain of salt.
I feel like "either they had too much ab, the detector pinhole set to > 1 airy disc or the pmt gain and offset settings were too high" type critiques boil down to "it's not pretty enough!" whereas "They also stated that the 2 proteins colocalized without providing information on the voxel (a 3 dimensional pixel) size used to collect the images" boils down to "I don't know if I can interpret these data".
I don't buy the notion that some seem to have that people who produce ugly data are automatically sloppy scientists and therefore their conclusions are suspect. You should at least give things a go with the antibody they were using before you draw that conclusion.
If they had one protein GFP tagged, why didn't they tag the other and FRET them? Or co-IP? Or provide some other parallel line of proof? I rarely see papers that just rely on confocal.
That said, I obviously prefer my data to be both pretty and interpretable.
I always pick apart people's figures like that. That's the point. You need to be able to believe the story they are telling according to the evidence they present--if they haven't done the right controls, or haven't performed the experiment in a way that legitimately allows their conclusion to be drawn, then than is essentially research misconduct no matter how hard they have worked on it and needs to be figured out. At least by me, so I use their interpretations appropriately when thinking about my own work.
If someone uses a technique I know well and I can tell they did a crappy job and pulled one over on a reviewer, they deserve to not be trusted (again, regardless of how hard they worked on it or whether they are higher on the totem pole than me). Your labmate needs to learn to be more critical him/herself, it makes a person a MUCH better scientist to not just trust whatever people say about their data.
And Becca, the problem with microscopy is that if it isn't pretty enough, that means you can't extract enough meaningful detail out of it and so it isn't interpretable. So, the reason to make pretty data is fundamentally so that people (including you) can actually answer questions with it, not just to be picky or make it look nice.
Becca - its not about the data being pretty. Especially with confocal data, if its pretty I want to know how they made it look pretty. Microscopy is very tricky because it is so easily manipulated to make it look the way you want. As some one who is highly trained in both the theoretical and practical microscopy, I am constantly frustrated with how little information is put into the methods with regards to how the image was both collected AND processed. This information is critical to being able to evaluate the data.
Oh and yes we have used those antibodies in our lab, which is why I know they should work better. Thats the other thing about Ab's you use to much and you get a ton of non-specific data...
Arlenna - I agree with you. The other issue, is that I was trained by individuals that had extremely high standards for microscopy practice. I would never have been able to submit images like that....maybe I am bitter.
Your labmate getting annoyed with you for critiquing someone else's data... is totally uncalled for. This is science. That is what we do.
I can't address confocal specifically having only attended a one day training class- and then never used it- I am certainly no expert. But in general, when looking at other types of data I always tend to be hyper critical. You need to know details, you need to have controls,and in this digital age it is so easy to alter images in subtle but significant ways that I tend to disbelieve until I see other supporting evidence. So... you were right.
Post a Comment