I’ve posted previously on our rush effort to get this paper out as quickly as possible, it was only about a week between starting writing and putting it up on a pre-print server. We then submitted it to a journal, mSystems, where I expected that it would undergo rushed pandemic-related peer review. Except that it didn’t, it was editor-approved with no modifications. I felt uncomfortable about this, but really wanted to get the paper out there and have it be useful.
And then the paper came out and the comments started to come in. In particular, we had mis-represented filtration efficiencies in relation to viral particles. Specifically we got a lot of pushback from folks in The American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) which is the organization that sets the standards for things like filter requirements in buildings. They were correct, we addressed those concerns and submitted the corrections linked above. Relatively painless, but shouldn’t have been necessary.
So firstly, if this all happened again I would not accept “editor triage” as sufficient for a publication like this, I would want to insist on peer review. Peer review would have fixed these problems before publication.
But more broadly, I see a lot of calls on Twitter that “peer review is broken” and that the whole system needs to be swept away. Of course peer review is not perfect, there are all sorts of problems with the current system. But it’s like democracy… the least bad of all systems. Some people argue that everything should just be put out there and let the crowds sort it out (as happened here). Except that doesn’t really work… the hundreds of downloads of our original article will forever contain sub-optimal information that could have been fixed by peer review. And fixing things after the fact takes more energy and effort than fixing them before.
So yeah, I’m still a fan of the current system (though I’d be happy to see any number of changes!)