ASSETS 2014 – Rochester, NY.

ASSETS Reviewing is now all finished,  and the interviewer discussions have been completed. This year as always there were many good papers in my batch of seven for review, and I imagine even some good ones will not make it through to the conference due to the limitations of the conference format. There also seemed to be some part works and some works which had not quite done enough due diligence into the background of the topic they were investigating. It’s something that comes up quite a lot, the more reviewing you do, but even so it shows a lack of maturity within the field. By this I mean there seems to be a high degree of churn of people entering the field and exiting the field quite quickly without spending years picking up a deep knowledge of the accessibility backgrounds which can only come through study and experience.

Indeed, this is a experience seems to also exist in the review committee, and in the papers I was reviewing there were a number of very short reviews, often very positive but with a very limited knowledge of the background to work that already had been done. It is quite difficult to bring up the quality of reviews in my opinion without a significant overhaul in the mindset of the reviewers. This is why I believe in open reviews because they remove the ability of a reviewer to be unconstructive, terse, incorrect in their review, or just plain vindictive. When you realise that the authors at the other end will know who you are and where you work you are far more circumspect about the quality of the review you create, the marks you give, and the tone with which you give them. In reality this means you become a more constructive reviewer and then the review process for everyone is a productive activity even if that paper ends up being rejected. This is why any paper which I have reviewed for assets will have a review statement on the bottom (as below).

I am tired of receiving unfair, unconsidered, short, or unhelpful reviews – I personally do not mind the rejection but I do mind not being able to make my next submission better. It seems to me that having reviewer information obscured may be a partial explanation for these kinds of reviews and this may not be the best way to conduct scientific research in the future. Therefore, in the interests of transparency, this paper was reviewed by Dr Simon Harper at the University of Manchester (UK). My objective was to give honest feedback to make your work better and my reviews more considered, I did not wish to say anything here which I would not say in a face-to-face discussion. Further, unlike many reviewers who are attempting to use the peer review process to determine whether or not an article reaches the level of ‘importance’ required by a given venue, I only review to determine whether a paper is technically sound and worthy of inclusion in the published scientific record. I think that once the work is published, the broader community is then able to discuss and evaluate the significance of the article (through the number of citations it attracts; the downloads it achieves; the media and blog coverage it receives; and the post-publication Notes, Comments and Ratings that it receives). Finally, if you feel this review was either unfair or unhelpful then please let me know, and I will endeavour to do a better job next time.

While it probably is not possible for a reviewer to completely be secure or that the authors of the paper for which they have reviewed will take their review in good part and see it as a constructive mechanism to make their work better it is view we should all strive for. There is no limit on the vindictiveness of an author spawned, indeed this is why anonymous reviewing was first created, however it is time for us to grow up and not hide behind anonymity. I can only hope that if I have rejected your paper then you view my comments as constructive criticism as they were intended and not as a personal attack on either the work or the researchers involved. If you really weren’t happy with my review come see me at the conference!