One of the most frustrating things when starting a career in academic research is getting your paper rejected.
Even with more experience, no one enjoys a rejection and we all prefer to see good news.
Quite often this happens on the first project you tackle. A new student might work for a year on a project, pouring in effort and passion and resulting in something that seems to have real merit ... only to be hit with a cold, hard rejection.
And then, with the computer science conference submission schedule, you have no opportunity to respond to the reviewers and you might have to wait six months or more for another appropriate chance to submit. Science is a slow process.
But there's one bit of silver lining: A paper's rejection doesn't mean your research is bad.
In fact, many or most of the papers you see in their final polished form in top venues went through rejections -- even the best papers. Here's a thought experiment. Among a set of published papers, some will have gotten in on the first try, some were rejected first, others were rejected multiple times. Ultimately, how impactful are the papers that got in on the first try, compared to the rejected ones?
To answer that let's (imperfectly) measure impact in terms of citations. Here are my own published research projects, showing the number of times the project received a rejection (X axis), and the number of citations per year (Y axis), as of a few weeks ago:
First you'll note that most of my projects have been rejected, either with a failed workshop, conference, or journal submission, before reaching successful publication later. And furthermore, among these published papers, there is apparently no correlation between whether the project has been rejected, and its eventual impact. In fact, if we were to draw a best fit line:
...then we see that my published projects have received 3.96 more citations per year, per rejection. Not bad considering the average number of citations per year here is 13.4. This is not a very robust method given the small sample and skewed distribution of citations. But a 2012 study by Calcagno et al of 923 journals similarly showed that rejected papers received "significantly" more citations than those accepted on the first submission.
This might be counterintuitive. Doesn't a rejection mean that the project is less exciting or less well executed, which certainly imply lower impact? Perhaps, but there are at least two other factors:
- A rejection can improve the quality of a project's next submission, due to feedback from peer review and time to improve the manuscript.
- Authors might judge, based on the rejection, to not bother resubmitting. These dead manuscripts dropped out of my sample.
Here's what I think this says: You should let your best judgement, not the reviewers' decision, guide your research. Certainly you should give careful consideration to how reviewers reacted to your paper, but don't automatically take a rejection as an indication of the quality of a project. If you still believe that this thing is a good thing, then you are probably right. It is a good thing, with at least as much impact potential as an immediately accepted paper.
OK ... now go have some ice cream.
Posts
