A longitudinal examination of SIGITE conference submission data, 2007-2012

This paper examines submission data for the SIGITE conference between the years 2007-2012 with an emphasis on the reliability and validity of the peer review process. Despite the centrality of peer review to academic endeavor, it is not easy to assess it due to the confidentiality of blind reviewing systems. This paper provides a unique addition to the study of peer review by examining reviews and submissions for a single international computing conference across an extended time period. It examines which external factors (such as nationality and familiarity of the reviewer) and which internal characteristics of the submissions (such as word length, number of references, and readability measures) are related to eventual reviewer ratings. Ramifications of the findings for future authors and conference organizers are also discussed.