The Center for the Study for Higher Education at the University of California, Berkeley, has published a new report on the role of peer review in academic tenure and review and in scholarly publishing.
Report citation and link: Harley, Diane, & Krzys Acord, Sophia. (2011). Peer Review in Academic Promotion and Publishing: Its Meaning, Locus, and Future. UC Berkeley: Center for Studies in Higher Education. Retrieved from: http://escholarship.org/uc/item/1xv148c8
Announcement:
Since 2005, and with generous funding from the Andrew W. Mellon Foundation, the Center for Studies in Higher Education (CSHE) has been conducting research to explore how academic values – including those related to peer review, publishing, sharing, and collaboration -influence scholarly communication practices and engagement with new technological affordances, open access publishing, and the public good.
This report includes (1) an overview of the state of peer review in the Academy at large, (2) a set of recommendations for moving forward, (3) a proposed research agenda to examine in depth the effects of academic status-seeking on the entire academic enterprise, (4) proceedings from the workshop on the four topics noted above, and (5) four substantial and broadly conceived background papers on the workshop topics, with associated literature reviews.
The document explores, in particular, the tightly intertwined phenomena of peer review in publication and academic promotion, the values and associated costs to the Academy of the current system, experimental forms of peer review in various disciplinary areas, the effects of scholarly practices on the publishing system, and the possibilities and real costs of creating alternative loci for peer review and publishing that link scholarly societies, libraries, institutional repositories, and university presses. We also explore the motivations and ingredients of successful open access resolutions that are directed at peer-reviewed article-length material. In doing so, this report suggests that creating a wider array of institutionally acceptable and cost-effective alternatives to peer reviewing and publishing scholarly work could maintain the quality of academic peer review, support greater research productivity, reduce the explosive growth of low-quality publications, increase the purchasing power of cash-strapped libraries, better support the free flow and preservation of ideas, and relieve the burden on overtaxed faculty of conducting too much peer review.
This latest report on the state and future of peer review is a natural extension of our findings in Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines (2010), which stressed the need for a more nuanced academic reward system that is less dependent on citation metrics, the slavish adherence to marquee journals and university presses, and the growing tendency of institutions to outsource assessment of scholarship to such proxies as default promotion criteria.
Links to the complete results of our ongoing work can be found at The Future of Scholarly Communication Project website.
2011-03-18
My favorite section thus far comes from Donald Waters, pages 63-4:
We have talked extensively in this meeting about the publication of journals and monographs. Another way to explore the efficacy of publication-based peer review, however, is to consider its application in an altogether new area of activity. One of the hottest emergent areas of scholarly communication is the growth of mechanisms to curate primary source data, particularly those in digital form, on which research and teaching depends in particular fields of study in the sciences, social sciences, and humanities. Along with curation, a related area of growth is in emergent means of publishing those data or subsets that are relevant to particular work. Few fields have worked out mechanisms for the evaluation of the quality, integrity, format, relevance, and importance of primary source curation and publication in digital form. Peer review is much needed as an organizational function because fields that depend on digital primary sources will prosper or suffer to the extent that experts in the field collectively develop appropriate standards and incentives for the proper handling and dissemination of that material.
Let’s focus for the moment on the case of the humanities, where the development of primary source material as a basis for research and teaching has a very long tradition in philology, editorial practice, and edition-making. Over the last 25 years, just as the transition from print to digital began, the practice of primary source curation and publication has been deeply undervalued professionally within many fields. This trend has produced perverse results, including the widening cultural gap between scholars and librarians. In addition, publishers have been engaged in a kind of “land grab” for digitized primary sources. High prices for digital access have created digital divides between the “haves” and “have-nots,” and this is increasingly worrisome in fields like medieval and early modern studies. Moreover, the quality of online materials is increasingly questioned as scholars become more interested and better trained in digital analysis methods. The Burney Collection at the British Library, for example, contains rare and important 18th century newspapers and periodicals. It has always been an important source for characterizing the early emergence of new print genres. Now that that this and other collections are being digitized and the use of optical character recognition (OCR) has made full-text searching possible, it is increasingly important to ask how representative of the total universe of early printed materials the Burney collection is. The answer to this critical question requires standard bibliographies of the period to be properly linked to the digitized materials. The poor quality of OCR on early printed material and analysis using computational techniques also require much more accurate transcriptions. The processes of identification and cataloging, and of structuring the primary sources in forms usable for scholarship, are age-old “curatorial” activities, but they need to be examined and applied anew in the digital age.
The curation of primary sources in digital form represents a new genre of scholarly communications activity across a whole range of disciplines: astronomical sky surveys, genomics/proteomics databases, architectural history databases, letters and papers of primary authors, papyri, and so on. All of these data are being converted to or generated in digital form and then organized as scholarly projects to propel research and teaching forward for the next generation. There is so much innovation and experimentation that it is difficult to classify this new genre. Some of these projects exhibit edition-like properties; they look like published editions, defining primary sources with contextual essays and other scholarly apparati. Some are book-like or journal-like because they produce scholarly material about a subject. Many are lab-like because they require an elaborate division of labor, with specialists of various kinds responsible for a variety of different tasks including design, technology development, and execution. Indeed, the assembly of the material and contextual apparatus can often be divided up so that students in an undergraduate classroom, or even the broader public outside the Academy, can contribute effectively to the work. Good examples of engaging the broader public include the kinds of amateur involvement that you see in astronomy, ornithology, and, more recently, geography, where the public is identifying photographs, individuals in photographs, and where photographs are located on a map.