Comments Coar-Repositories

parallel to ESSOAR https://eartharxiv.org/ was launched

Another example: https://scipost.org/

In this analogy, though, the wide range of recording technologies, musical genres, etc. is collapsed into the modularity of the song-as-file. But, especially for more dynamic forms of digital scholarship, Pubfair’s affordances would presumably determine whether a work could be presented in a certain format or not.
A central question would seem to be: how customizable are these channels and the functionalities that comprise them, and with what level of expertise? (To say that it’s open-source and can therefore be customized belies the reality, well understood from OJS, that many organizations do not have the capacity to do this themselves or to hire it out.)

What role, if any, would publishing professionals play in defining these functionalities and the relationships between them?

How will the relative power of different stakeholders (e.g., big and small disciplines) be addressed in the design process?

 
Peer review of Pubfair framework:
Ross-Hellauer, T.; Fecher, B.; Shearer, K.; Rodrigues, E. (2019). Pubfair: a framework for sustainable, distributed, open science publishing. White paper, version 1: Sept. 3, 2019. Confederation of Open Access Repositories (COAR).m Retrieved September 23, 2019 from https://www.coar-repositories.org/news-media/inviting-community-input-pubfair/
by: Dr. Heather Morrison, sustainingknowledgecommons.org
Highlights
“Science” is only one type of knowledge. There are nine faculties at the University of Ottawa; only one is named “science”, and this is typical at a large university. I strongly recommend replacing “science”, “scientists” and “open science” with more inclusive terminology such as “open scholarship” or “open knowledge”, “scholar” or “researcher” in the title and throughout the document. The Pubfair framework is an excellent beginning for a needed profound transformation in how scholars work together and disseminate research. This is the kind of approach most likely to achieve significant savings based on current spend on scholarly publishing, and these savings will be needed to support innovation in scholarly production and dissemination. My recommendation is to proceed with an iterative approach and an initial focus on helping scholarly communities with unmet needs for new forms of review and publishing, such as scholars who create and share datasets or tools using artificial intelligence, digital humanists, and scholarly bloggers. The specific needs for community input whether through review or collaboration in the planning process will vary by discipline and type of product. The work of defining needs and identifying potential solutions should be led by the scholarly community in consultation with repository managers. This is a reversal of the proposed leadership / consultation approach in the framework document. Finally, while I recommend an immediate start to this approach, my advice is to see this as a long-term radical transformation that will likely take decades to complete.
Details
Scholarship includes, but is not limited to, science. I suggest changing the title and wording throughout the document to more inclusive terminology such as “open scholarship” or “open knowledge”. To illustrate why this matters: the University of Ottawa (uO) is an indirect membership of the Confederation of Open Access Repositories (COAR) through our membership in the Canadian Association of Research Libraries (CARL).  uO has 9 faculties: Arts, Education, Engineering, Health Sciences, Law, Medicine, Sciences, Social Sciences, and the Telfer School of Management. If this framework for “open science publishing services” is to become a reality, is it intended to serve only one of our 9 faculties? This seems unlikely. This conflation of “science” with “knowledge” or “scholarship” is not unique to this group but reflects a broader trend in the open movements. The problem is much larger than mere semantics, it reflects a tendency in our society to devalue types of knowledge other than science. I argue that this is a danger to our collective knowledge of all types, including science. For example, would it be wise to practice science without ethics or logic? Ethics and logic are branches of philosophy. Readers of this review will probably agree that governments should base policy on scientific evidence. However, politics per se is not science, and achieving and maintaining a goal of basing policy on scientific evidence requires understanding of history, political science, society, and communications. I discussed this in a recent conference presentation called knowledge as a human right (Morrison, 2019a).
The Pubfair conceptual model of building a framework to transform scholarly publishing building on a distributed network of repositories is a timely initiative and worthy of support. This is the kind of approach most likely to facilitate transformative transition in terms of both technology and economics. Houghton et al. (2009) conducted the most comprehensive study of the potential for transformation for a single country (the UK), comparing potential costs of 3 models: gold open access publishing, green open access archiving, and a third more transformative approach involving a new publishing system building peer review on top of archives. The transformative approach was calculated as having the potential to substantially reduce costs and was seen as the most cost-effective approach. However, at the time the UK did not think the country was ready for this transformation and opted for a focus on gold and maintenance of a pre-existing green system.
Much has changed in the past 10 years. The number of open access repositories listed in the vetted OpenDOAR list has grown from 1,419 on June 20, 2009 to 4,150 on June 30, 2019. OpenDOAR lists repositories on 5 continents. The largest metasearch service for repositories and open access journals is the Bielefeld Academic Search Engine (BASE). From 2009 to 2019 (June 30 each year) BASE grew from 1,730 content providers and just over 25 million documents searched to 7,211 content providers and just under 150 million items searched. (Morrison, 2019b).
Scholarly works and their dissemination appear to be undergoing a period of rapid transformation in a way that has not been addressed by traditional approaches to evaluating scholarship, the traditional publishing business, or even the open access movement. In the digital humanities, scholars are creating collections of electronic works and developing innovative means of searching, processing, and displaying material. Scholars in a wide range of disciplines from art to engineering are using artificial intelligence to create new knowledge and practical tools. The disciplines themselves are undergoing change with new forms of scholarship often overlapping what used to be separate disciplines. A few researchers, like me, are publishing open research using blogs and likely other formats and sharing open data; more would likely follow suit if they could be confident that they would receive appropriate recognition for doing so when it comes time for tenure and promotion.
Given this context, for practical reasons I recommend an iterative approach, beginning with scholars who are interested in exploring alternatives and motivated to do so because current approaches do not meet their needs. There may be common themes across disciplines and types of research, but it will also be important to recognize differences based on the type of work and the nature of the communities that would need to configure or re-configure to accomplish this work.
Four examples:
·      Peer review of open datasets might focus on quality and completeness of data and documentation, and/or adherence to relevant standards, reference to related work, and/or importance of the dataset. Qualified peer reviewers need some expertise in quantitative data and the relevant domain; comments from those who might benefit from results would be helpful as well. The ideal outcome might involve collaboration in the process of developing datasets rather than peer review after publication, to avoid duplication or fully benefit from triangulation from different approaches to the same underlying problem.
·      Peer review of a digital humanities dataset and portal for users might focus on the quality of metadata, quality and comprehensiveness of content, usability and accessibility of the users’ portal, preservation planning, interoperability with relevant databases, or how licensing for re-use has been addressed. Here, different aspects of review involve different types of expertise, from content subject knowledge to user experience to electronic preservation. The benefits of collaboration in the planning process appear obvious.
·      Peer review of tools developed through AI to support the work of health professionals and/or to help patients monitor their own conditions may require triangulation using other methods to ensure accuracy of results, user experience analysis of the tools and/or periodic evaluation of the ongoing accuracy of AI assuming ongoing machine learning.
·      Peer review of scholarly and research blogs such as Retraction Watch, my scholarly blog The Imaginary Journal of Poetic Economics, and my research blog sustainingknowledgecommons.org might focus on the accuracy, originality, and/or importance of the contributions.  In this case, review by subject experts is essential, and technical advice, which may be provided by different reviewers, is useful. Individual authors or groups of authors may or may not see collaboration in the planning process as useful. As a researcher-blogger, I can see situations where different types of blogs and authors would benefit from different types of review.
These examples are just a few of many possible new types of scholarship made possible by the digital environment. The optimal form of review and publishing such new types of works is, at present, unknown. This is another reason to seek an iterative approach and look for leadership within the communities of scholars pursuing these approaches. To understand what kind of review is most helpful for the community, it is necessary to understand in depth the nature of the research and/or creative works that are being developed.
Finally, this transition is a major cultural shift in how academics might work in future. It will take time to figure out the questions that will arise in the process, and more time to develop solutions. In summary, while I see this as a long-term transition, an approach along the lines of Pubfair is the right direction and steps should be taken to move in this direction as soon as possible.
Thank you for providing the opportunity to comment and best wishes for Pubfair.
About me
I write as a researcher focused on the transition of scholarly communication from the demand (subscriptions / purchase) to the supply side to support a global open access knowledge commons. My research project, funded by Canada’s Social Sciences and Humanities Research Council from 2014 – 2021, is called Sustaining the Knowledge Commons. My comments will be posted in the uO institutional repository and cross-posted to my open research blog, sustainingknowledgecommons.org.

References
Houghton, J.; Rasmussen, B.; Sheehan, P.; Oppenheim, C.; Morris, A.; Creaser, C.; Greenwood, H.; Summers, M. & Gourlay, A. (2009). Economics implications of alternative scholarly publishing models: Exploring the costs and benefit. A report to the Joint Information Systems Committee. UK. Retrieved July 11, 2019 from:
https://pdfs.semanticscholar.org/45dd/cb9ebb9c8505a4ac86718734dda3311f91d8.pdf
Morrison, H. (2019a). Knowledge as a human right. Presentation Jan. 30, 2019, University of Ottawa, cc-UNESCO Science as a human right series. http://hdl.handle.net/10393/38890
Morrison, H. (2019b). The Dramatic Growth of Open Access. June 30, 2019 full dataset. https://hdl.handle.net/10864/10660
September 24, 2019.

Cross-posted: cite as:

Morrison, H. (2019). Peer review of Pubfair framework. Sustaining the Knowledge Commons. https://sustainingknowledgecommons.org/2019/09/24/peer-review-of-pubfair-framework/

The last sentence (starting with “By integrating and adding…”) perhaps needs to be revisited. 

Perhaps you could clarify what you mean by “more research-centric”?

diverse research objects

There are a lot of dependencies at the repository side, which Pubfair is right to highlight. One probable dependency will be CRIS systems. These are components of the scholarly communications lifecycle which have increasingly come under the oligopolistic control of large publishers and therefore frequently sit outside the repository landscape in terms of support for standards, interoperability, etc. This also means it is more difficult for COAR to profile requirements in these instances, or even influence system developments.
The problem to be solved by Pubfair is avoiding the disenfranchisement of institutions which:

Have retired their repository in favour of a combined CRIS/quasi-repository solution (I consider this a terrible situation for an institution to have found itself in, but there are many out there in precisely this position, esp. in the UK!). 🙁
Institutions which operate a repository and CRIS within a connected configuration.

Scenario #1 is clearly the bigger problem, and it might be necessary to take a dogmatic approach here. For example, I think it will be highly unlikely CRIS vendors will support the philosophical basis for Pubfair let alone the standards and technology necessary to make it work. It may therefore make sense to exclude them from the outset entirely rather than waste time trying to persuade vendors, all on the basis that, a) their systems are not open and are therefore inconsistent with the Pubfair ethos, b) will not support the technical requirements, and c) the institutions which retired their repositories shouldn’t have been so stupid in the first place. This may force those institutions to revisit their approach to repository systems thereby bringing them within the Pubfair fold. But it may also have the opposite effect. Are we comfortable with this?
Scenario #2 is what exists at my own institution (University of Strathclyde). We operate a repository which is the focus for dissemination, curation and preservation, but which is fed content from our connected CRIS. The fact that this connection exists immediately creates limitations on the extent to which certain repositories can participate meaningfully in the Pubfair repository layer. See for example, Repository and CRIS interoperability issues within a ‘connector lite’ environment. As someone who oversees repository work at our institution, I can confirm that we would do whatever it takes to ensure Pubfair participation. Initiatives like this are in our DNA! But it is important to note that there may be aspects of Pubfair which become impossible to satisfy while CRIS systems are feeding repositories. 
I suppose the comment I am making here is that there are quite a lot of ‘known unknowns’ surrounding the capacity for nodes within the repository layer to contribute meaningfully, especially when – as you rightly note – repositories are sitting at different levels of maturity but also may be contending with the issues highlighted in #1 and #2 above.
Overall, this is excellent work. Keep it up!