5. Use Cases

Most of the following use cases offer variations on a clear pattern, and this pattern can be supported by one or more of the notification examples outlined previously in this document.

Use Case #1: Push manuscript from repository to overlay platform

  1. An author has deposited a manuscript in publication repository PR and requests that the manuscript be reviewed by overlay platform OP.
  2. PR sends a “review” request to OP (with URI to manuscript and MD)
  3. OP acknowledge that it will carry out the Review
  4. OP notifies PR (which in turn may notify author!) that the review has been done with results and (possibly) URIs to review (and MD).
  5. Iteration
    1. Author uploads a new version to PR and confirms it can be sent to OP
    2. PR notifies OP that a new version of the manuscript is available as reply to the review
    3. OP notifies feedback of the review process
  6. PR updates the MD of the manuscript if the paper has been accepted.
  7. In the case of a rejection PR keeps a trace so that the author does not submit the same version of the manuscript to the OP.

Use Case #2: Fetch manuscript on repository from overlay platform

  1. An author is requesting, on the overlay platform OP, that a manuscript available from a publication repository PR be reviewed by a “journal” hosted on OP.
  2. OP sends a request to access the manuscript from PR (by Id).
  3. PR provides a URI to the manuscript and keeps a trace of the request (to avoid double submission on the same platform or more control depending on the edition)
  4. OP notifies author and possibly PR that the review has been done with results and (possibly) URIs to review (and MD).
  5. Iteration
    1. OP notifies that a new version of the manuscript is available to be uploaded as reply to the review from author (on the OP platform)
    2. OP provides feedback to the author (and possibly the PR) of the review process
  6. PR updates the MD of the manuscript if the paper has been accepted.

Use case #3: Submit manuscript to a journal (society-overlay journal, qualification platform, etc) and use repository for the archiving of manuscript versions and full-text reviews

This use case is based on the OPRM Module that was developed by DIGITAL.CSIC, Open Scholar and other partners within a [project] funded by OpenAIRE in 2016 to pilot the implementation of open peer review services on top of DSPACE repositories. Our use case is based on the current architecture of the OPRM module that was designed to serve the institutional community of the Spanish National Research Council (CSIC). This new use case is slightly different since it involves a collaboration between DIGITAL.CSIC and an external, non-institutional journal.

This use case rests on an agreement between DIGITAL.CSIC, Open Scholar and Psicologica journal and it is expected that as new technologies emerge to standardize workflows of overlay journals, more institutional repositories will be able to participate as manuscripts providers and final hosts of the published manuscripts and associated reviews. This will make it possible that repositories, instead of journals, become the entry points for preprint manuscripts.

  1. An author submits a manuscript to the Journal (J).
  2. The Journal sends the manuscript to DIGITAL.CSIC (DC).
  3. DC publishes an item with the manuscript’s metadata but keeps the full text invisible.
  4. The J sends invitations with a link to the manuscript’s metadata and the full text file to potential reviewers using the OPRM available at DC.
  5. A reviewer sends a review using the OPRM available at DC.
  6. The review is published at DC as a new item linked to the manuscript’s item
  7. The J notifies the author who can submit a new version of the manuscript and start a new iteration of the process.
  8. When a manuscript is accepted by the J the full text of all versions becomes visible at DC and is assigned a DOI. Associated reviews also receive DOIs.
  9. The J publishes a summary of the manuscript on its website with a link to the manuscript’s instance at DC.

Use Case #4: Submission from overlay into repository

  1. Author Submit manuscript on Overlay Platform
  2. Author selects open repository or use overlay platform preferred open repository
  3. Overlay platform pushes manuscript to Open repository
    1. Open repository notifies the author and OP the article is submitted, maybe pending moderation.
    2. Open Repository notifies Overlay platform and author when the article is online and ready for reviewing, feedback includes Open repository handle or DOI
    3. Overlay platform and author are notified if Open repository refuses to put manuscript online.
  4. On Overlay platform reviewing starts if manuscript is put online
    1. Open repository is notified Manuscript is being reviewed
  5. Optionally reviews, and author answer to reviewers are made available on the Publication repository
    1. notifications to reviewers, authors, editors when new reviews and review answers are made available.
  6. Author submits new versions either on the overlay platform or the Open repository
  7. Overlay platform accepts manuscript
    1. notifies Open repository manuscript is accepted
  8. Overlay platform publishes manuscripts and pushes the last accepted version on Open repository.
  9. The Overlay platform sends its own handle or DOI, and also pushes metadata updates on the open repository

Use case #5: Highly distributed research team reviewing a range of research outputs

Description

The OCSDNet initiative was composed of a highly distributed network of researchers in different countries across the world working in 12 different project teams that often included both university researchers and community members. The projects produced a range of research outputs/artefacts (e.g. blogs, technical reports, field notes, videos, sound recordings) that the group wanted to share among members of the full network during the research process, and preserve so that they could be considered as relevant contributions when the project is evaluated and further developed. The OCSD Network also wanted to maintain a record of discussions over the course of the initiative so they could return to these original ideas over the course of the project. These records would be beneficial to the funders: IDRC in Canada and DFID in the UK. As the initiative came to completion, the OCSD Network wanted to produce and publish a metasynthesis of project outcomes. Each project team produced a chapter that would be part of a compilation. The first draft of each chapter was reviewed by two other team members and the OCSD Network advisor. Based on the feedback from initial reviews, authors revised their drafts, and then they were reviewed by an external expert. While not necessarily made public, the OCSDNet wanted to have a copy of reviews available to the research team for future reference. (The review process was managed using primarily Google doc and the rich exchanges were not preserved) After the drafts were revised based on comments from external reviewers, the drafts were then copy-edited and published open access.

Use case #6: Pre-journal peer-review user stories.

Workflow

Variant 1: submission of a preprint

  1. As an Author, I want to notify Review Commons about my preprint so that Review Commons can potentially review it.
  2. As a Review Commons Staff, I want to be notified of new submissions so that I can assign appropriate staff to process the manuscript and make an initial decision on whether to send the manuscript for peer-review or not.

Variant 2: direct submission

  1. As an Author, I want to submit a manuscript directly to Review Commons so that it can be considered before it is public.
    Common trunk:
  2. As a Review Commons Staff, I want to notify authors on whether the preprint/manuscript will be reviewed or not
  3. As a Review Commons Staff, I want to have access to the metadata and files of the manuscript so that I can ask authors for corrections or fill in missing metadata fields before exposing the study to reviewers.
  4. As a Review Commons Staff, I want to notify authors and confidentially show them the reviews so that they can start revisions and/or draft a reply.
  5. As an Author, I want to notify Review Commons to publicly display the reviews so that they can be freely accessed by interested readers and linked from other resources.
  6. As an Author, I want to provide a reply to the reviews so that my arguments are always displayed alongside the reviews.
  7. As an Author, I want to make a new version of the manuscript visible so that readers can see how I addressed reviewers’ comments in a revision.
  8. As a Reader, I want to see the reviews and authors’ reply associated with a preprint so that I can easily read their content, share them or comment on them.
  9. As a Reader, I want to know if any prior version of a preprint has been reviewed so that
  10. As a Reader, I want to know which service has obtained the reviews so that I can know how the peer-review process was run.

Use case #7: Author deposits a manuscript in publication repository

Workflow (simple version)

  1. Clicks button in PR to submit to OP for recommendation/endorsement after peer-review.
  2. Iteration (Nothing displayed on PR nor OP):
    1. Reviews ask the author for a new version
    2. Author deposits a new version
  3. If reviews fail to consider preprint to be worth being recommended/endorsed, then end
  4. If reviews result in a recommendation/endorsement decision:
    1. OP publishes its recommendation/endorsement and assigns PID to it
    2. OP “notifies” PR “Preprint has been reviewed and recommended/endorsed by OP, and this is available at PID”

Workflow (intermediate version)

  1. Author can click on button in PR to directly submit its MS to overlay platform (OP) for peer-review evaluation and recommendation/endorsement.
  2. If OP accepts to undertake review, it asks PR to indicate “this MS is currently being peer-reviewed by OP”.
  3. Iteration (Nothing displayed on PR nor OP):
    1. Following peer-review, OP can ask the author to submit a new version of their MS.
      1. Author deposits a new version.
  4. If OP considers the MS not being worth recommended/endorsed:
    1. OP requires PR to remove the sentence “the MS is currently being reviewed by OP”.
  5. If OP considers the MS worth being recommended/endorsed:
    1. OP publishes a recommendation/endorsement text of this MS and assigns PID to it
      1. OP “notifies” PR “MS has been peer-reviewed and recommended/endorsed by OP, and this is available at PID”

Workflow (more complex/complete version)

  1. Clicks button for PR to “notify” OP “request for review”
  2. Or alternatively she asks overlay platform OP to perform reviews in view of recommendation/endorsement
  3. OP retrieves preprint from PR.
  4. If OP accepts to undertake review:
    1. it “notifies” PR “preprint is currently being reviewed”
      1. (PR asks author if she accepts this information be displayed in PR and behaves accordingly)
  5. Else tells the author and “notifies” PR “refuse review”
  6. Iteration (Nothing displayed on PR nor OP):
    1. Reviews ask the author for a new version.
      1. OP “notifies” PR “a new version is requested”
      2. Author deposits a new version.
      3. PR “notifies” OP “a new version is available” (include version numbering in a way or another)
  7. If reviews fail to consider preprint to be worth being recommended/endorsed:
    1. OP “notifies” PR “preprint is not anymore under review”
  8. If reviews result in a recommendation/endorsement decision
    1. OP publishes its recommendation/endorsement and assigns PID to it
      1. OP “notifies” PR “Preprint has been reviewed and recommended/endorsed by OP, and this is available at PID”

Use case #8: Publishing articles, data, methods, software, etc. and connecting to different kinds of repositories

SERVICE PROVIDERS

NameURLIDs
University Journalshttps://universityjournals.eu/Preferable DOI
IR (but can also be another trusted repository)http://dare.uva.nl/Handle = handle provided by CRIS-Pure)
Data repository Figsharehttps://uvaauas.figshare.com/DOI
CRIS-Purehttp://pure.uva.nl/adminHandle

Description

University-driven journal-like publishing-as-a-service with multiple options for peer-review and quality assessments.

  1. The author can submit directly to UJ, or use an IR or preprint server.
  2. UJ provides a workflow and management platform (I.e. OJS) to guide submissions and publish content.
  3. Quality is ensured through a tiered workflow. Baseline quality assessment is mandatory and managed by the institute itself, further review is optional. Traditional peer-review, if any, is handled by external reviewers.
  4. All quality assessment and review workflows are clearly documented and transparent. Peer-review reports and version control are available.
  5. The quality assessment status of a manuscript is communicated by the platform through badges or labels. These badges follow international standards and are integrated into metadata.
  6. Production and publication follows standards for content and metadata and always includes a DOI
  7. Distribution at UJ does not only rely on scraping/harvesting, published content is actively distributed to abstracting and indexing services.

Variant 1 submission to University Journals (UJs) via repository

As an Author, I want to notify University Journals about my publication in the Institutional repository or in Figshare to potentially review it and consider it for publication in UJs.

As a Scholarly Quality Assurance Officer (SQAO) of University Journals, I want to be notified of new submissions to decide on whether the manuscript can be published (as is), should be revised or has been declined.

As an Administrative Quality Assurance Officer (AQAO) of University Journals, I want to be notified of new accepted publication and decide if all administrative requirements of the journal are fulfilled or the author should be requested to supply extra details (orcids, funder statement, etc.)

As a reader of University Journals I want to know what quality assurance has been provided.

Workflow 1

1.     An author has deposited a manuscript in an institutional repository.

2. Author selects from a list an SQAO of the university to consider the manuscript (for publication, for internal review, for external review, etc.)

3.     Author pushes a review request to selected SQAO of UJs of his/her University (with URI of the manuscript, metadata (type, authors, affiliations, email addresses, title, abstract, date of submission)

4.     SQAO accepts or declines the request and informs the repository and the author about the decision.

a.     After acceptances to consider the manuscript, the SQAO informs the repository and the author about the decision: (i) rejected (repository only), (ii) accepted with revisions , (iii) push it for further review or (iv) accepted as is.

                                               i.     Push from UJs to repository and author: rejected The rejection should be registered in the repository and can be shown in the repository?

                                              ii.     Push from UJs to other person for further review.

                                            iii.     Push from UJs to repository and author accepted with revisions

1.     Author has revised manuscript

2.     Author uploads new version to repository

3.     Author notifies the SQAO of UJs via the repository of the revised version.

4.     Iteration of [4a] the academic reviewer informs the repository and the author about the decision: rejected (repository only), accepted with revisions or accepted as is

                                            iv.     Push from SQAO of UJs to AQAO of UJs that the publication has been accepted as is.

The administrative reviewer is responsible for the administrative quality assurance (completeness of the submission: orcid, abstract, licensing, data deposit etc.)

1.     AQAO accepts or declines

a.     AQAO declines and requests the repository / SQAO / author for more information

i.     Repository / SQAO / author provides information

 ii.     Iterance 4.a.iii.1

b.     AQAO accepts

i.     UJs pulls copy of the file from the repository

 ii.     UJs provides new URI

iii.     UJs publishes the accepted publication (incl. URI of the repository record (for previous version)
[may be by providing front matter page?]

c.      UJs pushes to repository and author acceptance and updates the metadata of the publication

d.     Repository updates metadata when the paper has been accepted (pulls URI and published version).

5.     Comments or full reviews can be given by readers at UJs

6.     Commentaries are pushed to the AQAO

7.     AQAO declines or accepts comments.

NB comments can be based on DOIs (decentral) and not be available on the UJ systems

8.     In some cases the AQAO might want to consult others (authors, peers, etc.)

9.     The comments can be rejected by the AQAO.

10.  The comments can be accepted by the AQAO and pushed to the author and SQAO.

11.  Author can push a revised version.

12.  As a reader I want to know what kind of quality assurances have been done by UJs.

13.  As a reader I want to know what kind of other quality assurances have been done by others.

14.  As an institute I want to archive content as well as review reports, decisions and timelines, following international metadata standards to ensure portability.

Variant 2 direct submission to University Journals (UJs)

As an Author, I want to submit my publication to University Journals to consider it for publication in UJs

As an SQAO of University Journals, I want to be notified of new submissions to decide on whether the manuscript can be published (as is), should be revised, or be rejected, or forwarded for further review (internal, or external).

As an AQAO of University Journals, I want to be notified of new accepted publication and decide if all administrative requirements of the journal are fulfilled or the author should be requested to supply extra details (orcids, funder statement, etc.)

As a reader of University Journals I want to know what quality assurance has been provided.

Workflow 2

1.     An author has registered and uploaded a manuscript directly to UJs

2.     Author selects from a list an SQAO who considers the manuscript

3.     Author pushes a review request to selected SQAO of UJs at his/her University (without URI of the manuscript, metadata (type, authors, affiliations, email addresses, title, abstract, date of submission)

4.     The SQAO will be notified about the manuscript

5.     SQAO accepts or declines the request and informs the author about the decision.

a.     After acceptance (to consider the manuscript) the SQAO needs access to the manuscript and metadata.

b.     After review (by SQAO, or someone else, notified by SQAO) the SQAO informs the author about the decision: (i) rejected (not published), (ii) accepted with revisions or (iii) accepted as is.

 i.     Push from UJs to author rejected.

1.     Manuscript and decision are archived.

ii.     Push from UJs to author accepted with revisions

1.     Author has revised manuscript

2.     Author uploads new version to UJs

3.     UJs notifies the AQAO of UJs about the revised version.

4.     Iteration of 5b] the SQAO informs the author about the decision: rejected (archived only), accepted with revisions or accepted as is.

iii.     Push from AQAO of UJs to administrative reviewer of UJs that the publication has been accepted as is.
The AQAO is responsible for the administrative quality assurance  (completeness of the submission: orcid, abstract, licensing, data deposit etc.)

1.     AQAO accepts or declines

a.     AQAO declines and requests the author for more information

i.     Author provides information

ii.     Iterance 5.a.iii.1

b.     AQAO accepts

 i.     UJs pulls copy of the file from the UJ submission system

ii.     UJs provides new URI

iii.     UJs publishes the accepted publication

[may be by providing front matter page?]

and if applicable with or with link to review comments

iv.     UJs pushes the final version to external partners for abstracting and indexing

c.      UJs pushes to author acceptance and updates the metadata of the publication

6.     Readers can write commentaries after publication

7.     Commentaries are pushed to the AQAO

8.     AQAO declines or accepts comments, or gets into touch with SQAO and/or author

9.     Accepted commentaries are pushed to the author

10.  Author can push a revised version.

11.  A revised version is a new publication but should be linked to the original version.

12.  As a reader I want to know what kind of quality assurances have been done by UJs.

13.  As a reader I want to know what kind of other quality assurances have been done by others.

14.  As an institute I want to archive content as well as review reports, decisions and timelines, following international metadata standards to ensure portability.

Use case #9: Perspective of a (climate) scientist

Description

Reading some of the previous use cases it sometimes felt like the repository stands between me as a scientist and the (overlay) journal. It regularly happens that parts of the paper become worse to pass the peer review. An author would prefer not to put such versions on a repository in case the manuscript is sent elsewhere and these concessions to the reviewers can be removed again. 

Also the manuscript version a journal would like tends to differ from the best formatting for a repository. For overlay journals this may not be the case, but other journals often require special formats; it would be useful for them to know the repository version, but they will likely require the scientists to upload a journal-specific manuscript.

With that in mind, I would see the following as a typical workflow for posting a manuscript to a repository and later forwarding it to a journal.

Workflow

1. Authors post a manuscript to a repository (REP).

2. Informal peer review of the manuscript; possible new versions of the manuscript uploaded to the repository.

3. (After some time) authors submit their manuscript to an (overlay) journal and give them the URL of the REP manuscript. The journal may upload the latest version of the manuscript (and any open reviews on this and earlier versions) from the repository. And the journal subscribes to this manuscript to get notifications of new manuscript versions.

4. Review at the journal. Authors may submit updated versions either to the journal or to the repository, of which the journal is notified and they can automatically download it to the journal.

5. Possible peer review at additional journals by giving them the URL of the REP manuscript.

6. Once the manuscript is accepted by one of the journals this is noted on the repository and the accepted version of the manuscript is posted on the repository (and journal). If not allowed immediately, it should be able to be published automatically on the repository when the embargo period runs out.

7. Ideally, the VOR is automatically posted to the repository (once ready or after any embargo period) by the journal.

Comments

The authors informing the journal about the manuscript URL avoids a complicated/centralized system at the repository to select thejournal to send the manuscript to. Informing the journal of the REP URL always works for any decentralized (overlay) journal implementing the communication protocol.

Authors uploading updated versions to the journal, rather than the repository provides the authors with privacy during the (closed) review stage; I am a fan of open review, but there are cases where authors will prefer closed review. If this were not possible this would make the system less attractive. Also not many researchers will use a repository that may mark their manuscripts as “rejected by journal X”. So I would only add “accepted by journal X” as metadata.

The (overlay) journal would subscribe to the ActivityStream or ActivityPub account of the manuscript at the repository to be informed on any new posted manuscripts. If the journal is also to provide the service of uploading the final versions to the repository, it would need a step where the authors log into the repository to authorize the journal to do so.

Use case #10: The system needed to spread review reports of a grassroots review journal.

Description

This is the use case of a grassroots review journal. This is an open post-publication peer review system, which is independent of (overlay) journals. For every article it has one page with (minor and major) comments, quantitative assessments by reviewers and a quantitative synthesis by the editors. 

The post-publication peer review is organized in a similar way as publishing journals with editors who invite colleagues to review, but review journals only assess the study, but do not publish the manuscript itself. Multiple review journals may review the same article and should thus be able to share review reports. It should also be possible to start a new review journal based on an existing one; so it should be possible to transfer all reviews of a review journal. It should be possible to ask a review journal whether it has reviews of an article (by doi). The review would be multiple review reports — text and information on its category (comment, review, synthesis, etc.) — and the quantitative assessments, which are a few numbers (per quantitative review and synthesis) and a link to an URL explaining the assessment system. 

The review page would also contain additional information to help the reviewers (endorsements and review reports from other systems, links to citations, references, code, data, event data, translations, refutations/reproductions, retractions, …) The other review journals could retrieve these themselves, but it may also be nice to also be able to directly download and subscribe to them.

Next to these transfers (which means: downloading the current information and subscribing to any future updates) between review journals (and hopefully repositories), users should also be able to keep up to date about journals and articles under review with RSS feeds, ActivityPub and Email.

This is also a system to interconnect repositories with reviews, but quite different from the other use cases. Still it may at least be worthwhile to use similar communication technologies. The information on a review page of a review journal (especially the open reviews themselves) would also be useful on the manuscript page of a repository to help readers assess the study they are reading.

Workflows

  1. Subscribe to all new reviews of a journal
    1. Follow a review journal with your ActivityStream app (for example Mastodon). This should also be possible with RSS and email, but that is not relevant here. 
  1. Subscribe to all new reviews of an article
    1. Follow an article at a review journal with your ActivityStream app (for example Mastodon). Similar for RSS and email, but that is not relevant here.
  1. Query a review journal whether they have a review of an article by doi (and later also title and authors, etc.). If yes, return the doi (or URL) of the review.
  1. Copy a journal (mostly to start a new competing journal)
    1. Query a review journal for all their reviews.
    2. Download all existing reviews. Or a subset of interest.
    3. Subscribe to all articles for new reviews.
  1. Copy the reviews of an article (typically for repository or a related review journal)
    1. Download all existing reviews.
    2. Subscribe to all articles for new reviews.
  1. Subscribe to an external information source, such as a repository, whether there is new information on the manuscript/article.

Postscript

A review journal would also ingest data from a wide range of additional data sources; information on the data on the article, on the code used, on protocols, on reviews elsewhere, etc. This information would help the reviews at a review journal, but could also help the readers of a repository, who basically have to do their own review. I have a long list of possible related information sources in this presentation (suggestions to make this list more complete are welcome): 

https://zenodo.org/record/3923961

Some coordination here may also be helpful to reduce the proliferation of communication protocols to retrieve such additional data sources. Currently most of them seem to use REST APIs, which would have to be queried regularly to see whether anything has changed. Some would need to be web scrapped. It would be helpful if more of these services would also move to ActivityStreams or ActivityPub so that one could subscribe to be notified of any changes.