diff --git a/_config.yml b/_config.yml
deleted file mode 100644
index c296059b..00000000
--- a/_config.yml
+++ /dev/null
@@ -1,30 +0,0 @@
-# Book settings
-title: The pyOpenSci Developer Guide
-author: The pyOpenSci Community
-logo: images/logo/logo.png
-exclude_patterns: ["README.md"]
-
-repository:
- url: https://github.com/pyOpenSci/contributing-guide
- branch: main
-
-#parse:
-# myst_enable_extensions:
-# - sphinx_copybutton
-
-
-html:
- favicon: images/logo/logo.png
- google_analytics_id: UA-141260825-1
- use_edit_page_button: true
- use_issues_button: true
- use_repository_button: true
- home_page_in_navbar: false
- announcement: "
- "
diff --git a/_static/pyos.css b/_static/pyos.css
index 41f9b319..ad6747d3 100644
--- a/_static/pyos.css
+++ b/_static/pyos.css
@@ -1,53 +1,60 @@
-/* HEADER BLOCK */
-div.header__block {
- color: #222;
+body, p {
+ font-size: 1.04em;
}
-/* not working and not sure why */
-.caption-text {
-
+h1, h2, h3 {
+ font-family:'Franklin Gothic Medium', Arial, sans-serif;
+ font-weight: bold;
}
+h1 {
+ margin-bottom: 1em;
+ font-size: 2.5rem;
+}
-/*
-.announcement div {
- background-color: #ccc;
-} */
+h2 {
+ margin-top: 1.7em;
+}
-.header-item.announcement, .header-item.announcement .noprint {
- background-color:#ccc;
+h3, .content-container h3 {
+ margin-top: 1.6em!important;
}
-div .announcement .header-item .noprint, header-item.announcement {
- background-color: #ccc!important;
+ul li p {
+ padding-bottom: .8em;
+ line-height: 1.5em;
}
-/* notes */
+.epigraph {
+ font-style: italic;
+ border-left: 4px solid #ccc!important;
+ margin-left: 0!important;
+ font-size: 1.2em!important;
+}
-div.admonition.note .admonition-title, div.admonition.note .admonition-title::before {
- background-color: #C1CFD4;
+div.header__block {
color: #222;
-
}
-div.admonition.note {
- border-color: #C1CFD4;
- background-color: #E3F4FA;
+/* not working and not sure why */
+.caption-text {
+
}
-div.admonition.important .admonition-title, div.admonition.important .admonition-title::before {
- background-color: #107762;
- color: #ffffff!important;
+/*
+.announcement div {
+ background-color: #ccc;
+} */
+/*
+.header-item.announcement, .header-item.announcement .noprint {
+ background-color:#ccc;
}
-div.admonition.important .admonition-title {
- background-color: #107762;
- border-color: #1abc9c;
+div.admonition.note {
+ margin-top: 4em;
+ margin-bottom: 4em;
+ background-color: #F8F9FB
}
-div.admonition.important {
- border-color: #0e6654;
- background-color: #ecfcf9;
-}
diff --git a/about-peer-review/code-of-conduct.md b/about-peer-review/code-of-conduct.md
index 6e7eafce..08a4f7ed 100644
--- a/about-peer-review/code-of-conduct.md
+++ b/about-peer-review/code-of-conduct.md
@@ -7,7 +7,7 @@ go there now.](https://www.pyopensci.org/governance/code-of-conduct)
## NOTE: we are in the process of moving this file to our governance documentation and making significant changes to our code of conduct.
-- We are committed to providing a friendly, safe and welcoming environment for all, regardless of gender, sexual orientation, disability, ethnicity, religion, or similar personal characteristic.
+- We are committed to providing a friendly, safe and welcoming environment for all, regardless of gender, sexual orientation, disability, ethnicity, religion, age, or any personal characteristics.
- Please avoid using openly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
- Please be kind and courteous. There’s no need to be mean or rude.
- Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
diff --git a/about-peer-review/how-peer-review-works.md b/about-peer-review/how-peer-review-works.md
index 50694cf0..54e72111 100644
--- a/about-peer-review/how-peer-review-works.md
+++ b/about-peer-review/how-peer-review-works.md
@@ -10,7 +10,7 @@ high quality tools
that supports scientists across domains with a suite of different data
types and structures.
-### Who reviews pyOpenSci packages ?
+### Who reviews pyOpenSci packages?
Our peer review process is run by volunteer members of the Python scientific
community:
@@ -30,19 +30,24 @@ Our entire peer review process occurs on GitHub in the
We use GitHub because:
-* It is free to create an account
-* Anyone can read the review discussion without an account making the process entirely open
-* It facilitates collaboration and supports community around a package
-* It facilitates open discussion between reviewers and package maintainers and the pyOpenSci volunteers
-* Numerous packages store their code bases on GitHub
+* It is free to create an account,
+* Anyone can read the review discussion without an account, making the process entirely open,
+* It facilitates collaboration and supports the community around a package,
+* It facilitates an open discussion between reviewers, package maintainers and the pyOpenSci volunteers,
+* Numerous packages store their code bases on GitHub.
### We use GitHub issue templates and labels to organize the review steps
* We use GitHub issue templates as submission templates for new reviews and pre-submission review questions.
-* We label issues to track every step of the package submission and review progress (e.g. [1/initial-editor-checks, 2/reviewers-needed, 6/pyopensci-approved](https://github.com/pyOpenSci/software-review/labels)
+* We label issues to track every step of the package submission and review progress (e.g. [1/initial-editor-checks, 2/reviewers-needed, 6/pyopensci-approved](https://github.com/pyOpenSci/software-review/labels)).
```{note}
-Click [here](https://github.com/ropensci/software-review/issues/24) to read the review thread from an rOpenSci review of the `ropenaq` package. Note that the process is an ongoing conversation until the package is accepted. Two external reviews are important milestones in the review process.
+[Click here to read the review thread from a December 2022
+pyOpenSci pre-submission issue.](https://github.com/pyOpenSci/software-review/issues/65) Note the conversational
+tone of the pre-review. In this case the package was improved
+before the formal review even began.
+
+In the actual review process, two external reviews are important milestones. The editor will also provide the author with some feedback.
```
For more detailed overview of the peer review process, [check out our process
diff --git a/about-peer-review/intro.md b/about-peer-review/intro.md
index 070fac65..e3012191 100644
--- a/about-peer-review/intro.md
+++ b/about-peer-review/intro.md
@@ -10,7 +10,7 @@ Software peer review refers to a peer-review process that focuses on open source
* Code quality,
* Documentation quality,
* Package usability,
-* Test coverage that supports both maintenance of code function. Test coverage also makes it easier for contributors to see how their contributions impacts other parts of the code,
+* Test coverage that supports the maintenance of code function. Test coverage also makes it easier for contributors to see how their contributions impact other parts of the code,
* Evaluation of infrastructure such as continuous integration to run rest suites and check code linters, that supports automated checks on pull requests. This infrastructure supports software quality and reliability.
## What types of packages does pyOpenSci review?
@@ -20,7 +20,7 @@ pyOpenSci reviews higher level software packages that support scientific workflo
-Diagram showing the tiers of software in the python ecosystem starting with Python itself and as you move out packages become more domain specific. In this image packages like xarray and numpy are considered core to scientific python. Packages and distributions like astropy, simpeg and metpy are considered to be domain specific.. pyOpenSci's review
+Diagram showing the tiers of software in the python ecosystem starting with Python itself and as you move out packages become more domain specific. In this image packages like xarray and numpy are considered core to scientific python. Packages and distributions like astropy, simpeg and metpy are considered to be domain specific. pyOpenSci's review
process focuses on domain specific packages rather than core packages as
these packages tend to have more variability in long term maintenance and
package infrastructure and quality compared to established core packages. **Source: ["Jupyter meets earth" project](https://jupytearth.org/jupyter-resources/introduction/ecosystem.html)**
@@ -54,7 +54,7 @@ faced within the scientific Python community are broader in scale given the
numerous and diverse applications that the Python programming language is used for.
```{note}
-[This blog post](https://www.numfocus.org/blog/how-ropensci-uses-code-review-to-promote-reproducible-science/) written by editors from our partner organization, rOpenSci, is a good introduction to pyOpenSci software peer review
+[This blog post](https://www.numfocus.org/blog/how-ropensci-uses-code-review-to-promote-reproducible-science/) written by editors from our partner organization, rOpenSci, is a good introduction to pyOpenSci software peer review.
```
### Peer review of open source software helps maintain consistent quality
diff --git a/about-peer-review/policies-guidelines.md b/about-peer-review/policies-guidelines.md
index aa3e2d97..1109b556 100644
--- a/about-peer-review/policies-guidelines.md
+++ b/about-peer-review/policies-guidelines.md
@@ -2,31 +2,25 @@
## Review Process Guidelines
-- Packages are reviewed for quality, fit, documentation, clarity and the review process
- is quite similar to a manuscript review (see our [packaging guide](../authoring/overview)
- and [reviewing guide](../peer-review-guides/reviewer-guide) for more details). Unlike a
- manuscript review, this process will be an ongoing conversation.
-- Once all major issues and questions, and those addressable with reasonable effort, are
- resolved, the editor assigned to a package will make a decision (accept, hold, or
- reject). Rejections are usually done early (before the review process begins, see the
- aims and scope section). In rare cases a package may also not be on-boarded after
- review & revision. It is ultimately editor’s decision on whether or not to reject the
- package based on how the reviews are addressed.
-- Communication between authors, reviewers and editors will first and foremost take
- place on GitHub, although you can choose to contact the editor by email. When
- submitting a package, please make sure your GitHub notification settings make it
- unlikely you will miss a comment.
-- The author can choose to have their submission put on hold (editor applies the holding
- label). The holding status will be revisited every 3 months, and after one year the
- issue will be closed.
-- If the author hasn’t requested a holding label, but is simply not responding, we
- should close the issue within one month after the last contact intent. This intent
- will include a comment tagging the author, but also an email using the email address
- listed in the DESCRIPTION of the package which is one of the rare cases where the
- editor will try to contact the author by email.
-- If a submission is closed and the author wishes to re-submit, they’ll have to start a
- new submission. If the package is still in scope, the author will have to respond to
- the initial reviews before the editor starts looking for new reviewers.
+pyOpenSci packages are reviewed for quality, fit, scope,
+documentation and usability. The review process
+is similar to a manuscript review, however it has a stronger
+focus on Python packaging best practices.
+
+Unlike a manuscript review, our peer review process is an ongoing conversation. Once all major issues and questions are addressed, the review editor package will make a decision to accept, hold, or reject the package.
+
+Rejections are usually done early in the process, before the review process begins. In rare cases a package may also not be on-boarded into the pyOpenSci ecosystem after review & revision.
+
+It is ultimately the editor’s decision on whether or not to reject the package based on how the reviews are addressed.
+
+## Review communication approach
+
+Communication between authors, reviewers and editors takes
+place on GitHub. You can, however choose to contact the editor by email if needed.
+
+When submitting a package, please make sure that your GitHub notification settings are setup to notify you when you receive feedback on the review issue.
+
+
## Submitting your package for review in other venues
diff --git a/about-peer-review/pyopensci-related-joss-ropensci.md b/about-peer-review/pyopensci-related-joss-ropensci.md
index 30f64ccf..a38eb744 100644
--- a/about-peer-review/pyopensci-related-joss-ropensci.md
+++ b/about-peer-review/pyopensci-related-joss-ropensci.md
@@ -11,11 +11,11 @@ The JOSS review process is about publication. A review from JOSS will provide
you with a citable, [Crossref digital object identifier (DOI)](https://www.crossref.org/).
pyOpenSci aligns closely with the broad mission of
JOSS to provide maintainers with credit for their open source work. However,
-our mission is also more focused. pyOpenSci not open source maintainers getting academic credit for their work. We also support:
+our mission is also more focused. pyOpenSci is not just about open source maintainers getting academic credit for their work. We also support:
-* the Python tools that drive scientific open reproducible science workflows;
-* enforcement and encouragement of Python-specific packaging standards across tools;
-* and curated lists of peer reviewed, and maintained Python scientific software.
+* the Python tools that drive scientific open reproducible science workflows;
+* Enforcement and encouragement of Python-specific packaging standards across tools;
+* Curated lists of peer reviewed, and maintained Python scientific software.
## How is review at pyOpenSci different from the JOSS review process?
@@ -23,16 +23,15 @@ We are not a typical publisher or journal. Rather we are a community that provid
The pyOpenSci review process is different from that of JOSS in a few ways:
-1. Our review is specifically design to enforce modern, community-accepted best practices for Python packaging.
+1. Our review is specifically designed to enforce modern, community-accepted best practices for Python packaging.
1. We place heavy emphasis on documentation and usability in our reviews and associated standardization of both.
1. We build community around and visibility for its tools.
1. We will promote packages and package maintainers once they are accepted into our ecosystem.
-1. We support long term maintenance of packages. If the maintainer needs to step down, we will ensure a new maintainer takes over OR sunset and remove the package from our ecosystem.
+1. We support long term maintenance of packages. If the maintainer needs to step down, we will ensure a new maintainer takes over or sunset and remove the package from our ecosystem.
1. We provide a welcoming forum for you to ask questions and get help with maintaining your package as needed.
-JOSS reviews are also [more limited in scope](https://joss.readthedocs.io/en/latest/review_criteria.html)
-compared to pyOpenSci. Some of the
-[JOSS submission criteria](https://joss.readthedocs.io/en/latest/review_criteria.html)
+JOSS reviews are also [more limited in scope](https://joss.readthedocs.io/en/latest/submitting.html?highlight=scope#submission-requirements)
+compared to pyOpenSci. Some of the JOSS submission criteria
are, in places, less stringent or less specific to the Python programming
language than those of pyOpenSci.
@@ -41,8 +40,8 @@ language than those of pyOpenSci.
You don't have to chose between pyOpenSci and JOSS; You can submit your package to both.
pyOpenSci and JOSS are complementary, partner organizations. You don't have
-to chose one or the other! After a package to pyOpenSci has been reviewed and
-accepted by pyOpenSci you can also choose to register it with JOSS.
+to choose one or the other! After a package to pyOpenSci has been reviewed and
+accepted you can also choose to register it with JOSS.
JOSS has [more limited scope](https://joss.readthedocs.io/en/latest/review_criteria.html)
for packages that it will review. For instance, while pyOpenSci will review
@@ -56,5 +55,5 @@ You do not need to go through two reviews!
Once accepted by JOSS, you now have both a pyOpenSci acceptance and one by JOSS.
* JOSS will give you a Crossref DOI for citation.
-* And you can display that and your pyOpenSci peer reviewed badge at the top of your package's README.md file.
+* You can display that and your pyOpenSci peer reviewed badge at the top of your package's README.md file.
diff --git a/accepted-checklist.md b/accepted-checklist.md
new file mode 100644
index 00000000..220866c2
--- /dev/null
+++ b/accepted-checklist.md
@@ -0,0 +1,3 @@
+
+- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly
+- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
\ No newline at end of file
diff --git a/appendices/editor-in-chief-checks.md b/appendices/editor-in-chief-checks.md
new file mode 100644
index 00000000..5cef103d
--- /dev/null
+++ b/appendices/editor-in-chief-checks.md
@@ -0,0 +1,36 @@
+```markdown
+## Editor in Chief checks
+
+Hi there! Thank you for submitting your package for pyOpenSci
+review. Below are the basic checks that your package needs to pass
+to begin our review. If some of these are missing, we will ask you
+to work on them before the review process begins.
+
+- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g. conda-forge, bioconda).
+ - [ ] The package imports properly into a standard Python environment `import package-name`.
+- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-scope) and [overlap](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-overlap).
+- [ ] **Documentation** The package has sufficient online documentation (README, sphinx docs) to allow us to evaluate package function and scope *without installing the package*. This includes:
+ - [ ] Short tutorials or [vignettes](https://kbroman.org/pkg_primer/pages/vignettes.html) that help a user understand how to use the package and what it can do for them (often these have a name like "Getting started")
+ - [ ] API documentation: this includes clearly written docstrings with variables defined using a standard docstring format. *We suggest using the [Numpy](https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard) docstring format*.
+ - [ ] **README** The package has a `README.md` file with clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
+ - [ ] **Contributing File** The package has a `CONTRIBUTING.md` file that details how to install and contribute to the package.
+- [ ] **Issue Submission Documentation** All of the information is filled out in the `YAML` header of the issue (located at the top of the issue template).
+- [ ] **Automated tests** Package has a testing suite and is tested via GitHub actions or another Continuous Integration service.
+- [ ] **License** The package has an [OSI approved license](https://opensource.org/licenses).
+- [ ] **Repository** The repository link resolves correctly.
+- [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
+- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly.
+- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
+
+---
+- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
+We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
+Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
+---
+
+*******
+
+## Editor comments
+
+
+```
\ No newline at end of file
diff --git a/appendices/editor-initial-response-template.md b/appendices/editor-initial-response-template.md
new file mode 100644
index 00000000..cfbd9f55
--- /dev/null
+++ b/appendices/editor-initial-response-template.md
@@ -0,0 +1,21 @@
+```markdown
+## Editor response to review:
+
+---
+
+## Editor comments
+:wave: Hi @reviewer-one and @reviewer-two! Thank you for volunteering to review
+for PyOpenSci!
+
+The following resources will help you complete your review:
+
+1. Here is the **[reviewers guide](https://www.pyopensci.org/contributing-guide/peer-review-guides/reviewer-guide.html)**. This guide contains all of the steps and information needed to complete your review.
+2. Here is the **[review template](https://www.pyopensci.org/contributing-guide/appendices/templates.html#review-template)** that you will need to fill out and submit
+here as a comment, once your review is complete.
+
+Please get in touch with any questions or concerns! Your review is due:
+---
+
+Reviewers:
+Due date:
+```
\ No newline at end of file
diff --git a/appendices/issue-review-template.md b/appendices/issue-review-template.md
new file mode 100644
index 00000000..f1d1b8da
--- /dev/null
+++ b/appendices/issue-review-template.md
@@ -0,0 +1,22 @@
+```markdown
+---
+name: Submit Software for Review
+about: Use to submit your Python package for pyOpenSci peer review
+title: ''
+labels: 1/editor-checks, New Submission!
+assignees: ''
+---
+Submitting Author: Name (@github_handle)
+All current maintainers: (@github_handle1, @github_handle2)
+Package Name: Package name here
+One-Line Description of Package: Description here
+Repository Link:
+Version submitted:
+Editor: TBD
+Reviewer 1: TBD
+Reviewer 2: TBD
+Archive: TBD
+Version accepted: TBD
+Date accepted (month/day/year): TBD
+---
+```
\ No newline at end of file
diff --git a/appendices/package-approval-template.md b/appendices/package-approval-template.md
new file mode 100644
index 00000000..fc619e5d
--- /dev/null
+++ b/appendices/package-approval-template.md
@@ -0,0 +1,24 @@
+```markdown
+----------------------------------------------
+🎉 has been approved by pyOpenSci! Thank you for submitting and many thanks to for reviewing this package! 😸
+
+There are a few things left to do to wrap up this submission:
+- [ ] Activate [Zenodo](https://zenodo.org/) watching the repo if you haven't already done so.
+- [ ] Tag and create a release to create a Zenodo version and DOI.
+- [ ] Add the badge for pyOpenSci peer-review to the README.md of . The badge should be `[](https://github.com/pyOpenSci/software-review/issues/issue-number)`.
+- [ ] Add to the pyOpenSci website. , please open a pr to update [this file](https://github.com/pyOpenSci/pyopensci.github.io/blob/main/_data/packages.yml): to add your package and name to the list of contributors.
+- [ ] if you have time and are open to being listed on our website, please add yourselves to [this file](https://github.com/pyOpenSci/pyopensci.github.io/blob/main/_data/contributors.yml) via a pr so we can list you on our website as contributors!
+
+
+It looks like you would like to submit this package to JOSS. Here are the next steps:
+
+- [ ] Login to the JOSS website and fill out the JOSS submission form using your Zenodo DOI. **When you fill out the form, be sure to mention and link to the approved pyOpenSci review.** JOSS will tag your package for expedited review if it is already pyOpenSci approved.
+- [ ] Wait for a JOSS editor to approve the presubmission (which includes a scope check).
+- [ ] Once the package is approved by JOSS, you will be given instructions by JOSS about updating the citation information in your README file.
+- [ ] When the JOSS review is complete, add a comment to your review in the pyOpenSci software-review repo that it has been approved by JOSS.
+
+🎉 Congratulations! You are now published with both JOSS and pyOpenSci! 🎉
+
+
+All -- if you have any feedback for us about the review process please feel free to share it here. We are always looking to improve our process and documentation in the [contributing-guide](https://www.pyopensci.org/contributing-guide). We have also been updating our documentation to improve the process, so all feedback is appreciated!
+```
\ No newline at end of file
diff --git a/appendices/pre-review-package-requirements.md b/appendices/pre-review-package-requirements.md
new file mode 100644
index 00000000..c55233fe
--- /dev/null
+++ b/appendices/pre-review-package-requirements.md
@@ -0,0 +1,12 @@
+
+- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-scope) and [overlap](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-overlap).
+ - [ ] **Package overlap** That package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
+- [ ] **Documentation** The package has sufficient online documentation (README, sphinx docs) to allow us to evaluate package function and scope *without installing the package*. This includes:
+ - [ ] Get started vignettes or tutorials that help users understand how to use the package and what it can do for them.
+ - [ ] API documentation - this includes clearly written doc strings with variables defined using a standard docstring format.
+- [ ] **README.md File** The package has a README.md file with a clear explanation of what the package does and instructions on how to install it, along with development instructions.
+ - [ ] README file clearly articulates the function of the package.
+- [ ] **CONTRIBUTING.md file** The package has a `CONTRIBUTING.md` file that details how to install and contribute to the package.
+- [ ] **Automated tests** Package has a testing suite and is tested via GitHub actions or another Continuous Integration service.
+- [ ] **License** The package has an OSI accepted license.
+- [ ] **Repository** The package repository link resolves correctly.
\ No newline at end of file
diff --git a/appendices/review-template.md b/appendices/review-template.md
new file mode 100644
index 00000000..b38a2ff4
--- /dev/null
+++ b/appendices/review-template.md
@@ -0,0 +1,94 @@
+## Peer Review Template
+
+```
+## Package Review
+
+*Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide*
+
+- [ ] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor _before_ starting your review).
+
+#### Documentation
+
+The package includes all the following forms of documentation:
+
+- [ ] **A statement of need** clearly stating problems the software is designed to solve and its target audience in README.
+- [ ] **Installation instructions:** for the development version of the package and any non-standard dependencies in README.
+- [ ] **Vignette(s)** demonstrating major functionality that runs successfully locally.
+- [ ] **Function Documentation:** for all user-facing functions.
+- [ ] **Examples** for all user-facing functions.
+- [ ] **Community guidelines** including contribution guidelines in the README or CONTRIBUTING.
+- [ ] **Metadata** including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a `pyproject.toml` file or elsewhere.
+
+Readme file requirements
+The package meets the readme requirements below:
+
+- [ ] Package has a README.md file in the root directory.
+
+The README should include, from top to bottom:
+
+- [ ] The package name
+- [ ] Badges for:
+ - [ ] Continuous integration and test coverage,
+ - [ ] Docs building (if you have a documentation website),
+ - [ ] A [repostatus.org](https://www.repostatus.org/) badge,
+ - [ ] Python versions supported,
+ - [ ] Current package version (on PyPI / Conda).
+
+*NOTE: If the README has many more badges, you might want to consider using a table for badges: [see this example](https://github.com/ropensci/drake). Such a table should be more wide than high. (Note that the a badge for pyOpenSci peer-review will be provided upon acceptance.)*
+
+- [ ] Short description of package goals.
+- [ ] Package installation instructions
+- [ ] Any additional setup required to use the package (authentication tokens, etc.)
+- [ ] Descriptive links to all vignettes. If the package is small, there may only be a need for one vignette which could be placed in the README.md file.
+ - [ ] Brief demonstration of package usage (as it makes sense - links to vignettes could also suffice here if package description is clear)
+- [ ] Link to your documentation website.
+- [ ] If applicable, how the package compares to other similar packages and/or how it relates to other packages in the scientific ecosystem.
+- [ ] Citation information
+
+#### Usability
+Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole.
+Package structure should follow general community best-practices. In general please consider whether:
+
+- [ ] Package documentation is clear and easy to find and use.
+- [ ] The need for the package is clear
+- [ ] All functions have documentation and associated examples for use
+- [ ] The package is easy to install
+
+
+#### Functionality
+
+- [ ] **Installation:** Installation succeeds as documented.
+- [ ] **Functionality:** Any functional claims of the software been confirmed.
+- [ ] **Performance:** Any performance claims of the software been confirmed.
+- [ ] **Automated tests:** Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
+- [ ] **Continuous Integration:** Has continuous integration setup (We suggest using Github actions but any CI platform is acceptable for review)
+- [ ] **Packaging guidelines**: The package conforms to the pyOpenSci [packaging guidelines](https://www.pyopensci.org/python-package-guide).
+ A few notable highlights to look at:
+ - [ ] Package supports modern versions of Python and not [End of life versions](https://endoflife.date/python).
+ - [ ] Code format is standard throughout package and follows PEP 8 guidelines (CI tests for linting pass)
+
+#### For packages also submitting to JOSS
+
+- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements](http://joss.theoj.org/about#submission_requirements).
+
+*Note:* Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.
+
+The package contains a `paper.md` matching [JOSS's requirements](http://joss.theoj.org/about#paper_structure) with:
+
+- [ ] **A short summary** describing the high-level functionality of the software
+- [ ] **Authors:** A list of authors with their affiliations
+- [ ] **A statement of need** clearly stating problems the software is designed to solve and its target audience.
+- [ ] **References:** With DOIs for all those that have one (e.g. papers, datasets, software).
+
+#### Final approval (post-review)
+
+- [ ] **The author has responded to my review and made changes to my satisfaction. I recommend approving this package.**
+
+Estimated hours spent reviewing:
+
+---
+
+#### Review Comments
+
+
+```
\ No newline at end of file
diff --git a/appendices/reviewer-request-template.md b/appendices/reviewer-request-template.md
new file mode 100644
index 00000000..0d5eef0c
--- /dev/null
+++ b/appendices/reviewer-request-template.md
@@ -0,0 +1,31 @@
+```markdown
+Dear [REVIEWER-Name],
+
+Hi, this is [EDITOR-Name]. [FRIENDLY BANTER]. I'm writing to ask if you have time
+to review a package for pyOpenSci. pyOpenSci has an open peer review of open source Python packages that support science. Accepted packages become a part of our pyOpenSci ecosystem of vetted community-adopted tools. Our review process is similar to that
+of open journals however it focuses on ensuring high quality packaging approaches
+that are inline with community adopted Python standards.
+
+The package, [PACKAGE] by [AUTHOR(S)], does [FUNCTION]. You can find it on GitHub here: [REPO LINK]. We conduct our open review process via GitHub as well, here: [ONBOARDING ISSUE]
+
+If you accept, note that we ask reviewers to complete reviews in three weeks (We’ve found it takes a similar amount of time to review a package as an academic paper.).
+
+Our [reviewers guide](https://www.pyopensci.org/peer-review-guide/software-peer-review-guide/reviewer-guide.html) details what we look for in a package review, and includes links to example reviews. We also include a reviewer template on that page that you can use to guide your review. Our Python packaging standards are detailed in our [packaging guide](https://www.pyopensci.org/python-package-guide).
+
+If you have time, please have a look at the package first, to make
+sure that you do not have a conflict of interest with the package authors.
+
+If you have questions or feedback, feel free to ask me.
+
+{IF APPLICABLE: The authors have also chosen to jointly submit their package to the Journal of Open Source Software, so this package includes a short `paper.md` manuscript describing the software that we ask you include in your review.}
+
+Are you able to review? If you can not, I welcome any suggestions that you
+have for other reviewers. If I don't hear from you within a week, I will assume
+that you are unable to review this package at this time.
+
+Thank you for your time.
+
+Sincerely,
+
+[EDITOR]
+```
\ No newline at end of file
diff --git a/scope.md b/appendices/scope.md
similarity index 82%
rename from scope.md
rename to appendices/scope.md
index 45463e29..233a7cca 100644
--- a/scope.md
+++ b/appendices/scope.md
@@ -12,7 +12,15 @@ pyOpenSci domain scope.
- **Education:** Packages to aid with instruction.
- **Data visualization:** Packages for visualizing and analyzing data.
-### Notes on Scope Categories
+## Package technical scope
+
+To be in technical scope for a pyOpenSci review, your package:
+
+* Should have maintenance workflows documented.
+* Should be structured in a way that someone else could contribute to it.
+* Should declare vendor dependencies using standard approaches rather than including code from other packages within your repository.
+
+### Notes on scope categories
- pyOpenSci is still developing as a community. If your scientific Python
package does not fit into one of the categories or if you have any other
questions, we'd encourage you to open a pre-submission inquiry. We're happy to help.
@@ -20,11 +28,11 @@ questions, we'd encourage you to open a pre-submission inquiry. We're happy to h
hyper-specific methods for one type of data to general, do-it-all packages
(e.g. matplotlib). pyOpenSci accepts packages that are somewhere in between the
two. If you're interested in submitting your data visualization package, please
-open a pre-submission inquiry on first.
+open a pre-submission inquiry first.
## Python package technical scope
-pyOpenSci is may continue to update it's technical scope criteria for package
+pyOpenSci may continue to update its technical scope criteria for package
review as more packages with varying structural approaches are reviewed.
Your package **may not be in technical scope** for us to review at this time if
fits any of the out-of-technical-scope criteria listed below.
@@ -37,10 +45,8 @@ pyOpenSci has a goal of supporting long term maintenance of open source
Python tools. It is thus important for us to know that if you need to step down as a maintainer, the package infrastructure and documentation is
in place to support us finding a new maintainer who can take over you
package's maintenance.
-```
-A few examples of technically complex package structures that may
-make it difficult for us to review are below:
+## Examples of technically complex package structures that may be difficult for us to review
### Example 1: Your package is an out of sync fork of another package repository that is being actively maintained.
@@ -50,16 +56,17 @@ package repository to a new organization along with PyPI credentials. A new
organization would allow transfer of ownership of package maintenance rather
than several forks existing.
-Of your package is a divergent fork of a maintained repository we will encourage you
+If your package is a divergent fork of a maintained repository we will encourage you
to work with the original maintainers to merge efforts.
However, if there is a case where a forked repository is warranted, please
consider submitting a pre-submission inquiry first and explain why the package is a
fork rather than an independent parent repository.
-### Example 2 Vendored dependencies
+### Example 2: Vendored dependencies
If your package is a wrapper that wraps around another tool, we prefer that
the dependency be added as a dependency to your package. This allows
maintenance of the original code base to be independent from your package's
maintenance.
+```
\ No newline at end of file
diff --git a/appendices/templates.md b/appendices/templates.md
index e730960f..fab91960 100644
--- a/appendices/templates.md
+++ b/appendices/templates.md
@@ -4,133 +4,18 @@ These are templates to be used by editors and reviewers. When a package is submi
## Editor's Template
+```{include} editor-initial-response-template.md
```
-## Editor checks:
-
-- [ ] **Fit**: The package meets criteria for [fit](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-scope) and [overlap](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-overlap).
-- [ ] **Automated tests:** Package has a testing suite and is tested via Travis-CI or another CI service.
-- [ ] **License:** The package has an OSI accepted license
-- [ ] **Repository:** The repository link resolves correctly
-- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly
-- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
-
----
-
-## Editor comments
-
----
-
-Reviewers:
-Due date:
-```
-
-## Review Template
-
-```
-## Package Review
-
-*Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide*
-
-- [ ] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor _before_ starting your review).
-
-#### Documentation
-
-The package includes all the following forms of documentation:
-
-- [ ] **A statement of need** clearly stating problems the software is designed to solve and its target audience in README
-- [ ] **Installation instructions:** for the development version of package and any non-standard dependencies in README
-- [ ] **Vignette(s)** demonstrating major functionality that runs successfully locally
-- [ ] **Function Documentation:** for all user-facing functions
-- [ ] **Examples** for all user-facing functions
-- [ ] **Community guidelines** including contribution guidelines in the README or CONTRIBUTING.
-- [ ] **Metadata** including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a `setup.py` file or elsewhere.
-
-Readme requirements
-The package meets the readme requirements below:
-
-- [ ] Package has a README.md file in the root directory.
-
-The README should include, from top to bottom:
-
-- [ ] The package name
-- [ ] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: [see this example](https://github.com/ropensci/drake). Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
-- [ ] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
-- [ ] Installation instructions
-- [ ] Any additional setup required (authentication tokens, etc)
-- [ ] Brief demonstration usage
-- [ ] Direction to more detailed documentation (e.g. your documentation files or website).
-- [ ] If applicable, how the package compares to other similar packages and/or how it relates to other packages
-- [ ] Citation information
-
-#### Usability
-Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole.
-Package structure should follow general community best-practices. In general please consider:
-
-- [ ] The documentation is easy to find and understand
-- [ ] The need for the package is clear
-- [ ] All functions have documentation and associated examples for use
-
-
-#### Functionality
-
-- [ ] **Installation:** Installation succeeds as documented.
-- [ ] **Functionality:** Any functional claims of the software been confirmed.
-- [ ] **Performance:** Any performance claims of the software been confirmed.
-- [ ] **Automated tests:** Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
-- [ ] **Continuous Integration:** Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
-- [ ] **Packaging guidelines**: The package conforms to the pyOpenSci [packaging guidelines](https://www.pyopensci.org/contributing-guide/authoring/index.html#packaging-guide).
-
-#### For packages co-submitting to JOSS
-
-- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements](http://joss.theoj.org/about#submission_requirements).
-
-*Note:* Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.
-
-The package contains a `paper.md` matching [JOSS's requirements](http://joss.theoj.org/about#paper_structure) with:
-
-- [ ] **A short summary** describing the high-level functionality of the software
-- [ ] **Authors:** A list of authors with their affiliations
-- [ ] **A statement of need** clearly stating problems the software is designed to solve and its target audience.
-- [ ] **References:** with DOIs for all those that have one (e.g. papers, datasets, software).
-
-#### Final approval (post-review)
-
-- [ ] **The author has responded to my review and made changes to my satisfaction. I recommend approving this package.**
-
-Estimated hours spent reviewing:
-
----
-
-#### Review Comments
+```{include} review-template.md
```
## Review Request Template
Editors may make use of the e-mail template below in recruiting reviewers:
-```
-Dear [REVIEWER],
-
-Hi, this is [EDITOR]. [FRIENDLY BANTER]. I'm writing to ask if you would be willing to review a package for pyOpenSci. As you probably know, pyOpenSci conducts peer review of Python packages contributed to our collection in a manner similar to journals.
-
-The package, [PACKAGE] by [AUTHOR(S)], does [FUNCTION]. You can find it on GitHub here: [REPO LINK]. We conduct our open review process via GitHub as well, here: [ONBOARDING ISSUE]
-
-If you accept, note that we ask reviewers to complete reviews in three weeks. (We’ve found it takes a similar amount of time to review a package as an academic paper.)
-
-Our [reviewers guide] details what we look for in a package review, and includes links to example reviews. Our standards are detailed in our [packaging guide], and we provide reviewer [template] for you to use. Please make sure you do not have a conflict of interest preventing you from reviewing this package. If you have questions or feedback, feel free to ask me.
-
-{IF APPLICABLE: The authors have also chosen to jointly submit their package to the Journal of Open Source Software, so this package includes a short `paper.md` manuscript describing the software that we ask you include in your review.}
-
-Are you able to review? If you can not, suggestions for alternate reviewers are always helpful. If I don't hear from you within a week, I will assume you are unable to
-review at this time.
-
-Thank you for your time.
-
-Sincerely,
-
-[EDITOR]
+```{include} reviewer-request-template.md
```
[reviewers guide]: https://www.pyopensci.org/contributing-guide/peer-review-guides/reviewer-guide.html
diff --git a/checks.md b/checks.md
deleted file mode 100644
index 44c61d4a..00000000
--- a/checks.md
+++ /dev/null
@@ -1,15 +0,0 @@
-- [ ] **Fit**: The package meets criteria for [fit](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-scope) and [overlap](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-overlap).
-- [ ] **Documentation** The package has sufficient documentation available online (README, sphinx docs) to allow us to evaluate package function and scope *without installing the package*. This includes:
- - [ ] Get started vignettes or tutorials that help a user understand how to use the package and what it can do for them
- - [ ] API documentation - this includes clearly written doc strings with variables defined using a standard docstring format
- - [ ] README that clearly articulates the function of the package
- - [ ] Contributing documentation that details how to install a development environment and how to contribute to the package
-- [ ] **Issue Header Information**: All of the required issue header information is contained within the top of the issue.
-- [ ] **Automated tests:** Package has a testing suite and is tested via GitHub actions or another Continuous Integration service.
-- [ ] **License:** The package has an OSI accepted license
-- [ ] **Repository:** The repository link resolves correctly
-- [ ] **README:** The package has a README with clear explanation of what the packages does and instructions on how to install it along with development instructions.
-- [ ] **Contributing statement:** The package has a `contributing.md` file that details how to contribute to the package.
-- [ ] **Package overlap:** That package doesn't fully overlap with functionality of other packages that have already been submitted to pyOpenSci
-- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly
-- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
\ No newline at end of file
diff --git a/conf.py b/conf.py
index cb867026..252664fd 100644
--- a/conf.py
+++ b/conf.py
@@ -36,10 +36,12 @@
"sphinx.ext.intersphinx",
"sphinx_sitemap",
"sphinxext.opengraph",
+ "sphinx_copybutton",
]
# colon fence for card support in md
myst_enable_extensions = ["colon_fence"]
+myst_heading_anchors = 3
# For generating sitemap
html_baseurl = 'https://www.pyopensci.org/peer-review-guide/'
@@ -56,8 +58,6 @@
"announcement": "🚧 UNDER CONSTRUCTION: this guide is under heavy construction right now. 🚧"
}
-
-
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -80,7 +80,6 @@
# a list of builtin themes.
#
html_theme = 'sphinx_book_theme'
-html_static_path = ["_static"]
html_title = "pyOpenSci Peer Review Guide"
html_logo = "images/logo/logo.png"
@@ -88,4 +87,4 @@
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
-html_css_files = ["pyos.css"]
\ No newline at end of file
+html_css_files = ['pyos.css']
diff --git a/index.md b/index.md
index c1b77768..ffb0de27 100644
--- a/index.md
+++ b/index.md
@@ -123,12 +123,13 @@ Code of Conduct
:hidden:
:caption: Peer Review Guides
-Peer review process overview
+Peer Review Timeline
Author Guide
Reviewer Guide
Editor Guide
Editor in Chief Guide
-Onboarding Guide
+Onboarding Editors & Reviewers
+Finding Reviewers
```
```{toctree}
diff --git a/software-peer-review-guide/author-guide.md b/software-peer-review-guide/author-guide.md
index 99e0f044..37810ed4 100644
--- a/software-peer-review-guide/author-guide.md
+++ b/software-peer-review-guide/author-guide.md
@@ -1,105 +1,109 @@
# pyOpenSci Review Guide for Python Open Source Package Maintainers and Authors
Are you considering submitting a package for review with pyOpenSci? You've
-come to the right place!
-
-Below you will find the steps that you need to follow to submit a package
+come to the right place! Below you will find the steps that you need to follow to submit a package
to pyOpenSci for peer review.
Before you begin this process, [please be sure to read the review process guidelines](../about-peer-review/policies-guidelines).
+
+```{note}
+**Before you consider submitting to us please consider the following:**
+
+1. Please be sure that you have time to devote to making changes to your
+package. During review, you would get feedback from an editor and two reviewers. Changes could
+take time. Please consider this before submitting to us. You can read more about the timeline to make changes in our [peer review policies page](../about-peer-review/policies-guidelines).
+2. Peer review is lead by a diverse group of volunteer editors and reviewers.
+Please be considerate when engaging with everyone online.
+```
+
## 1. Do you plan to continue to maintain your package?
One of the goals of pyOpenSci is to maintain a curated list of
community-approved, maintained and vetted tools that support open science workflows.
-As such we review packages that will be useful to the community
+As such, we review packages that will be useful to the community
and maintained over time. While we understand that burnout is real,
and you may move on in the future to other projects, we ask that you commit
-to maintaining your package for at least 1-2 years.
+to maintaining your package for at least 1-2 years after the review process
+is complete.
If that maintenance commitment is too much, you might consider submitting
-your package to another Journal.
+your package to a journal that is more focused on publication only.
-If you need to step from maintaining your package, please let us know
-in advance. We will try to help you either.
-* Find a new maintainer to take over your project or
-* Sunset your package.
-
-If the package is sunsetted we will remove it from our curated list
-of vetted tools.
### Who should submit the package for review?
If you have a team of people maintaining your package, please be sure
-that the submitting author is the person who "owns" or leads that maintenance. That person will become the long term point of contact
-for pyOpenSci.
+that the submitting author is the person who "owns" or leads that maintenance.
+That person will become the long term point of contact
+for pyOpenSci.
+* Please also include the names of all maintainers on the project
+as we also want to ensure that everyone working on the project gets full credit
+for their effort.
+
```{note}
If your package is more of a tool to support a specific workflow that
-either may not be maintained long term or that may be so specific that
-it won't have a broad user base, you might consider submitting it
-directly to the Journal of Open Source Software (JOSS).
+either:
+* may not be maintained long term or
+* may be so specific that it won't have a user base outside of your lab or work environment
+
+please consider submitting it directly to a publisher like the
+Journal of Open Source Software (JOSS). A publisher like JOSS has less
+emphasis on long term software maintenance and focuses more on
+publication quality and citation / credit.
```
-
## 2. Does Your Package Meet Packaging Requirements?
Before submitting your project for review with pyOpenSci, make sure that it
follows the standards and guidelines outlined in the
-[Python packaging guide](../authoring/index).
+[pyOpenSci Python packaging guide](https://www.pyopensci.org/python-package-guide).
```{note}
**Package Prep Help - Can we actually support this now??**
If you would like help preparing your package for review, create
-a [Help Request issue](https://github.com/pyOpenSci/software-review/issues/new/choose)
-in the software-review repo. We can assign someone to help you prep your code
-and add things like testing, documentation, and/or continuous integration. We're
-happy to help. Also check out the rest of our [Packaging Guide](../authoring/overview), which may help answer some of your questions about packaging your code.
+a [new help request post in our Discourse forum under `coding-help`](https://pyopensci.discourse.group/c/coding-help/10). We can find someone to help you prep your code
+and add things like testing, documentation, and/or continuous integration.
+
+Also check out the rest of our [Packaging Guide](https://www.pyopensci.org/python-package-guide), which may help answer some of your questions about packaging your code.
+
+**NOTE: we are currently updating our packaging guide and intend to add
+tutorials on Python packaging. Please expect these resources to be online sometime by Spring, 2023.**
```
## 3. Is Your Package in Scope for pyOpenSci?
-Next, check to see if your package falls into the scope of pyOpenSci. If you aren't
+Next, check to see if your package falls into the topical and technical scope of pyOpenSci. If you aren't
sure about whether your package fits within pyOpenSci's scope (below), submit
a [presubmission inquiry issue on the software-review](https://github.com/pyOpenSci/software-review/issues/new?assignees=&labels=0%2Fpresubmission&template=presubmission-inquiry.md&title=)
repository. After you submit an inquiry, a pyOpenSci editor will get back to you
with input regarding the fit of your package for pyOpenSci review. This can take
up to a week.
-```{include} ../scope.md
-```
-
-```{note}
-**Before you consider submitting to us please consider the following:**
+Our current categories for determining package scope are below:
-1. Please be sure that you have time to devote to making changes to your
-package. You may get feedback from an editor and two reviewers. Changes could
-take time. Please consider this before submitting to us. You can read more about the timeline to make changes in our [peer review policies page](../about-peer-review/policies-guidelines).
-2. Peer review is lead by a diverse group of volunteer editors and reviewers.
-Please be considerate when engaging with everyone online.
+```{include} ../appendices/scope.md
```
-## 4. Presubmission Questions
-
-If your package does not clearly fit within one of the categories outlined in
-our [scope](../about-peer-review/aims-and-scope), create
-a [Presubmission Inquiry issue](https://github.com/pyOpenSci/software-review/issues/new?assignees=&labels=0%2Fpresubmission&template=presubmission-inquiry.md&title=)
-in our software-review repo. You can use the same issue template if you have
-other questions. An editor will get back to you in a few days to answer your
-questions and to help determine the fit.
-
-## 5. Submit Your Package for Peer Review
+## 4. Submit Your Package for Peer Review
To submit your package for peer review, you can
open an issue in our [pyopensci/software-review repo](https://github.com/pyOpenSci/software-review/issues/new/choose/)
repository and fill out the [Submit Software for Review](https://github.com/pyOpenSci/software-review/issues/new?assignees=&labels=1%2Feditor-checks%2C+New+Submission%21&template=submit-software-for-review.md&title=) issue template.
-## 6. Editor in Chief Reviews Package for Scope and Minimal Infrastructure Criteria
-Once the issue is opened, an editor will review your submission within
+## 5. Editor in Chief Reviews Package for Scope and Minimal Infrastructure Criteria
+Once the issue is opened, our editor-in-chief and an editor from our editorial board will review your submission within
**2 weeks** and respond with next steps. The editor may request that you make updates
to your package to meet minimal criteria before review. They also may reject your
package if it does not fall within our scope.
-## 7. The Review Begins
+```{button-link} https://www.pyopensci.org/software-peer-review-guide/editor-in-chief-guide.html#editor-checklist-template
+:color: secondary
+:class: sd-rounded-pill
+Click here to view the editor checks that will be used to evaluate your package.
+```
+
+## 6. The Review Begins
If your package meets minimal criteria for being
reviewed it may then be given to an editor with appropriate domain experience
to run the review. That editor will assign 2-3 reviewers to review your
@@ -108,13 +112,14 @@ issue within **3 weeks**. Reviewers also can open issues in your package reposit
We prefer issues that link back to the review as they document changes made to your
package that were triggered by our review process.
-## 8. Response to Reviews
+## 7. Response to Reviews
You should respond to reviewers’ comments within **2 weeks** of the
last-submitted review. You can make updates to your package at any time. We
encourage ongoing conversations between authors and reviewers. See the
-reviewing guide for more details.
+[guide for package reviewers](reviewer-guide.md) for more details about how reviewers engage with package
+maintainers during a review.
-## 9. Acceptance into pyOpenSci
+## 8. Acceptance into pyOpenSci
Once the reviewers are happy with changes that you've made to the package, the
editor will review everything and accept your package into the pyOpenSci ecosystem.
Congratulations! You are almost done!
@@ -128,13 +133,15 @@ Once your package is approved, a few things will happen:
Zenodo. You will then want to created a tagged release representing the version of the
package accepted by pyOpenSci.
2 We will ask you to add the pyOpenSci badge [](https://github.com/pyOpenSci/software-review/issues/issue-number) to the
-top of your README.md file.
+top of your **README.md** file.
1. We will ask you to add your package to the [pyOpenSci website](https://www.pyopensci.org/our-community/). This will give
your package more visibility in the community as a vetted pyOpenSci tool.
- * We also will ask you (and those who reviewed your package) to add yourself to our list of contributors as a maintainer!
-1. We will promote your package on our twitter channel!
-2. If you wish to go on to submit your package to JOSS, this is when you'd start
-that process.
+ * We also will ask you (and those who reviewed your package) to add yourself to our list of [pyOpenSci community members](https://www.pyopensci.org/our-community/)!
+1. We will promote your package on our social media channels!
+1. We will invite you to write a blog on our website spotlighting your package. The blogs that our maintainers write are some of the most popular content on the website!
+
+
+If you wish to go on to submit your package to JOSS, you can do so now. Remember that JOSS will accept our review as theirs so you DO NOT need to go through another review. Read more below.
### Journal of Open Source Software (JOSS) Submission
@@ -147,11 +154,10 @@ will evaluated by JOSS through the pyOpenSci review
- To complete the JOSS submission, you will also need to craft a **paper.md**
file describing the package following JOSS' standards (see below). More detail on the requirements for JOSS can be found on [their website](https://joss.readthedocs.io/en/latest/submitting.html#what-should-my-paper-contain).
- If you choose to opt into the pyOpenSci / JOSS partnership in your review,
-you do NOT need to go through a second review with JOSS. JOSS accepts our review
-for theirs. Please just start a review process with JOSS and reference the pyOpenSci
-review issue that you have been working within for your review with us. Make sure
-that you let the JOSS editor know that we have already accepted you. They will use
-our review for their process and fast track you through their review!
+you DO NOT need to go through a second review with JOSS. JOSS accepts our review
+for theirs. Please start a review process with JOSS and reference the pyOpenSci
+review issue where your package was accepted. Make sure
+that you let the JOSS editor know that we have already accepted your package. The JOSS editor will review your paper and once that is accepted you now have a JOSS DOI and badge to display on your README file as well!
```{note}
Acceptance to pyOpenSci does not guarantee acceptance to JOSS. In particular,
@@ -162,3 +168,24 @@ are usually not accepted by JOSS. Be sure to review JOSS's
[submission requirements](https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements)
before writing up a paper about your package.
```
+
+## Post review - now what?
+
+Once you have been accepted into the pyOpenSci ecosystem, you will
+be invited to join our Slack channel.
+
+We also will keep in touch with you, periodically checking in to ensure that package maintenance is going well and to better understand ways in which pyOpenSci can support you.
+
+If at any time, you need to step down from maintaining your package,
+or you need help with maintenance, please let us know - preferably
+in advance. We will try to help you by either:
+
+* Find a new maintainer to take over your project (or additional maintainers to support maintenance) or
+* Sunset your package.
+
+If the package is sunsetted, we will remove it from our curated list
+of vetted tools.
+
+### Communication with pyOpenSci and removing tools from our vetted tool list
+
+To ensure packages that we support and advocate for are maintained, if your package is accepted and we are not able to get in touch with you through normal communication channels (GitHub, email) after reaching our for at least 1-2 months, we will remove your package from our list of vetted tools. We will also remove any blogs written that highlight your tool.
diff --git a/software-peer-review-guide/editor-in-chief-guide.md b/software-peer-review-guide/editor-in-chief-guide.md
index f8f90677..3412c046 100644
--- a/software-peer-review-guide/editor-in-chief-guide.md
+++ b/software-peer-review-guide/editor-in-chief-guide.md
@@ -40,93 +40,81 @@ the scope and readiness of a package to be reviewed, they may opt to assign an e
When a new package is submitted for review, the Editor in Chief will:
-### 1. ✅ Tag the issue with `1/editor-checks` tag in GitHub
+### 1. ✔️ Tag the issue with `1/editor-checks` tag in GitHub
-### 2. ✅ Next, they will use the {ref}`template ` below in the issue
+### 2. ✔️ Add the editor checks to the issue
-This step serves to check whether the package has
-the bare minimum requirements to initiate a review (or they will assign that task to an editor as stated above).
-These checks ensure that the package is ready to be reviewed.
+```{important}
+It is important that this step occur in one post rather than a string of
+conversational feedback that is more difficult to follow. This allows the author to address all issues at
+one time. Thus the EIC should:
-Following this step will ensure that we are using our volunteer reviewer time effectively.
+1. Review the checklist.
+1. Give the author a rough estimate of how long the checks might take to complete.
+1. Perform all of the checks locally.
+1. When all of the above are complete, post the checklist with any feedback for the author in the issue. This should be one single post.
+```
+
+Copy the {ref}`template below `
+and add it to the issue.
-In some situations, these checks may be passed down to an editor including:
+Editor checks ensure that the package has
+the bare minimum requirements to initiate a review.
+We hope that even the process of going through these checks will
+improve the quality of the package.
+
+In some situations, the editor-in-chief initial checks may be passed down to an editor as follows:
-* If the Editor in Chief is overwhelmed with package submissions to evaluate
-* If the Editor in Chief simply is busy at the time and needs support with checks
+* If the Editor in Chief is overwhelmed with package submissions to evaluate.
+* If the Editor in Chief simply is busy at the time and needs support with checks.
* If the Editor in Chief thinks that the checks might be better served if done by an Editor
-(For instance if a specific domain or technical expertise would support more effective checks)
-
+(For instance, if a specific domain or technical expertise would support more effective checks).
(editor-checklist-template)=
-### Editor checklist (copy template below to use in the issue)
-
-```markdown
-## Editor in Chief checks
-
-Hi there! Thank you for submitting your package for pyOpenSci
-review. Below are the basic checks that your package needs to pass
-to begin our review. If some of these are missing, we will ask you
-to work on them before the review process begins.
-
-- [ ] **Fit**: The package meets criteria for [fit](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-scope) and [overlap](https://www.pyopensci.org/peer-review-guide/about-peer-review/aims-and-scope.html#package-overlap).
-- [ ] **Documentation** The package has sufficient documentation available online (README, sphinx docs) to allow us to evaluate package function and scope *without installing the package*. This includes:
- - [ ] tutorials or vignettes that help a user understand how to use the package and what it can do for them (often these have a name like "Getting started")
- - [ ] API documentation - this includes clearly written doc strings with variables defined using a standard docstring format
- - [ ] README that clearly articulates the function of the package
- - [ ] Contributing documentation that details how to install a development environment and how to contribute to the package
-- [ ] **Issue Submission Documentation**: All of the information is filled out in the `YAML` header of the issue (located at the top of the issue template).
-- [ ] **Automated tests:** Package has a testing suite and is tested via GitHub actions or another Continuous Integration service.
-- [ ] **License:** The package has an [OSI approved license](https://opensource.org/licenses)
-- [ ] **Repository:** The repository link resolves correctly
-- [ ] **README:** The package has a README with clear explanation of what the package does and instructions on how to install it along with development instructions.
-- [ ] **Contributing statement:** The package has a contributing.md file that details how to contribute to the package.
-- [ ] **Package overlap:** That package doesn't fully overlap with functionality of other packages that have already been submitted to pyOpenSci
-- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly
-- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
-
----
-- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
-We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
-Thank you authors in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
----
-
-*******
-
-## Editor comments
+### Editor-in-chief checklist
+Copy the template below to use it in the issue.
+```{include} ../appendices/editor-in-chief-checks.md
```
-### 3. ✅ Finally they will ensure the onboarding survey is filled out.
+### 3. ✔️ Ensure that the package onboarding survey is filled out.
-Thank the authors graciously for doing this. They can skip sections of it if they wish, but we do need their contact information and we do want to track their experience with our review process and organization.
+Thank the authors graciously for filling out our survey. They can
+skip sections of it if they wish. We do need their contact
+information to stay in touch about package maintenance. We also
+want to track their experience with our review process and
+organization.
-### 4. ✅ Assign an editor to the issue to manage the rest of the review
+### 4. ✔️ Assign an editor to the issue to manage the rest of the review
Once the package initial checks are complete, and it is determined that
the package is in scope for pyOpenSci review, the Editor in Chief will assign an
editor to the review issue. The editor will begin the process of finding reviewers
for the package. [Check out the editor guide for more information on this process](editors-guide.md)
-### 5. ✅ Update the YAML header of the issue
+### 5. ✔️ Update the YAML header of the issue
Once all of the above is complete, the Editor in Chief should:
-- Add any comments to the bottom of your editor checks comment
-- Update the yaml in the header of the issue with the editor assigned to the review
-- Add the tag `2/seeking-reviewer(s)` to the issue.
+* Add any comments to the bottom of your editor checks comment.
+* Update the YAML in the issue's header with the editor assigned to the review.
+* Add the tag `2/seeking-reviewer(s)` to the issue.
+
+## A note about submissions that are incomplete or vague
+
+In some cases:
-## A Note about submissions that are incomplete or vague
+* Online documentation of a package is sparse.
+* README is minimal or hard to understand.
+* No clear documentation setup.
+* Elements of the YAML template at the top of the issue are not filled out.
-In some cases online documentation of a package is sparse, the README is
-minimal or hard to understand or there is no clear documentation setup. Or
-in other cases some of the YAML at the top of the issue is missing.
This makes assessment of the package's scope much harder.
-In this case, please ask for more information. Even if the package is deemed
+In this case, please ask the author for more information. Even if the package is deemed
out-of-scope, the package documentation will improve as a result of your questions.
-Example text
+Example text:
```markdown
Hello and many thanks for your submission.
@@ -134,7 +122,7 @@ Hello and many thanks for your submission.
We are discussing whether the package is in scope and need a bit more
information.
-Would you mind adding more details and context to your `README` file?
+Please add more details and context to your `README` file.
After reading it, someone with little domain knowledge should understand
the aim, goals and functionality of the package.
@@ -145,8 +133,9 @@ to mention in your documentation (README) and in this issue [how it is "best in
```
-### Responding to out-of-scope submissions
+## Responding to out-of-scope submissions
-If the package is determined to be out-of-scope, the Editor in Chief should thank authors for their submission,
-explain the reasons for the decision, and direct them to other publication venues
-if relevant. If further discussion is warranted that can take place within the issue.
\ No newline at end of file
+If the package is determined to be out-of-scope, the Editor in Chief should
+thank authors for their submission, explain the reasons for the decision, and
+direct them to other publication venues if relevant. If further discussion is
+warranted that can take place within the issue.
\ No newline at end of file
diff --git a/software-peer-review-guide/editors-guide.md b/software-peer-review-guide/editors-guide.md
index 80447af8..e6026bc2 100644
--- a/software-peer-review-guide/editors-guide.md
+++ b/software-peer-review-guide/editors-guide.md
@@ -1,28 +1,39 @@
-# Guide for Editors
+# pyOpenSci Guide for Peer Review Editors
+
+
Thank you for your time in serving as an editor for a PyOpenSci package! Below you will find some
information about the role that editors have in the
-pyOpenSci Python open source peer review process.
+pyOpenSci Python open peer review process.
+
+## Experience needed to become an editor
Editors generally should:
* Have completed a review for *at least* 1 package for pyOpenSci.
* Have some experience with open source software that supports the scientific
-Python community, be it maintaining or contributing to packages or being involved
-in the community in some way.
+Python community. This experience could be maintaining or contributing to packages. It also could be experience related to usability of open source software and/or documentation, tutorials etc. Or finally it could be involvement in the scientific Python community in some other way.
-There are two types of editors in our peer review process:
+## Two types of editors
+There are two types of editors involved in our open peer review process:
* Guest editors and
* Full editors
Both types of editors are considered a part of the editorial board for
-pyOpenSci. The major difference between guest and full editors is the time
-that they may or may not serve on the editorial board. And the support that
-they may need in serving as editor.
+pyOpenSci. The major difference between guest and full editors is:
+* A guest editor may only join the board for a single review.
+* A guest editor may be new to pyOpenSci's review process and thus require a bit more support in their first review.
-## Guest Editors
+### Guest editors
A guest editor is is invited to lead a review in the
case where we need specific expertise for a single review. We also consider editors who are
performing their first review as guest editors as they may require more
@@ -32,19 +43,19 @@ New editors who wish to continue on as full editors for pyOpenSci may do so
as long as both parties (pyOpenSci and the guest editors) feel like it is a
healthy fit for them and the organization.
-## Editors and the Editorial Board
+### "Full" editors
-A full editor most often someone who has experience with the
+A full editor is most often someone who has experience with the
pyOpenSci open package review process. A full editor ideally:
-* has completed a review for *atleast* 1 package for pyOpenSci
+* has completed a review for *at least* 1 package for pyOpenSci
* and/or has submitted and gone through the pyOpenSci package review process
* and/or has experience reviewing for an organization such as JOSS or rOpenSci.
-We also appreciate when editors have some experience working with or in the
-Python open source software community be it maintaining or contributing to
-packages, or being involved in the community in some way. This is certainly
-now a required however if you are interested in getting involved through
+We also appreciate when editors have experience working with or in the
+Python open source software community be it maintaining packages, contributing to
+packages, or supporting the community. This is certainly
+not a requirement however if you are interested in getting involved with
pyOpenSci!
```{note}
@@ -52,7 +63,6 @@ There could be certain situations when an editor is on boarded with less experie
```
## What does an editor do? (Responsibilities)
-
An editor is normally recruited by the Editor in Chief, other editors on the
board, or the software review lead. [More on recruiting editors can be found here](onboarding-guide.md).
@@ -67,16 +77,16 @@ on package review are generally made in the private `editorial-board` channel in
are comfortable with checking Slack regularly.
```
-### Supporting other reviews
-While editors are not charged with tracking other submissions that they are
-not leading, if you do notice an issue with another review, please feel free
-to raise that issue either directly with the editor for that review OR raise
-the issue in the editors channel in slack.
-
+## Editor support of other reviews
+Editors are not charged with tracking other submissions that they are
+not leading. However, if you are serving as an editor and notice an
+issue with another review, please raise that issue either directly with
+the editor for that review. Or you can raise the issue in the editors
+channel in slack.
-### Editor in chief rotation
+## Editor-in-Chief rotation
The editorial board normally participates in the Editor in Chief rotation.
-You are eligible to enter this rotation after 3 months of serving on the board
+You are eligible to enter this rotation after 3 months of serving on the editorial board
and/or after your first review as it makes sense. [Read more about the roles
and responsibilities of the Editor in Chief, here.](editor-in-chief-guide.md)
@@ -96,203 +106,203 @@ editors staying on for longer as long as they are happy serving with us and
they get along well with other members of the editorial board, the software
review lead and the current Editor in Chief.
-
-## Get Started With Leading a Package Review -- Checklist
+## Editor checklist: Get Started With Leading a Package Review
Follow the checklist below when serving as an editor for a package submitted to
pyOpenSci for review.
-### ✅ 1. First, tag the submission issue on GitHub
+All reviews happen in GitHub issues. The template for the
+`yaml` header of a review submission below will be
+referenced multiple times in the steps below:
+
+```{include} ../appendices/issue-review-template.md
+```
+
+### ✔️ 1. First, tag the submission issue on GitHub
Once you begin the review process as an editor:
-* Tag the submission issue with the `2/seeking-reviewer(s)` tag.
-* Check the submission by the author to ensure that mandatory parts of the template are complete.
+* Tag the submitted GitHub issue with the `1/editor-checks` tag if it hasn't already been tagged by the editor-in-chief.
+* Check the YAML template at top of the submitted GitHub issue, make sure that mandatory parts of the template are filled out.
- If elements are incomplete, direct the authors toward filling in any missing pieces.
-### ✅ 2. Respond to the submitter in the GitHub issue
+```markdown
+Submitting Author: SUBMITTING AUTHOR NAME HERE (Name @github_handle)
+All current maintainers: ALL CURRENT MAINTAINERS LISTED HERE (Name @github_handle1, Name @github_handle2)
+Package Name: PACKAGE-NAME-HERE
+One-Line Description of Package: DESCRIPTION OF THE PACKAGE HERE
+Repository Link: REPO-LINK
+Version submitted: VERSION-SUBMITTED
+```
+
+```{note}
+The editor in chief who initially engaged with this review should have already evaluated the package level Editor Checks section for `Fit`, `Automated Tests`, `Documentation`, `License`, and `Repository`.
-Once the above is complete, you are ready to add editor checks to the issue.
-The goal of this step is to ensure that the package is ready to be reviewed.
-Following this step will ensure that we are using our volunteer
-reviewer time effectively.
+They also should have checked whether the package is [in scope for pyOpenSci](../about-peer-review/aims-and-scope.html#package-categories).
+And whether there is [functionality overlap with functionality of any other existing Python packages](../about-peer-review/aims-and-scope.html#package-overlap).
-- Add a comment to the issue that contains a copy of the Editor Checks template (see below) filled out with your response to the checks that begin the review.
+However, in some instances the editor-in-chief may request that an editor
+perform these tasks. Be sure to check the issue to ensure the above checks have been implemented prior to initiating the review.
-- In this comment, you will add reviewers and the review deadline date once you have reviewers assigned (see below).
+If the package does not fit the pyOpenSci scope and policies and needs to be
+rejected, see
+[this section in the editor in chief guide](editor-in-chief-guide.md#responding-to-out-of-scope-submissions)
+about how to respond.
+```
+### ✔️ 2. Respond to the submitter in the GitHub issue
+Once the above is complete, you are ready to add an editor response to the issue.
+This step ensures that the package is ready to be reviewed. It also ensures that
+we are using our volunteer reviewer time effectively.
-```markdown
-## Editor checks:
+* Add a comment to the issue that contains a copy of the Editor Response template (see below) filled out with your response to the checks that begin the review.
+* Change the label of the issue to `2/seeking-reviewer(s)`
----
+```{include} ../appendices/editor-initial-response-template.md
+```
-## Editor comments
-:wave: Hi @reviewer-one and @reviewer-two! Thank you for volunteering to review
-for PyOpenSci!
-The following resources will help you complete your review:
+```{important}
+**Important: If in the initial checks you find any major gaps in the package's
+structure, request changes before assigning reviewers.**
+```
-1. Here is the **[reviewers guide](https://www.pyopensci.org/contributing-guide/peer-review-guides/reviewer-guide.html)**. This guide contains all of the steps and information needed to complete your review.
-2. Here is the **[review template](https://www.pyopensci.org/contributing-guide/appendices/templates.html#review-template)** that you will need to fill out and submit
-here as a comment, once your review is complete.
+### ✔️ 3. Identify Python package reviewers
-Please get in touch with any questions or concerns! Your review is due:
----
+Within **one week of responding to the issue as the editor**, identify two people to
+review the Python package.
-Reviewers:
-Due date:
+If you wish, you can use the email template below to invite reviewers.
+
+```{include} ../appendices/reviewer-request-template.md
```
-- Fill out the Editor Checks section for `Fit`, `Automated Tests`, `Documentation`, `License`, and `Repository`.
-- Check against policies for [package fit within identified categories for the pyOpenSci ecosystem](../about-peer-review/aims-and-scope.html#package-categories).
-- Check against policies for [package overlap of functionality with other packages](../about-peer-review/aims-and-scope.html#package-overlap).
+When inviting reviewers, include something like "if I don't hear from
+you in a week, I'll assume you are unable to review," so as to give a clear
+deadline when you'll move on to looking for someone else to keep the processing
+moving.
-If the package does not fit the pyOpenSci scope and policies and needs to be
-rejected, see
-[this section in the editor in chief guide](editor-in-chief-guide.html#responding-to-out-of-scope-submissions)
-about how to respond.
+* Once you have assigned reviewers to the review, you will update the editor response above with:
-```{note}
-PyOpenSci has a partnership with JOSS where packages that are in-scope for JOSS
-can be directly accepted into the JOSS ecosystem through the pyOpenSci review.
-The JOSS component of the review happens after all of the review on the
-pyOpenSci side is complete and it begins through direct communication with a
-JOSS editor.
-```
+1) reviewer GitHub handles and
+2) the review deadline date.
+* Change the label on the issue to `3/reviewer(s)-assigned`
-- `Archive` and `Version` within the editor checks for JOSS may be filled out at the end of the review.
- * `Archive` refers to an archive created through a release. You can use zenodo to
- create this archive and provide the package with a citable DOI. If zenodo is used, please add the zenodo link here.
- * `Version` refers to the final package version that was accepted by pyOpenSci.
- This is the final version as presented after all feedback from the reviews has
- been considered and implemented.
+```{note}
+[Click here for more information about finding reviewers for a package.](where-to-find-python-package-reviewers)
+```
-**Important: If initial checks show major gaps, request changes before assigning reviewers.**
+### ✔️ 4. Onboard reviewers
-### 3. Identify Reviewers
+Once reviewers have been identified:
-Within **one week of completing the editor checks**, identify two reviewers for
-the package.
+* Tag issue with `3/reviewer(s)-assigned` tag.
+* Add reviewer names and review due date to the Editor Comment that you left above in step 2.
+* Also add the reviewer names to the YAML template at the very top of the issue.
-If you wish, you can use the [email template](../appendices/templates#review-request-template) to invite reviewers. When inviting reviewers, include something like "if I don't hear from
-you in a week, I'll assume you are unable to review," so as to give a clear
-deadline when you'll move on to looking for someone else to keep the processing
-moving.
+```markdown
+Editor: Full Name (@github_username)
+Reviewer 1: Full Name (@github_username)
+Reviewer 2: Full Name (@github_username)
+Archive: Filled out when the review is complete.
+Version accepted: Filled out when the review is complete.
+```
-#### Where to Look for Reviewers?
+## Editor responsibilities during the review
-As a (guest) editor, you can find reviewers through:
-* Suggestions made by the submitter(s) (although submitters may have a narrow view of the types of expertise needed. We suggest not using more than one of suggested reviewers).
-* Authors of existing [pyOpenSci packages](https://www.pyopensci.org/python-packages/).
-* Other [contributors to pyOpenSci](https://www.pyopensci.org/our-community/).
+During the review process, it is important to check in with the reviewers to
+ensure that things are moving smoothly:
-When these sources of information are not enough:
-* Ping other editors for ideas.
-* Look for users of the package or the data source/upstream service the package connects to (via their opening issues in the repository, starring it, citing it in papers, talking about it on Twitter).
-* You can also search for authors/maintainers of related packages on [PyPI](https://pypi.org/search/).
-* Post on Twitter and ensure pyOpenSci retweets your post.
+- Check in with reviewers and authors occasionally. Offer clarification and help as needed.
+- In general aim for 3 weeks for review, 2 weeks for subsequent changes, and 1 week for reviewer approval of changes.
+- If a review has not been submitted after 2 weeks, ping the reviewer(s) within the review issue to ensure they are aware of the 3 week deadline.
-#### Criteria for Choosing Reviewers
+### ✔️ 5. What to do when reviews are in
-Here are criteria to keep in mind when choosing a reviewer. You might need to
-piece this information together by searching `PyPI`, `Conda` / `Conda-forge` and
-the potential reviewer’s GitHub page and general online presence (personal
-website, Twitter).
+* Once all reviews are submitted, change the review status tag to `4/review-in-awaiting-changes`.
-* Has not reviewed a package for us within the last 6 months.
-* Some package development / contribution experience.
-* Some domain experience in the field of the package or data source.
-* No [conflicts of interest](../about-peer-review/policies-and-guidelines.html#conflict-of-interest).
+If the author stops responding, refer to [the policies](peer_review_proc#review-process-guidelines) and/or ping the other editors in the Slack channel <*Not available publically yet*> for discussion.
-Try to balance your sense of the potential reviewer’s experience against the complexity of the package.
+Once the author has responded to the reviews and made appropriate changes made changes:
-* **Diversity** - if you have two reviewers both shouldn’t be cis white males.
-* **Openness** - reviewers should also have demonstrated interest in open source or Python community activities, although blind emailing is fine.
+* Briefly check to ensure that the changes were indeed made.
+* Change the issue status tag to `5/awaiting-reviewer-response`.
-Each submission should be reviewed by _two_ package reviewers. Although it is
-fine for one of them to have less package development experience and more domain
-knowledge, the review should not be split into two parts. Both reviewers need to review
-the package comprehensively, from their particular perspectives. In
-general, at least one reviewer should have prior reviewing experience, and of
-course inviting one new reviewer expands our pool of reviewers.
+```{important}
+If for some reason during the process you need to halt the review and close the issue, be sure to let all parties know (maintainer and reviewers) prior to closing the issue.
-Reviewers should ideally have some subject matter expertise associated
-with the package functionality. It is ok and even welcome if one reviewer has
-more technical expertise and the other focuses on usability and is less technical.
-Read through the Guidelines for Reviewers Section to learn more about finding and
-selecting reviewers.
+Be sure to:
-```{note}
-PyOpenSci has been piloting a new reviewer mentorship program where we pair a
-new reviewer with someone in the community with previous review experience. If
-a new reviewer is interested in this, get in touch with the **editor in chief**.
+* explain why the decision was made
+* thank them for their work
+* make a note to assign the reviewer to another submission with high chance of smooth
+review next time (e.g. a package author who has already submitted packages to us).
```
+## Putting a review on hold & handling non-responsive authors
-### 4. Onboard Reviewers
+In some cases, an author may need more time to respond to
+review comments. In this case, the author can choose to have
+their submission put on hold. As an editor you should apply the `holding label` to the GitHub issue.
-Once reviewers have been identified:
+The holding status will be revisited every 3 months.
-- Modify the Editor Comments under Editor Checks to add reviewer names and assign due date (typically a 2-3 week turn around).
-- Also add the reviewers to the initial top comment in the issue.
+After one year the issue will be closed if there is no movement towards responding to reviews by the author.
+If the author simply not responding, the editor should:
-```markdown
-Reviewers: Full Name (@github_username) and Full Name (@github_username)
-Due date: Date`
-Include in your comment to the reviewers:
- - Link to the reviewer guide for reviewers
- - Link to the review template
-```
+* Tag the author (`@author-github-handle`) in an issue comment notifying them that we will close ths issue in one month if there is no response.
+* Close the issue if one month has passed without a reply.
-- Edit the original comment submitted by author to fill in the Editor and Reviewer Information:
+If needed you can also chose to email the author. using the email provided in the package metadata file of the package. The email
+could be in any of the three files in the package: `setup.py`, `pyproject.toml` (preferred) or `setup.cfg`.
-```markdown
-Editor: Full Name (@github_username)
-Reviewer 1: Full Name (@github_username)
-Reviewer 2: Full Name (@github_username)
-Archive: Filled out when the review is complete.
-Version accepted: Filled out when the review is complete.
-```
+```{important}
+If a submission is closed and the author wishes to re-submit, they will have to start a new submission.
-- Tag issue with `3/reviewer(s)-assigned` tag.
+If the package is still in scope, the author will have to
+respond to the first round of reviews first. After that is
+complete, you can begin looking for new reviewers to
+evaluated the package. This ensures that none of the review
+energy spent in the first review, goes to waste.
+```
+### ✔️ 6. How to accept a package into the pyOpenSci ecosystem
-## Editor Responsibilities During the Review:
+Once the package has been accepted through the review process:
-During the review process, it is important to check in with the reviewers to
-ensure that things are moving smoothly:
+* Use the approval template below in the issue. This template
+asks the authors and reviewers to add themselves and the package that was
+approved to the pyOpenSci website. They can perform this step regardless of the
+JOSS submission process.
-- Check in with reviewers and authors occasionally. Offer clarification and help as needed.
-- In general aim for 3 weeks for review, 2 weeks for subsequent changes, and 1 week for reviewer approval of changes.
-- If a review has not been submitted after 2 weeks, ping the reviewer(s) within the review issue to ensure they are aware of the 3 week deadline.
-- Once all reviews are submitted, change the review status tag to `4/review-in-awaiting-changes`.
-- If the author stops responding, refer to [the policies](peer_review_proc#review-process-guidelines) and/or ping the other editors in the Slack channel <*Not available publically yet*> for discussion.
-- Upon changes being made, change the review status tag to `5/awaiting-reviewer-response`.
+```{include} ../appendices/package-approval-template.md
+```
-::::{important}
-If a reviewer was assigned to a closed issue, contact them when closing the
-issue to explain the decision, thank them once again for their work, and make a
-note in our database to assign them to a submission with high chance of smooth
-software review next time (e.g. a package author who has already submitted packages to us).
-::::
+* Change the status tag of the issue to `6/pyOS-approved6 🚀🚀🚀`.
+* Update the YAML at the top of the issue with the version of the package that was approved, Zenodo DOI for the approved version and the date approved.
-### General Review guidelines
+```markdown
+Archive: UPDATE-THIS-WITH-A-ZENODO-ARCHIVE-BADGE-TBD
+Version accepted: UPDATE-THIS-TBD
+Date accepted (month/day/year): UPDATE-THIS-TBD
+```
-- For packages needing continuous integration on multiple platforms ([criteria in this section of the packaging chapter](../authoring/overview#continuous-integration)), make sure the package gets tested on multiple platforms (e.g. MAC, Windows, Linux).
-- Wherever possible when asking for changes in the review process, direct authors to automatic tools and online resources.
- - If the package raises a new issue for pyOpenSci policy, create an issue on [pyOpenSci's governance repo](https://github.com/pyOpenSci/governance).
+```{note}
+* `Archive` refers to an archive created through a release. You can use zenodo to create this archive and provide the package with a citable DOI. If zenodo is used, please add the zenodo link and/or badge link here.
-## Editor Responsibilities After the Review:
+* `Version` refers to the final package version that was accepted by pyOpenSci.
+This is the final version as presented after all feedback from the r
+reviews has been considered and implemented
+```
-Once the package has been accepted through the review process:
-- Change the status tag of the issue to `6/pyOS-approved`.
-- Update the top of the issue with the version of the package that was approved and the DOI.
+## Closing notes about the editorial process
+* If the package raises a new issue for pyOpenSci policy, create an issue on [pyOpenSci's governance repo](https://github.com/pyOpenSci/governance).
+* If the package review raises a new issue in our peer review process, please [open an issue in our peer review guide repo.](https://github.com/pyOpenSci/peer-review-guide).
-### Instructions for Submitting to JOSS:
+## ✔️ OPTIONAL: Instructions for Submitting to JOSS
If the package fits within the JOSS Scope, once the package has been approved
by pyOpenSci:
@@ -306,72 +316,41 @@ These instructions loosely include:
2. Wait for a JOSS editor to approve the presubmission (which includes a scope check).
-::::{important}
+```{important}
The scope of packages accepted by pyOpenSci is sometimes different from those
accepted by JOSS. Not all pyOpenSci accepted packages will be accepted by JOSS.
Further, packages that have been previously published elsewhere may not be
eligible to be published with JOSS unless **significant** changes and improvements
to package functionality have been made.
-::::
+```
JOSS will accept the pyOpenSci review and direct the author to check their `paper.md` file. Once the package is accepted by JOSS,
the author will be instructed to add the JOSS DOI badge to their package **README** file.
Once the package is accepted by JOSS and the DOI badge resolves properly:
-If the package was accepted by JOSS:
-
-* Fill out `Archive` and `Version` accepted in the Editor Checks comment and at the top of the issue.
-* Fill out `Archive` and `Version accepted` in the original comment submitted by the author.
* Tag the issue with `9/joss-approved`.
-
-- You may wish to use the approval template below in the issue. This template
- asks the authors and reviewers to add themselves and the package that was
- approved to the pyOpenSci website. They can do this step regardless of the
- JOSS submission process.
-
-```
-----------------------------------------------
-🎉 has been approved by pyOpenSci! Thank you for submitting and many thanks to for reviewing this package! 😸
-
-There are a few things left to do to wrap up this submission:
-- [ ] Activate Zenodo watching the repo if you haven't already done so.
-- [ ] Tag and create a release to create a Zenodo version and DOI.
-- [ ] Add the badge for pyOpenSci peer-review to the README.md of . The badge should be `[](https://github.com/pyOpenSci/software-review/issues/issue-number)`
-- [ ] Add to the pyOpenSci website. , please open a pr to update [this file](https://github.com/pyOpenSci/pyopensci.github.io/blob/main/_data/packages.yml): to add your package and name to the list of contributors
-- [ ] if you have time and are open to being listed on our website, please add yourselves to [this file](https://github.com/pyOpenSci/pyopensci.github.io/blob/main/_data/contributors.yml) via a pr so we can list you on our website as contributors!
-
-
-It looks like you would like to submit this package to JOSS. Here are the next steps:
-
-- [ ] Login to the JOSS website and fill out the JOSS submission form using your Zenodo DOI. **When you fill out the form, be sure to mention and link to the approved pyOpenSci review.** JOSS will tag your package for expedited review if it is already pyOpenSci approved.
-- [ ] Wait for a JOSS editor to approve the presubmission (which includes a scope check)
-- [ ] Once the package is approved by JOSS, you will be given instructions by JOSS about updating the citation information in your README file.
-- [ ] When the JOSS review is complete, add a comment to your review in the pyOpenSci software-review repo that it has been approved by JOSS.
-
-🎉 Congratulations! You are now published with both JOSS and pyOpenSci! 🎉
-
-
-All -- if you have any feedback for us about the review process please feel free to share it here. We are always looking to improve our process and documentation in the [contributing-guide](https://www.pyopensci.org/contributing-guide). We have also been updating our documentation to improve the process, so all feedback is appreciated!
-```
-
-#### Last Steps Before Closing the Review Issue
+## Last Steps Before Closing the Review Issue
Once the review is complete, you can close the issue. Before doing that:
-* Be sure that the issue is tagged with `6/pyOS-approved`.
-* Followup with authors and reviewers to ensure:
+* Be sure that the issue is correctly tagged with `6/pyOS-approved` (and `9/joss-approved` if authors decided to submit to JOSS and were accepted).
+* Check the pyOpenSci website to ensure:
* The package was properly added to the [pyOpenSci website](https://www.pyopensci.org/python-packages/).
* Reviewers and maintainers are listed on the [contributors page](https://www.pyopensci.org/our-community/).
-* If the package is approved by JOSS, be sure that the issue is tagged with `7/JOSS-approved` and that the archive / DOI information is updated before closing the issue.
+ * Make sure the YAML at the top of the issue is fully filled out and up to date.
-Congratulations, you have completed a review for pyOpenSci!
+* If the package is approved by JOSS, be sure that the issue is tagged with `7/JOSS-approved` and that the archive / DOI information at the top is updated with the JOSS archive before closing the issue.
+Congratulations, you have completed a review for pyOpenSci!
-#### Optional - Move Package to PyOpenSci Organization (BETA)
+
+
+
+
+
+
+
diff --git a/software-peer-review-guide/intro.md b/software-peer-review-guide/intro.md
index 37914ea7..8f726275 100644
--- a/software-peer-review-guide/intro.md
+++ b/software-peer-review-guide/intro.md
@@ -4,28 +4,27 @@
There are several components to the pyOpenSci peer review process. Below, we overview the entire process
from start to finish.
-### Step 1 *optional*: Author submits pre-submission inquiry:*
-A **presubmission inquiry** is wise to do if you are unsure of whether your package
+### Step 0. *optional* : Author submits pre-submission inquiry
+A **presubmission inquiry** is useful if you are unsure whether your package
is in scope. To submit a pre-submission inquiry, open up an issue using the presubmission template in our [pyopensci/software-review repository](https://github.com/pyOpenSci/software-review/issues/new/choose/). During this time an Editor in Chief will review for scope and performs
basic checks for package infrastructure
- Estimated time: ~1-2 weeks
-
**These are the basic checks that your package should have prior to being submitted for peer review**
-```{include} ../checks.md
+```{include} ../appendices/pre-review-package-requirements.md
```
-### 2. Author submits their package for review
+### 1. Author submits their package for review
To do this, you open an issue using the software submission template in our
[pyopensci/software-review repository](https://github.com/pyOpenSci/software-review/issues/new/choose/).
-### 3. Editor in Chief reviews package submission
+### 2. Editor in Chief reviews package submission
The Editor in Chief will review your submission at this point for both package scope and minimal infrastructure criteria
-(listed above)
+(listed above).
- TIME ~2 weeks (or longer if editor requests changes that take the author longer to implement)
### 3. Editor finds reviewers for package
@@ -34,39 +33,43 @@ requirements and is in scope the Editor in Chief will assign an editor
to review your package. That editor will then look
for reviewers.
-Time: ~ 2-3 weeks
+Time: ~2-3 weeks
-### 4. Once we had an editor and 2 reviewers on board, the review begins
-Reviewers have 3 weeks to return a review. To do this
-they will use our reviewer template in the [reviewer guide](reviewer-guide.md) and past that, filled out, into the issue.
+### 4. Peer review of submitted Python Package begins
+Once we have an editor and 2 reviewers on board, review begins. **Reviewers have 3 weeks to return a review.** To do this
+they will use our reviewer template in the [reviewer guide](reviewer-guide.md) and paste that, filled out, into the issue.
TIME: ~3 weeks
-### 5. Author response to reviews
+### 5. Author responds to reviews
-At this point the authors should respond to the review. We prefer that authors
-respond within 2 weeks of the submitted review. We also understand that it may
+At this point the authors should respond to the review. We prefer that **authors
+respond within 2 weeks of the submitted review**. We also understand that it may
take longer to actually implement the changes requested in the review. But we
request that authors respond to reviews to acknowledge
that they have seen them.
-The reviewers are encouraged to open pull requests and issues in to help the
+The reviewers are encouraged to open pull requests and issues to help the
maintainers update their package.
-### 6. Often there is some back and forth between reviewers and maintainers at this step
+*Often there is some back and forth between reviewers and maintainers at this step.*
There is no specified duration for this period. Rather as long as all
parties are responsive within 2 weeks the review shall continue until the author has completed work addressing the reviews.
-### 7. Package acceptance
+### 6. Package acceptance
Once the maintainers have completed updating the package, the assigned editor
will ask the reviewers if they are happy with changes made. At this point the
editor performs one last check on the package. And accepts it if that is appropriate.
- Now, there are a few final cleanup activities including adding a pyOpenSci
- badge to the package README file, and creating a new release on GitHub from the reviewed version. The package is
- now accepted into the pyOpenSci ecosystem!
+Now, there are a few final cleanup activities including:
+
+ * Adding the pyOpenSci peer-reviewed badge to the package README file.
+ * Creating a new release on GitHub from the reviewed version.
+ * Adding package authors and the package to the pyOpenSci website.
+
+The package is now accepted into the pyOpenSci ecosystem!
### JOSS submission
@@ -74,18 +77,61 @@ JOSS refers to the [Journal of Open Source Software](https://joss.theoj.org/). I
be fast tracked through the JOSS review process (another review is not required
for this step).
+## Peer review guides
+
If you want to learn more about each step listed above, we suggest that you read
-through the guides below:
-
-* **[Authors Guide](author-guide):** If you are a **package maintainer** who is
-interested in submitting a package to
-pyOpenSci, check out the authors guide.
-* **[Reviewers guide](reviewer-guide):** If you have volunteered to be a
-**reviewer** for a pyOpenSci package, you will want to carefully read
-through the reviewer guide for guidance on how we run our reviews. Thank you in
-advance for volunteering your time to support open science and open source software!
-* **[Editor guide](editors-guide):** Editors for pyOpenSci have previous
-experience reviewing packages for pyOpenSci. The editor guide
-will walk you through the best practices for the editor role.
-* **[Editor in Chief guide](editor-in-chief-guide):** The Editor in Chief is a rotating position within pyOpenSci held by members of the
-pyOpenSci editorial board.
+through the peer-review guides below that are tailored to each role in the peer review process:
+
+
+::::{grid} 1 1 1 2
+:class-container: text-center
+:gutter: 3
+
+:::{grid-item-card} {octicon}`code-square;1.5em;sd-mr-1` Author / Maintainer Guide
+:link: author-guide
+:link-type: doc
+:class-header: bg-light
+
+Learn everything that you need to know about the peer review for **package maintainers** who submit a package to pyOpenSci for peer review.
++++
+Learn more »
+:::
+
+:::{grid-item-card} {octicon}`pencil;1.5em;sd-mr-1` Editor Guide
+:link: editors-guide
+:link-type: doc
+:class-header: bg-light
++++
+
+Learn about the process that editors follow in the pyOpenSci peer review
+process.
++++
+Learn more »
+:::
+
+:::{grid-item-card} {octicon}`codescan-checkmark;1.5em;sd-mr-1` Reviewer Guide
+:link: software-peer-review-guide/reviewer-guide
+:link-type: doc
+:class-header: bg-light
+
+Click here to read more about the process that reviewers take when reviewing
+a Python package for pyOpenSci.
+
++++
+Learn more »
+:::
+
+:::{grid-item-card} {octicon}`pencil;1.5em;sd-mr-1` ✨ Editor in Chief Guide
+:link: software-peer-review-guide/editors-in-chief-guide
+:link-type: doc
+:class-header: bg-light
+
+The Editor in Chief is a rotating position within pyOpenSci held by members
+of the pyOpenSci editorial board. Learn more about the processes involved with
+being an editor in chief for pyOpenSci.
++++
+Learn more »
+:::
+
+::::
+
diff --git a/software-peer-review-guide/reviewer-guide.md b/software-peer-review-guide/reviewer-guide.md
index 2ba1d029..32274f06 100644
--- a/software-peer-review-guide/reviewer-guide.md
+++ b/software-peer-review-guide/reviewer-guide.md
@@ -11,22 +11,54 @@ Please be respectful and kind to the authors in your reviews. Our
[code of conduct](../about-peer-review/code-of-conduct) is mandatory for everyone involved in our
review process.
-## A Guide for New Reviewers
+## A guide for new reviewers
-% TODO: Add text about the criteria for being a reviewer here ... outline idea below
-% * Familiar with Python
-% * course review for usability and documentation vs technical reviews
-% * we like to have both in each review so it can be split up across multiple reviewers (technical, usability, etc)
+Thank you for considering reviewing for pyOpenSci. We welcome reviewers
+from a diversity of background and with varying levels of Python technical
+expertise.
+
+Some of the basic things that we look for in a review include:
+
+* Familiarity with using the Python programming language.
+* Ability to evaluate a Python package for usability and documentation quality.
+* Ability to provide a technical review of Python package structure and code quality / approach to solving the problems that the package seeks to address.
+
+We like to have a mix of technical and usability focus in our reviews so it's ok if you don't have all of the above skills!
+
+```{note}
+## New to package review? We offer mentorship!
+``
+
+If you are interested in peer review but have never reviewed before,
+we offer a mentorship program where we will pair you up with someone
+who has more experience reviewing code. From this experience you can
+learn more and empower yourself with code review skills. Software review skills
+are generally useful in data science, so they are skills worth investing in!
+```
## Get Started With Your Review
+As a new reviewer you should start by installing the package that you are
+reviewing locally to test it out.
+
+To install the package, you can either:
+
+* fork the package repository and install it in
+development/ editable mode `pip install -e .`
+* or you can install it directly from `pip` or `conda-forge`.
-As a new review you want to start by installing the package that you are
-reviewing locally to test it out. You can either fork and install the package in
-development mode `pip install -e .` or install from `pip` or `conda-forge`. Be sure
-that the version that you are reviewing aligns with the version submitted the
-version submitted by the author. The package version can be found at the top of
-the review issue. In the example below you can see that pyrolite version 0.2.5
-was submitted for review. That is the version that you should install!
+```{important}
+Be sure
+that the version that you are reviewing aligns with the version
+submitted by the author. The package version can
+be found at the top of the review issue. In the example below you can
+see that `pyrolite` version 0.2.5
+was submitted for review.
+
+That is the version that you should install!
+
+```
+
+Example header of a peer review submission issue on GitHub:
```
Submitting Author: Name (@morganjwilliams)
@@ -46,19 +78,40 @@ thread to inform them about possible delays.
```
To review a package, please begin by copying our
-[review template](../appendices/templates#review-template) and using it as a
-high-level checklist. In addition to checking off the minimum criteria specified
-in the review- template, please also provide general comments addressing the following:
-
-- Does the code comply with general principles in the [Mozilla reviewing guide](https://mozillascience.github.io/codeReview/review.html)?
-- Does the package comply with the [pyOpenSci packaging guide](../authoring/overview)?
-- Are there improvements that could be made to the code style?
-- Is there code duplication in the package that should be reduced?
-- Are there user interface improvements that could be made?
-- Are there performance improvements that could be made?
-- Is the documentation (installation instructions/vignettes/examples/demos) clear and sufficient? Does it use the principle of *multiple points of entry* i.e. takes into account the fact that any piece of documentation may be the first encounter the user has with the package and/or the tool/data it wraps?
-- Were functions and arguments named to work together to form a common, logical programming API that is easy to read, and autocomplete?
-- If you have your own relevant data/problem, work through it with the package. You may find rough edges and use-cases the author didn't think about.
+review template (see below) and using it as a
+high-level checklist.
+
+```{include} ../appendices/review-template.md
+```
+
+### Other review guidelines to consider
+
+In addition to checking off the minimum criteria specified
+in the review template, please also provide general comments addressing the following:
+
+- Is the code well written and efficient?
+ * Are there improvements that could be made to the code style?
+ * Is there code duplication in the package that should be reduced?
+ * Are there performance improvements that could be made?
+ * Are functions and arguments named to work together to form a common, logical programming API that is easy to read, and autocomplete?
+
+* Does the package comply with the pyOpenSci packaging guide](https://www.pyopensci.org/python-package-guide)?
+
+* Are there user interface improvements that could be made?
+* Is the documentation (installation instructions/vignettes/examples/demos) clear and sufficient?
+ * Does the documentation use the principle of *multiple points of entry* i.e. takes into account the fact that any piece of documentation may be the first encounter the user has with the package and/or the tool/data it wraps?
+
+If you have your own relevant data/problem that you could use the package to solve, work through it with the package. You may find rough edges and use-cases the author didn't think about.
+
+## Submit your review
+* When your review is complete, paste it as a comment into the package software-review issue.
+* Additional comments are welcome in the same issue. We hope that package reviews will work as an ongoing conversation with the authors as opposed to a single round of reviews typical of academic manuscripts.
+* You may also submit issues or pull requests directly to the package repo if you choose, but if you do, please comment about them and link to them in the software-review repo comment thread so we have a centralized record and text of your review.
+* Please include an estimate of how many hours you spent on your review afterwards.
+
+## Review follow-Up
+Authors should respond within 2 weeks with their changes to the package in response to your review. At this stage, we ask that you respond as to whether the changes sufficiently address any issues raised in your review. We encourage ongoing discussion between package authors and reviewers, and you may ask editors to clarify issues in the review thread as well.
+
### Examples of Past pyOpenSci Reviews
@@ -67,29 +120,24 @@ experiences of other reviewers. In general you can find submission threads of
onboarded packages here. Here are a few chosen examples of reviews (note that
your reviews do not need to be as long as these examples):
-
* `MovingPandas` [review 1](https://github.com/pyOpenSci/software-review/issues/18#issuecomment-579520816) and [review 2](https://github.com/pyOpenSci/software-review/issues/18#issuecomment-581752433)
* `Pandera` [review 1](https://github.com/pyOpenSci/software-review/issues/12#issuecomment-527622205) and [review 2](https://github.com/pyOpenSci/software-review/issues/12#issuecomment-531491008)
-
-
-### Lessons Learned from rOpenSci
-
+## Lessons Learned from rOpenSci
You can read blog posts written by reviewers about their experiences [via this link](https://ropensci.org/tags/reviewer/). In particular, in [this blog post by Mara Averick](https://ropensci.org/blog/2017/08/22/first-package-review/) read about the "naive user" role a reviewer can take to provide useful feedback even without being experts of the package's topic or implementation, by asking themselves _"What did I think this thing would do? Does it do it? What are things that scare me off?"_. In [another blog post](https://ropensci.org/blog/2017/09/08/first-review-experiences/) Verena Haunschmid explains how she alternated between using the package and checking its code.
-As both a former reviewer and package author [Adam Sparks](https://adamhsparks.com) [wrote this](https://twitter.com/adamhsparks/status/898132036451303425) "[write] a good critique of the package structure and best coding practices. If you know how to do something better, tell me. It’s easy to miss documentation opportunities as a developer, as a reviewer, you have a different view. You’re a user that can give feedback. What’s not clear in the package? How can it be made more clear? If you’re using it for the first time, is it easy? Do you know another R package that maybe I should be using? Or is there one I’m using that perhaps I shouldn’t be? If you can contribute to the package, offer."
+As both a former reviewer and package author [Adam Sparks](https://adamhsparks.com) [wrote this](https://twitter.com/adamhsparks/status/898132036451303425) "[write] a good critique of the package structure
+and best coding practices. If you know how to do something better, tell
+me. It’s easy to miss documentation opportunities as a developer, as a
+reviewer, you have a different view. You’re a user that can give
+feedback. What’s not clear in the package? How can it be made more
+clear? If you’re using it for the first time, is it easy? Do you know
+another `R` package that maybe I should be using? Or is there one I’m
+using that perhaps I shouldn’t be? If you can contribute to the package,
+offer."
+### Feedback on the pyOpenSci open peer review process
-### Feedback on the Process
+We welcome any feedback and/or questions about the review process! We encourage you to post any issues, feedback or questions in our [Discourse forum](https://pyopensci.discourse.group/c/review-process/7).
-We welcome any feedback and/or questions about the review process! We encourage you to open an issue on our [governance repository](https://github.com/pyOpenSci/governance/issues/new).
-
-## Submitting the Review
-- When your review is complete, paste it as a comment into the package software-review issue.
-- Additional comments are welcome in the same issue. We hope that package reviews will work as an ongoing conversation with the authors as opposed to a single round of reviews typical of academic manuscripts.
-- You may also submit issues or pull requests directly to the package repo if you choose, but if you do, please comment about them and link to them in the software-review repo comment thread so we have a centralized record and text of your review.
-- Please include an estimate of how many hours you spent on your review afterwards.
-
-## Review Follow-Up
-Authors should respond within 2 weeks with their changes to the package in response to your review. At this stage, we ask that you respond as to whether the changes sufficiently address any issues raised in your review. We encourage ongoing discussion between package authors and reviewers, and you may ask editors to clarify issues in the review thread as well.
diff --git a/software-peer-review-guide/where-to-find-python-package-reviewers.md b/software-peer-review-guide/where-to-find-python-package-reviewers.md
new file mode 100644
index 00000000..1c168c7c
--- /dev/null
+++ b/software-peer-review-guide/where-to-find-python-package-reviewers.md
@@ -0,0 +1,50 @@
+# Reviewer Resources
+
+## Where to Look for Reviewers?
+
+As a (guest) editor, you can find reviewers through:
+* Suggestions made by the submitter(s) (although submitters may have a narrow view of the types of expertise needed. We suggest not using more than one of the suggested reviewers).
+* Authors of existing [pyOpenSci packages](https://www.pyopensci.org/python-packages/).
+* Other [contributors to pyOpenSci](https://www.pyopensci.org/our-community/).
+
+When these sources of information are not enough:
+* Ping other editors for ideas.
+* Look for users of the package or the data source/upstream service the package connects to (via their opening issues in the repository, starring it, citing it in papers, talking about it on Twitter).
+* You can also search for authors/maintainers of related packages on [PyPI](https://pypi.org/search/).
+* Post on Twitter and ensure pyOpenSci retweets your post.
+
+## Criteria for Choosing Reviewers
+
+Here are criteria to keep in mind when choosing a reviewer. You might need to
+piece this information together by searching `PyPI`, `Conda` / `Conda-forge` and
+the potential reviewer’s GitHub page and general online presence (personal
+website, Twitter).
+
+* Has not reviewed a package for us within the last 6 months.
+* Some package development / contribution experience.
+* Some domain experience in the field of the package or data source.
+* No [conflicts of interest](../about-peer-review/policies-and-guidelines.html#conflict-of-interest).
+
+Try to balance your sense of the potential reviewer’s experience against the complexity of the package.
+
+* **Diversity** - if you have two reviewers, both shouldn’t be cis white males.
+* **Openness** - reviewers should also have demonstrated interest in open source or Python community activities, although blind emailing is fine.
+
+Each submission should be reviewed by _two_ package reviewers. Although it is
+fine for one of them to have less package development experience and more domain
+knowledge, the review should not be split into two parts. Both reviewers need to review
+the package comprehensively, from their particular perspectives. In
+general, at least one reviewer should have prior reviewing experience, and of
+course inviting one new reviewer expands our pool of reviewers.
+
+Reviewers should ideally have some subject matter expertise associated
+with the package functionality. It is ok and even welcome if one reviewer has
+more technical expertise and the other focuses on usability and is less technical.
+Read through the Guidelines for Reviewers Section to learn more about finding and
+selecting reviewers.
+
+```{note}
+PyOpenSci has been piloting a new reviewer mentorship program where we pair a
+new reviewer with someone in the community with previous review experience. If
+a new reviewer is interested in this, get in touch with the **editor in chief**.
+```