• search google for related work, and write to the authors of that related work

Maybe also add something about finding the right reviewers is the job of the editor, and while the submitter can help by providing suggestions, the editor still needs to verify that they are appropriate, and can feel free to ignore reviewer suggestions if the editor has better reviewers in mind

• check the work being referenced in the submission:
  • authors of software that is being built on might be interested in reviewing this submission
  • users of the the software that is being submission be interested in reviewing this submission

Authors of previous JOSS papers that are connected to the current one.

I have also tested to look for reviewers by searching github users with relevant keywords. This didn't work well though.

Few additional thoughts that have not already been said:

• Sometimes I can't find a reviewer that can cover all aspects of a submission (e.g., knows the language really well and knows the scientific discipline well). In these circumstances I try and make sure that between the set of reviewers, everything is covered.
• I point to the JOSS COI policy as part of my review request and ask that interested reviewers verify that they meet the requirements.

I have a feeling that I was the catalyzer for this, apologies and thank you for letting me learn and stress test the documentation for editors.

Something that I think would be helpful to add is time frames and number of review requests to send:

• For example, is it recommended to send 3-4 review invites, give them a week to respond, ping them again, and if they haven't responded by "x" day reach out to a second batch of reviewers?
  (I don't have a handle on what the average acceptance rate and time to respond are yet, having some guidelines would help me calibrate)
• If a paper already has a reviewer but getting the second one is taking much longer, is it okay to start the review and add the second reviewer later on?

Perhaps this PR should add another section focused on procedure that covers the most common logistics for how many/how often, starting a review without all the reviewers found. I should be able to make a start of this later today if no one else has gotten to it.

Another thought that I didn't mention above. We should codify how reviewers and the COI policy relate here. Based on the email discussions, I'm pretty sure the policy is that it is OK for one of two reviewers to have a disclosed COI... but that should be stated somewhere.

The COI policy is in https://joss.theoj.org/about#ethics

JOSS reviewers should ideally have no conflict of interest. In practice, this is not always possible. If a reviewer has a conflict of interest, it must be declared and recorded, and the editors may choose to waive it if this is in the best interest of the review process.

and https://joss.readthedocs.io/en/latest/reviewer_guidelines.html#joss-conflict-of-interest-policy

If you have a conflict of interest with a submission, you should disclose the specific reason to the submissions’ editor. This may lead to you not being able to review the submission, but some conflicts may be recorded and then waived, and if you think you are able to make an impartial assessment of the work, you should request that the conflict be waived.

starting a review without all the reviewers found.

While I think it's ok for us to document how this can be done, we should also say that it should be uncommon to need to do this.

One more question that I ran in today and we could potentially add as a clarification: before creating the actual review issue, is it a requirement to have all (or most of) the DOIs covered? Asking because when reviewing it might be important and helpful to be able to find them to get context and more information - but also waiting for them to be added slows the process and blocks the reviewers...

Not sure about this one, but would appreciate some opinions on the matter.

One more question that I ran in today and we could potentially add as a clarification: before creating the actual review issue, is it a requirement to have all (or most of) the DOIs covered? Asking because when reviewing it might be important and helpful to be able to find them to get context and more information - but also waiting for them to be added slows the process and blocks the reviewers...

I think it's reasonable to ask the author to do this

OK, this is already very useful as-is. I'm going to merge at this point and we can make further changes as required. Thanks all!

One more question that I ran in today and we could potentially add as a clarification: before creating the actual review issue, is it a requirement to have all (or most of) the DOIs covered? Asking because when reviewing it might be important and helpful to be able to find them to get context and more information - but also waiting for them to be added slows the process and blocks the reviewers...

I don't think of missing DOIs as something that would prevent a paper moving from pre-review to review. Though it would be something that would prevent topical editor recommending acceptance of a paper for which the reviewers recommend acceptance.