Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposed changes to checklist #542

Open
wants to merge 5 commits into
base: master
from

Conversation

@Kevin-Mattheus-Moerman
Copy link
Member

commented Jun 11, 2019

I did the following:

  1. Added additional requirements for paper.
  2. Removed version tick box.
  3. Updated the text about authorship and contributions. I attempted to use the following to link to the project contributors:
<%= link_to “contributors”, paper.repository_url +/graphs/contributors”, :target => “_blank%>

I also merged the authorship point from the paper section with this point.
4. I removed the question about authors/affiliations. I propose we let Whedon check for affiliations instead, perhaps based on ORCID, and to list missing affiliations if any. It feels odd to ask the reviewers to check the affiliations, i.e. they'd need to google the authors and check where they work. Not sure if that is their job.

I feel perhaps the References point (asking about DOI's) can be removed too since Whedon checks that at the end. But perhaps we want to start the DOI discussion earlier on in the review process so I left it for now.

Proposed changes to checklist
Added additional requirements for paper. Removed version tick box. Updated general checks in relation to authorship and contributions.
@arfon
Copy link
Member

left a comment

Sorry for the delay, left some comments that I think need addressing before a merge.

@@ -10,8 +10,7 @@

- [ ] **Repository:** Is the source code for this software available at the <%= link_to "repository url", paper.repository_url, :target => "_blank" %>?
- [ ] **License:** Does the repository contain a plain-text LICENSE file with the contents of an [OSI approved](https://opensource.org/licenses/alphabetical) software license?
- [ ] **Version:** Does the release version given match the GitHub release (<%= paper.software_version %>)?
- [ ] **Authorship:** Has the submitting author (<%= paper.submitting_author.github_username %>) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
- [ ] **Contribution and authorship:** Based on the list of <%= link_to "contributors", paper.repository_url + "/graphs/contributors", :target => "_blank" %> to this project, can you confirm: 1) that the submitting author (<%= paper.submitting_author.github_username %>) made major contributions to the software?, and 2) That the full list of paper authors seems appropriate and complete?

This comment has been minimized.

Copy link
@arfon

arfon Jun 26, 2019

Member

So my only problem with this is that the link will only work if the source code is on GitHub. We prefer submissions that are on GitHub and stored in git but adding URLs like this makes this even more locked in. Thoughts?

This comment has been minimized.

Copy link
@Kevin-Mattheus-Moerman

Kevin-Mattheus-Moerman Jun 26, 2019

Author Member

I see the problem. Did not think about that. We can remove the URL so.

Show resolved Hide resolved app/views/content/github/_review_checklist.erb
Show resolved Hide resolved app/views/content/github/_review_checklist.erb Outdated
@arfon

arfon approved these changes Aug 10, 2019

@arfon

This comment has been minimized.

Copy link
Member

commented Aug 10, 2019

@danielskatz @kyleniemeyer @labarba - could you take a look at these changes and see what you think?

@kyleniemeyer

This comment has been minimized.

Copy link
Collaborator

commented Aug 10, 2019

While I'm ok removing the paper authors question, I'm hesitant to rely too much on automation, at least based on ORCID—the vast majority of ORCID profiles do not have affiliation listed (anecdotally, based on profiles I've visited), and if there is one it may not exactly match what's in the paper.

Checking authors & affiliations should probably be an editorial job; I usually ensure that they are complete (i.e., with full details) when I'm doing a final check as AEiC.

I'm also comfortable with modifying the specific question about DOIs, with these caveats:

  • whedon's DOI checker is not foolproof, and should only be used as a guide. I don't want authors, reviewers, or editors to assume that they don't have to look at references themselves. As is, I find some people over-relying on this output, for example when it misidentifies something as needing a DOI.
  • So, with that in mind, perhaps the question could be something like "Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)?"
@arfon

This comment has been minimized.

Copy link
Member

commented Aug 10, 2019

The rationale for removing the author question is that the paper simply doesn't compile without the authors and affiliations in the paper.md so it seems weird to check something like this.

That said, I agree, these do need at least eyeballing but I figure this is part of the editor's (and reviewer's) job when reading the paper.

I'm also comfortable with modifying the specific question about DOIs, with these caveats:

If you look at the changes proposed in this PR, I don't actually think there are any changes to this now.

Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)?

I actually think this phrasing is better than what we have now and would be happy to add it.

@jedbrown

This comment has been minimized.

Copy link
Member

commented Aug 10, 2019

There are reasons why affiliations on a paper would not match even a well-maintained ORCID, such as when the work was done at a prior institution or as part of a visiting appointment. This is especially prevalent for software in which key authors may have founded the project several years ago. We might want to document this publishing convention because I bet lots of submissions just list the current affiliation.

@labarba

This comment has been minimized.

Copy link
Member

commented Aug 11, 2019

Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)?

I would add here: "and do references in the text use the proper citation syntax"

... since bad citation syntax is such a common problem!

@danielskatz

This comment has been minimized.

Copy link
Collaborator

commented Aug 11, 2019

Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)?

I would add here: "and do references in the text use the proper citation syntax"

... since bad citation syntax is such a common problem!

I agree with this, and also think we might want to adjust our sample paper to use some of the citation syntax other than [@abc]

- [ ] **A statement of need:** Do the authors clearly state what problems the software is designed to solve and who the target audience is?
- [ ] **References:** Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
- [ ] **State of the field:** Do the authors describe how this software compares to other available packages? Are key relevant or alternative approaches cited and briefly discussed?

This comment has been minimized.

Copy link
@danielskatz

danielskatz Aug 11, 2019

Collaborator

I'm a bit dissatisfied with this.

For software that was new, this might be a useful point in understanding if people would cite it, but for software that is old and well-used but is not really distinct from other software, what would the authors say?

And I fear this could lead to the author saying something, the reviewer saying "you also should mention X", but still not being complete.

I wonder if we could change it to more simply:

State of the field: Do the authors describe how this software compares to other commonly-used packages?

@kyleniemeyer

This comment has been minimized.

Copy link
Collaborator

commented Aug 11, 2019

@danielskatz regarding citation syntax, in the # Citations section we do now include the different formats authors can use (@author:2001, [@author:2001], [@author:2001; @author2:2010]).

@danielskatz

This comment has been minimized.

Copy link
Collaborator

commented Aug 12, 2019

Thanks - you are correct - I hadn't read this lately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.