New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review #76

Closed
danielskatz opened this Issue Oct 4, 2018 · 7 comments

Comments

Projects
None yet
5 participants
@danielskatz
Collaborator

danielskatz commented Oct 4, 2018

Based on discussion in Section 4.2 and 4.5 of A&P google doc

  • Does code review (section 4.2) fit our scope?
  • Does grant review (section 4.5) fit out scope?
@moranegg

This comment has been minimized.

Contributor

moranegg commented Oct 10, 2018

When considering the peer-review cycle there should be guidance about software citation, checking if all software that needs to be cited and is cited 'correctly' with a minimal set of important/mandatory metadata.
Code review seems like a very big step in the peer-review cycle and is not directly related to software citation.
Grant review seems like a subject on its own to determine how the peer-review cycle is established, as with code review.

Both might affect or be affected by software citation, but in my opinion, is out of scope (#80).

@alee

This comment has been minimized.

Contributor

alee commented Oct 10, 2018

We currently require that a software package go through peer review before granting a DOI to a specific code release. This is a cursory review of the software artifacts accompanying a publication before it becomes citable to evaluate whether it meets some minimal standards: (a) archived in a trusted digital repository (b) can be compiled / executed (c) documented & described at an appropriate level of detail.

This depends in part on how critical the software artifacts are to the publication's findings though, we've gotten some pushback on code archival as a "who cares" type of problem.

@ljhwang

This comment has been minimized.

ljhwang commented Oct 11, 2018

We asked for a citable paper when accepting a code into our repository. This can be any combination of a benchmark paper, a paper on the code, or a research paper. The first indicates some validation or verification of the code has been performed and the latter that the methodology has been peer reviewed.

@lkellogg

This comment has been minimized.

lkellogg commented Nov 12, 2018

A lot of scholarly journals provide their reviewers with a list of questions that include things like "has the literature been adequately cited?" I don't think I've seen any in a discipline journal that ask the reviewer to consider "if software was used in this research, was it adequately cited?" Adding that one question to the review template would provide a prompt for reviewers to think about this without asking them to do a code review (which they may not feel qualified or ready to do.)

@danielskatz

This comment has been minimized.

Collaborator

danielskatz commented Nov 13, 2018

I agree with @lkellogg but don't think this is the goal of this issue - it better fits #77

@danielskatz

This comment has been minimized.

Collaborator

danielskatz commented Nov 20, 2018

the writing group thinks we need to think about citation in the context of review, but that much of review itself (such as code review) is out of scope. @danielskatz will update document

@danielskatz

This comment has been minimized.

Collaborator

danielskatz commented Nov 20, 2018

updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment