Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW]: ENZO: An Adaptive Mesh Refinement Code for Astrophysics (Version 2.6) #1636

Open
whedon opened this issue Aug 9, 2019 · 36 comments

Comments

@whedon
Copy link
Collaborator

commented Aug 9, 2019

Submitting author: @bwoshea (Brian O'Shea)
Repository: https://github.com/enzo-project/enzo-dev
Version: v2.6
Editor: @danielskatz
Reviewer: @zingale, @rtfisher
Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295"><img src="https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295/status.svg)](https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@zingale & @rtfisher, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.

Please try and complete your review in the next two weeks

Review checklist for @zingale

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Version: Does the release version given match the GitHub release (v2.6)?
  • Authorship: Has the submitting author (@bwoshea) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Authors: Does the paper.md file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @rtfisher

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Version: Does the release version given match the GitHub release (v2.6)?
  • Authorship: Has the submitting author (@bwoshea) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Authors: Does the paper.md file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2019

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @zingale, @rtfisher it looks like you're currently assigned to review this paper 🎉.

⭐️ Important ⭐️

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2019

Attempting PDF compilation. Reticulating splines etc...
@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2019

@danielskatz

This comment has been minimized.

Copy link

commented Aug 9, 2019

I had to update the repository URL on the top comment of this issue. (note to @arfon - whedon didn't use the URL in the pre-report issue, and this may also lead to a problem when we do the final acceptance.)

@danielskatz

This comment has been minimized.

Copy link

commented Aug 9, 2019

@whedon generate pdf

@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2019

Attempting PDF compilation. Reticulating splines etc...
@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2019

@danielskatz

This comment has been minimized.

Copy link

commented Aug 9, 2019

@bwoshea - merging enzo-project/enzo-dev#93 will fix 2 small bib entries

@danielskatz

This comment has been minimized.

Copy link

commented Aug 9, 2019

👋 @zingale, @rtfisher - we are ready for you to review this now. Please read the first 2 comments in this issue carefully. Your job now is to go through the paper (the latest "check article proof" in this issue) and the repository, and check off items in your checklist until you are done, and if you find problems, either report them here or create an issue in the repo and mention this issue in that one.

If you have any questions or problems, please let me know.

@rtfisher

This comment has been minimized.

Copy link
Collaborator

commented Aug 10, 2019

I am just beginning the review process. I find the documentation link (https://enzo-project.org/docs/2.6/) under https://github.com/enzo-project/enzo-dev returns a 404.

@danielskatz

This comment has been minimized.

Copy link

commented Aug 11, 2019

@bwoshea - merging enzo-project/enzo-dev#93 will fix 2 small bib entries

Now that this has been done, I will recompile - @bwoshea, you can do this too, just the way I am, with an instruction in whedon in a new comment (letting you know in case more changes are needed)

@danielskatz

This comment has been minimized.

Copy link

commented Aug 11, 2019

@whedon generate pdf

@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 11, 2019

Attempting PDF compilation. Reticulating splines etc...
@whedon

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 11, 2019

@zingale

This comment has been minimized.

Copy link
Collaborator

commented Aug 11, 2019

regarding

References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)

There are 2 references to Enzo itself, referring to github. Should these be changed to an archived version of the code with a DOI?

@zingale

This comment has been minimized.

Copy link
Collaborator

commented Aug 14, 2019

I'm done with my pass through the docs and have filed a number of issues, some which I would like to see addressed for the review (marked with [JOSS REVIEW]) and others that are suggestions to take into consideration.

I will try running some standard test problems next.

@danielskatz

This comment has been minimized.

Copy link

commented Aug 15, 2019

@zingale - thanks for all your comments so far. If possible, when you open an issue in https://github.com/enzo-project/enzo-dev, just mention this review thread (#1636) in that issue rather than posted that issue in this thread. This will insert a note here that will show if the issue is open or closed.

@zingale

This comment has been minimized.

Copy link
Collaborator

commented Aug 16, 2019

oh, I see, I did it backwards. I'll fix that.

@danielskatz

This comment has been minimized.

Copy link

commented Aug 26, 2019

👋 @bwoshea - note that we're waiting for you to respond to a bunch of open issues - listed above...

@rtfisher

This comment has been minimized.

Copy link
Collaborator

commented Aug 26, 2019

Just to clarify, I am still in the process of completing my review. @bwoshea indicates that the ENZO team is awaiting this completion prior to starting their responses.

@danielskatz

This comment has been minimized.

Copy link

commented Aug 26, 2019

That's ok, but JOSS reviews typically are more interactive. Maybe with a complex team like this, it makes sense to wait.

@bwoshea

This comment has been minimized.

Copy link

commented Aug 29, 2019

@danielskatz Sorry about that. Given the vigor of @zingale 's review and the distributed nature of the Enzo team, we would prefer to wait until the reviews are completed (or at least the first wave of feedback) until we proceed. If you insist on doing it otherwise I'm sure we can make that happen, though.

@danielskatz

This comment has been minimized.

Copy link

commented Aug 29, 2019

It’s fine to wait, as I said previously

@rtfisher

This comment has been minimized.

Copy link
Collaborator

commented Aug 29, 2019

Ok, I've completed my initial review. It looks like I also identified some of the same issues as @zingale, though our final assessments of the JOSS requirements differ slightly.

@danielskatz

This comment has been minimized.

Copy link

commented Aug 29, 2019

Thanks @rtfisher! Now over to @bwoshea and the Enzo team...

@bwoshea

This comment has been minimized.

Copy link

commented Sep 7, 2019

@danielskatz We're on it! @rtfisher and @zingale , thank you for all of your feedback!

@bwoshea

This comment has been minimized.

Copy link

commented Sep 12, 2019

@danielskatz @rtfisher @zingale The Enzo developers have been working on the issues you raised (at https://github.com/enzo-project/enzo-dev/issues), and we currently have a bunch of open pull requests to address them (at https://github.com/enzo-project/enzo-dev/pulls). Some of the issues have comments on them that directly address reviewer comments without a PR (e.g., enzo-project/enzo-dev#108), but mostly the changes in the PRs themselves should address your comments.

Please let us know how you'd like to proceed at this point!

@danielskatz

This comment has been minimized.

Copy link

commented Sep 12, 2019

@bwoshea - it's up to you, with the goal of getting the reviewers to agree that the submission meets the criteria. You could make all the changes, then ask the reviewers to check them, or ask reviewers for specific opinions on possible changes first.

@zingale

This comment has been minimized.

Copy link
Collaborator

commented Sep 12, 2019

I would suggest that you refer to the PRs in the issues we raised and when we are satisfied, we will close the issues that we opened for the review, and that will be reflected here automatically.

@bwoshea

This comment has been minimized.

Copy link

commented Sep 12, 2019

@danielskatz -- duly noted, thank you! We'll engage the reviewers on the open PRs rather than waiting. That way we'll be able to respond more quickly.

@zingale -- Great suggestion, thanks. I've done that now! We have a couple of outstanding issues that still need developer attention, but the bulk have been addressed.

@bwoshea

This comment has been minimized.

Copy link

commented Sep 13, 2019

@zingale @rtfisher just to give you a heads-up, we've addressed all of the issues you have raised but one, and that should be done in the next day or so. We've left all off the PRs open so that you can see what has been changed, and all PRs and issues are cross-referenced. We're ready for your feedback whenever you are available.

Edit: we now (as of 2019-09-15) have PRs that address all issues raised!

@zingale

This comment has been minimized.

Copy link
Collaborator

commented Sep 16, 2019

It looks like all of my issues have been addressed via PRs. Once those PRs are merged, I am happy with accepting.

@rtfisher

This comment has been minimized.

Copy link
Collaborator

commented Sep 16, 2019

I am fine with accepting after merger as well.

@bwoshea

This comment has been minimized.

Copy link

commented Sep 16, 2019

@zingale @rtfisher thanks!

@rtfisher to clarify, there are two open issues that you raised (enzo-project/enzo-dev#108 and enzo-project/enzo-dev#109 ) - would you mind making a quick note saying you're OK with what we've done for those?

@rtfisher

This comment has been minimized.

Copy link
Collaborator

commented Sep 17, 2019

Thank you, @bwoshea. Sorry for the delayed response -- I was serving on a panel today.

I concur that the issues are now resolved, and have closed both.

@danielskatz

This comment has been minimized.

Copy link

commented Sep 18, 2019

The next steps will be

  1. The authors (@bwoshea) to make any remaining changes (sorry, I can't easily tell if there's more to do or if you've already done this because the PRs didn't reference this review thread) and saying that they are done here.
  2. The reviewers (@zingale and @rtfisher) to confirm that they are satisfied by checking off the remaining items in their review checklists above and by saying so in this thread.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.