Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW]: py-pde: A Python package for solving partial differential equations #2158

Closed
whedon opened this issue Mar 10, 2020 · 59 comments
Closed

Comments

@whedon
Copy link
Collaborator

@whedon whedon commented Mar 10, 2020

Submitting author: @david-zwicker (David Zwicker)
Repository: https://github.com/zwicker-group/py-pde
Version: v0.5
Editor: @xuanxu
Reviewer: @celliern, @mstimberg
Archive: 10.5281/zenodo.3739300

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/198f770408b1a7051b5b6af09db2209b"><img src="https://joss.theoj.org/papers/198f770408b1a7051b5b6af09db2209b/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/198f770408b1a7051b5b6af09db2209b/status.svg)](https://joss.theoj.org/papers/198f770408b1a7051b5b6af09db2209b)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@celliern & @@mstimberg, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @xuanxu know.

Please try and complete your review in the next two weeks

Review checklist for @celliern

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@david-zwicker) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @mstimberg

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@david-zwicker) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Mar 10, 2020

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @celliern, @@mstimberg it looks like you're currently assigned to review this paper 🎉.

⭐️ Important ⭐️

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Mar 10, 2020

Reference check summary:

OK DOIs

- 10.1145/2833157.2833162 is OK
- 10.21105/joss.01356 is OK
- 10.11588/ans.2015.100.20553 is OK
- 10.1109/MCSE.2009.52 is OK
- 10.1007/s10444-019-09666-0 is OK
- 10.5194/gmd-12-1165-2019 is OK
- 10.1109/MCSE.2007.53 is OK
- 10.1109/MCSE.2007.55 is OK
- 10.7717/peerj-cs.103 is OK
- 10.21105/joss.01277 is OK

MISSING DOIs

- https://doi.org/10.1137/110856976 may be missing for title: PyClaw: Accessible, Extensible, Scalable Tools for Wave Propagation Problems

INVALID DOIs

- https://doi.org/10.1038/s41592-019-0686-2 is INVALID because of 'https://doi.org/' prefix
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Mar 10, 2020

PDF failed to compile for issue #2158 with the following error:

pandoc-citeproc: reference mstimberg not found
Error producing PDF.
! TeX capacity exceeded, sorry [input stack size=5000].
\reserved@a ->\def \reserved@a
*{\let @xs@assign @xs@expand@and@detokenize...
l.369 }

Looks like we failed to compile the PDF

@mstimberg

This comment has been minimized.

Copy link

@mstimberg mstimberg commented Mar 10, 2020

I think there was an issue with @@mstimberg instead of @mstimberg being added as a reviewer. Maybe removing and readding will fix it?

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Mar 10, 2020

This looks like a problem with a reference in the .md and .bib files - the .md file seems to be trying to reference something that is not in the bib file (or the bib file has a problem that leads to it not being parsed correctly)

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Mar 10, 2020

👋 @openjournals/dev - can you look at this?

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Mar 10, 2020

@whedon generate pdf

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Mar 10, 2020

@mstimberg

This comment has been minimized.

Copy link

@mstimberg mstimberg commented Mar 11, 2020

@xuanxu I cannot edit the checklist and I did not receive an invite – is this because this is my first review for JOSS and I have to do something else first? Or is it because of the earlier @@mstimberg issue?

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Mar 11, 2020

@mstimberg I guess it's the previous @@ issue, let's try to reinvite you.

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Mar 11, 2020

@whedon re-invite @mstimberg as reviewer

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Mar 11, 2020

OK, the reviewer has been re-invited.

@mstimberg please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

@mstimberg

This comment has been minimized.

Copy link

@mstimberg mstimberg commented Mar 11, 2020

Thanks @xuanxu, it's working now 👍

@celliern

This comment has been minimized.

Copy link

@celliern celliern commented Mar 13, 2020

First, I wanted to say that I'm impressed ! The library is clean and well organized and I will definitively add it in my research toolbox. It also fit well the Python scientific ecosystem, especially compared with my work () that has not be designed to easily write tensor-based PDE equation. This is a welcome approach as this is a very common task among a wide range of scientific topics.

The features offered by py-pde are multiple and the dependencies despite very narrowed mandatory dependencies (all of them being in the standard scientific stack and shipped with distributions like Anaconda). This should guarantee an easy deployment on clusters.

All tests have run with a fresh conda environment with py=3.6 on my computer.

I still have to play a bit with it to ensure the functional claims of the software. During this time, I have some optional (but welcome) improvement that I could suggest :

Packaging improvement

The optional dependencies could be listed in the setup.py (see in the setuptools doc) to make the process more automatic for the user.

Documentation improvement

Some information are available in the paper may be added also in the documentation: the method used for the operator discretization (finite difference), the temporal solvers available for the explicit as well as for the implicit ones. In general, a "advanced user documentation" could highly improve the adoption of the software : more detail on how to implement custom PDE (with or without numba), some info on the internals (discretization, time-steps choice), and if / how the library can be extended to fit more specific needs.

Similarly, there is no in-between the very short overview and trivial example and the full API documentation (which is welcome but hard to decipher).

Some doc-strings lack of details : the allowed boundary conditions are well described in the raised ValueError, but not in the doc-string itself.

There is more examples available on the repository than shown in the documentation : using Sphinx-Gallery, these examples could easily be integrated in the documentation.

The software seems to embed a benchmark tool : it could be nice to have a summary of the performance of implemented models with and without numba and with and without parallelization.

Community guidelines

It could be nice to add a Code of Conduct (see the scipy one for an extended example, and here for a standard one that you can easily adapt for your repository).

@arfon

This comment has been minimized.

Copy link
Member

@arfon arfon commented Mar 14, 2020

Dear authors and reviewers

We wanted to notify you that in light of the current COVID-19 pandemic, JOSS has decided to suspend submission of new manuscripts and to handle existing manuscripts (such as this one) on a "best efforts basis". We understand that you may need to attend to more pressing issues than completing a review or updating a repository in response to a review. If this is the case, a quick note indicating that you need to put a "pause" on your involvement with a review would be appreciated but is not required.

Thanks in advance for your understanding.

Arfon Smith, Editor in Chief, on behalf of the JOSS editorial team.

@david-zwicker

This comment has been minimized.

Copy link

@david-zwicker david-zwicker commented Mar 30, 2020

Dear @celliern, thank you very much for your suggestions! I managed to include most of your points in the revised code. Thank you especially for suggesting the example gallery – I didn't know about this, but I think it helps showcasing the features of the package tremendously.

I'm looking forward to the remaining comments, but I obviously understand if this will now take much longer than usual. Stay safe and healthy!

@celliern

This comment has been minimized.

Copy link

@celliern celliern commented Mar 30, 2020

@david-zwicker Nice your gallery is wonderful !

Two minor points :

  • "Do the authors clearly state what problems the software is designed to solve and who the target audience is?"
    This could be highlighted in the "Getting started" : I guess that such software is designed with researcher and student in mind (more than engineer / industrial target) due to the well known finite difference limitation (especially on complex geometry).

  • The CoC is in the repo, but should be linked in the doc and / or in the repository README.
    I'm looking at the paper today.

As a side suggestion (not blocking for the publication) : you will have a strong limitation on hyperbolic equations. This can be (partially) mitigated by introducing "advection" operator that could either implement really simple Gudunov finite volume scheme or upwind finite difference scheme. This will extend the scope of the software.

Stay safe and healthy as well !

@celliern

This comment has been minimized.

Copy link

@celliern celliern commented Mar 30, 2020

For the paper:

  • there is still some typo : they could be easily spotted by a spell checker.
  • pyclaw is the Clawpack python interface, clawpack is specialized in finite volume (especially Gudunov-like methods), not finite difference.
  • you could cite the Dedalus Project for the pseudo-spectral methods.

Otherwise, the paper look fine. I will recommand the paper / software for publication once these different point are fixed.

You have written a nice scientic tool. Nice work !

@mstimberg

This comment has been minimized.

Copy link

@mstimberg mstimberg commented Mar 31, 2020

I join @celliern in congratulating you to your software, this is really an excellent tool and I have rarely seen that much attention to code quality in scientific software (type annotations, etc.)!

I have reported two minor issues (zwicker-group/py-pde#2, zwicker-group/py-pde#3) and three suggestions (zwicker-group/py-pde#4, zwicker-group/py-pde#5, zwicker-group/py-pde#5) directly in the py-pde repository, but @celliern's thorough review certainly made my job much easier.

Fixing the typos in the article is the only real blocker for me – if I wanted to be really nitpicky, I could also mention that the capitalization in the references list is somewhat inconsistent (e.g. "Python" vs. "python").

Pending these minor issues I can easily recommend this paper for publication. 👍

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Mar 31, 2020

Fixing the typos in the article is the only real blocker for me – if I wanted to be really nitpicky, I could also mention that the capitalization in the references list is somewhat inconsistent (e.g. "Python" vs. "python"

Please do be nitpicky - we need to fix all of these before publication :)

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Apr 3, 2020

OK @david-zwicker, everything looks good, here are the next steps:

  • Please release a new tagged version from the current master so it includes all the changes made during the review process
  • Then archive that latest release in Zenodo
  • Check the Zenodo deposit has the correct metadata: title and author name should match the paper; you may also add your ORCID.

Once you do that please report here the version number and archive DOI

@david-zwicker

This comment has been minimized.

Copy link

@david-zwicker david-zwicker commented Apr 3, 2020

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Apr 3, 2020

@whedon set v0.5 as version

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

OK. v0.5 is the version.

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Apr 3, 2020

@whedon set 10.5281/zenodo.3739300 as archive

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

OK. 10.5281/zenodo.3739300 is the archive.

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Apr 3, 2020

Thanks @mstimberg and @celliern for your reviews!

@xuanxu

This comment has been minimized.

Copy link
Member

@xuanxu xuanxu commented Apr 3, 2020

@whedon accept

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

Attempting dry run of processing paper acceptance...
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

Reference check summary:

OK DOIs

- 10.1145/2833157.2833162 is OK
- 10.21105/joss.01356 is OK
- 10.11588/ans.2015.100.20553 is OK
- 10.1137/110856976 is OK
- 10.1109/MCSE.2009.52 is OK
- 10.1007/s10444-019-09666-0 is OK
- 10.5194/gmd-12-1165-2019 is OK
- 10.1109/MCSE.2007.53 is OK
- 10.1109/MCSE.2007.55 is OK
- 10.7717/peerj-cs.103 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.21105/joss.01277 is OK

MISSING DOIs

- None

INVALID DOIs

- None
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1405

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1405, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true
@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Apr 3, 2020

Thanks @xuanxu - I'll take over

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Apr 3, 2020

👋 @david-zwicker - I've suggested some minor changes in zwicker-group/py-pde#8

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Apr 3, 2020

@whedon accept

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

Attempting dry run of processing paper acceptance...
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

Reference check summary:

OK DOIs

- 10.1145/2833157.2833162 is OK
- 10.21105/joss.01356 is OK
- 10.11588/ans.2015.100.20553 is OK
- 10.1137/110856976 is OK
- 10.1109/MCSE.2009.52 is OK
- 10.1007/s10444-019-09666-0 is OK
- 10.5194/gmd-12-1165-2019 is OK
- 10.1109/MCSE.2007.53 is OK
- 10.1109/MCSE.2007.55 is OK
- 10.7717/peerj-cs.103 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.21105/joss.01277 is OK

MISSING DOIs

- None

INVALID DOIs

- None
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1406

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1406, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true
@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Apr 3, 2020

@whedon accept deposit=true

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

Doing it live! Attempting automated processing of paper acceptance...
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited 👉 openjournals/joss-papers#1407
  2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02158
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Apr 3, 2020

Thanks to @celliern & @mstimberg for reviewing, and @xuanxu for editing!

Congratulations to @david-zwicker!

@danielskatz danielskatz closed this Apr 3, 2020
@whedon

This comment has been minimized.

Copy link
Collaborator Author

@whedon whedon commented Apr 3, 2020

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02158/status.svg)](https://doi.org/10.21105/joss.02158)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02158">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02158/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02158/status.svg
   :target: https://doi.org/10.21105/joss.02158

This is how it will look in your documentation:

DOI

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

@david-zwicker

This comment has been minimized.

Copy link

@david-zwicker david-zwicker commented Apr 3, 2020

Thank you everyone for the very smooth process and for your numerous suggestions that surely improved the quality of the code, the documentation, and the paper.

There is some small issue still: I cannot see the final pdf of the paper and I actually receive a 404 when I want to download it. Is this something that would just take a while until it is finally there?

@danielskatz

This comment has been minimized.

Copy link

@danielskatz danielskatz commented Apr 3, 2020

There's likely a caching issue on your side - it works fine for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.