New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW]: OApackage: A Python package for generation and analysis of orthogonal arrays and conference designs #1097

Open
whedon opened this Issue Nov 26, 2018 · 39 comments

Comments

Projects
None yet
7 participants
@whedon
Copy link
Collaborator

whedon commented Nov 26, 2018

Submitting author: @eendebakpt (Pieter Eendebak)
Repository: https://github.com/eendebakpt/oapackage
Version: 2.5.1
Editor: @danielskatz
Reviewer: @djmitche, @gsagnol
Archive: Pending

Status

status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/452707998bc50b787c0ba41cf544b1c3"><img src="http://joss.theoj.org/papers/452707998bc50b787c0ba41cf544b1c3/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/452707998bc50b787c0ba41cf544b1c3/status.svg)](http://joss.theoj.org/papers/452707998bc50b787c0ba41cf544b1c3)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@djmitche & @ gsagnol, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @danielskatz know.

Please try and complete your review in the next two weeks

Review checklist for @djmitche

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Version: Does the release version given match the GitHub release (2.5.1)?
  • Authorship: Has the submitting author (@eendebakpt) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Authors: Does the paper.md file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @gsagnol

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Version: Does the release version given match the GitHub release (2.5.1)?
  • Authorship: Has the submitting author (@eendebakpt) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Authors: Does the paper.md file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Nov 26, 2018

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @djmitche, it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐️ Important ⭐️

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands
@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Nov 26, 2018

Attempting PDF compilation. Reticulating splines etc...
@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Nov 26, 2018

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Nov 26, 2018

👋 @djmitche, @tanaken-basis - thanks for agreeing to review this submission. Please see the instructions above and feel free to ask me if you have any questions. Otherwise, I will assume that you will work through the checklist items above, and will bring up any issues with the submission with the author (via very short discussion here or via new issues in the source repo), who will resolve them.

@djmitche

This comment has been minimized.

Copy link
Collaborator

djmitche commented Nov 26, 2018

It's worth noting that the most recent version is 2.5.2 now. That is the same in the repo and the package repository.

@peendebak

This comment has been minimized.

Copy link

peendebak commented Nov 27, 2018

Any updates to the package to improve based on review comments will go to the dev branch. On a regular basis I will merge dev to master and update the PyPi packages.

@eendebakpt

This comment has been minimized.

Copy link

eendebakpt commented Dec 4, 2018

@whedon generate pdf from branch feat/updates_paper_alan

@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Dec 4, 2018

Attempting PDF compilation from custom branch feat/updates_paper_alan. Reticulating splines etc...
@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Dec 4, 2018

@arfon

This comment has been minimized.

Copy link
Member

arfon commented Dec 4, 2018

@whedon generate pdf from branch feat/updates_paper_alan

Just a heads up @eendebakpt - compiling from a branch doesn't seem to work very well. You might have to merge to master to see new changes with @whedon.

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 6, 2018

All - @tanaken-basis has informed me by email that he needs to be taken off this review because of a personal matter. So I need to find another reviewer - @djmitche, do you have any suggestion for someone else?

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 6, 2018

@gsagnol has been added as the second reviewer - thanks!

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 6, 2018

👋 @gsagnol - thanks for agreeing to review this submission. Please see the instructions above and feel free to ask me if you have any questions. Otherwise, I will assume that you will work through the checklist items above, and will bring up any issues with the submission with the author (via very short discussion here or via new issues in the source repo), who will resolve them.

@djmitche

This comment has been minimized.

Copy link
Collaborator

djmitche commented Dec 6, 2018

This is my first JOSS review so no, I don't have ideas :/

@peendebak

This comment has been minimized.

Copy link

peendebak commented Dec 10, 2018

@whedon list reviewers

@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Dec 10, 2018

Here's the current list of reviewers: https://bit.ly/joss-reviewers

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 16, 2018

👋 @gsagnol - how are things going? When will you be able to start on the review (at least the first 2 checkboxes regarding COIs and the JOSS CoC?)

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 17, 2018

Hi @danielskatz , I wanted to start with the review this week.

I tried to check some of the boxes, but this will not work, although I'm logged in to github. Is google-chrome under linux not supported ? I tried under firefox, but this did not work neither.

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 17, 2018

As in the first comment in this thread, can you:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations
@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 17, 2018

That's it... I probably did not accept the invitation, but I guess the invitation link expired:
"Sorry, we couldn't find that repository invitation. It is possible that the invitation was revoked or that you are not logged into the invited account."

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 17, 2018

👋 @arfon - can you help with this? Or tell me what I can do?

@arfon

This comment has been minimized.

Copy link
Member

arfon commented Dec 17, 2018

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 18, 2018

I installed the latest release from github, and oapackage.__version__ returns 2.5.2, while the check list above asks for 2.5.1. Should the checklist be updated, or should I review a former version ?

@peendebak

This comment has been minimized.

Copy link

peendebak commented Dec 18, 2018

@gsagnol I don't know about JOSS policy on this matter. I submitted version 2.5.1 to JOSS, but since that moment (also based on reviewers comments) I updated the code and the paper and made a new release. I will continue to push updates based on the review to dev and once in a while merge dev to master (with an increase in version number)

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 18, 2018

@peendebak : I would like to have a look at your unit tests. I tried to run
coverage run --source='./oapackage' -m pytest,
but numpy raises some error:
ImportError: /homes/combi/sagnol/.local/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct

Do I need a particular version of numpy to execute the testing suite ? Or is there another way to run the unit tests ?

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 18, 2018

@peendebak : I posted an issue on the OApackage repository about confusing locations for the documentation (readthedocs.io vs. http://pietereendebak.nl)

@peendebak

This comment has been minimized.

Copy link

peendebak commented Dec 18, 2018

@peendebak : I would like to have a look at your unit tests. I tried to run
coverage run --source='./oapackage' -m pytest,
but numpy raises some error:
ImportError: /homes/combi/sagnol/.local/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: _Py_ZeroStruct

Do I need a particular version of numpy to execute the testing suite ? Or is there another way to run the unit tests ?

The minimal version of numpy is specified in the setup.py (1.13, although perhaps earlier versions of numpy work as well). Can you provide me with some more details of your system (e.g. platform, version of numpy installed, how did you install (pip or compile from source), send output of installation process, version of swig installed)

Can you also send me the output of the following commands?

python -c "import numpy; print(numpy); print(numpy.__version__')
python -c "import oapackage; print(oapackage)"
pytest - v

@gsagnol

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 18, 2018

@peendebak and @gsagnol - please review the "latest" and be sure that all the docs are consistent, then we will accept that version into the repository and the journal once everything is approved.

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 19, 2018

@peendebak I created an issue on the oapackage repo for the tests.

  • On the PC of my institute, I am not able to run the tests. Here, I used a local information (pip install --user)
  • On my laptop, the testing suite seems to work, but some tests fail. On my laptop I used a standard global installation without the --user flag.
@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 20, 2018

@whedon generate pdf

@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Dec 20, 2018

Attempting PDF compilation. Reticulating splines etc...
@whedon

This comment has been minimized.

Copy link
Collaborator

whedon commented Dec 20, 2018

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 20, 2018

@peendebak : Concerning the last "check-box", it seems that some doi's are missing in the software paper. For example,
The book "Orthogonal Arrays: Theory and Applications" of Hedayat et al. should have doi https://doi.org/10.1007/978-1-4612-1478-6

Moreover, this reference appears in two separate bib entries (once as a book, once as a website).

@gsagnol

This comment has been minimized.

Copy link
Collaborator

gsagnol commented Dec 20, 2018

@peendebak : I created a number of issues in the source repository. This can be considered as my list of requests for a minor revision.

@eendebakpt

This comment has been minimized.

Copy link

eendebakpt commented Dec 22, 2018

@gsagnol Thanks for the review! A couple of issues we already addressed (in particular the tests that failed) and we made a new release to pypi based on that. The other issues we will probably address after new year.

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Dec 23, 2018

@eendebakpt - please let us know when these are done

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Jan 7, 2019

Hi @eendebakpt - just checking on the post-holiday status of this...

@eendebakpt

This comment has been minimized.

Copy link

eendebakpt commented Jan 12, 2019

@danielskatz We started addressing the remaining comments.

@danielskatz

This comment has been minimized.

Copy link
Collaborator

danielskatz commented Feb 4, 2019

@eendebakpt - Is there any further update at this point?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment