Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @sarats, it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐️ Important ⭐️

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

Attempting PDF compilation. Reticulating splines etc...

👉 Check article proof 📄 👈

@sarats @sjvrijn @mmenickelly 👋 Welcome and thanks for agreeing to review! The comments from @whedon above outline the review process, which takes place in this thread (possibly with issues filed in the CEGO repository). I'll be watching this thread if you have any questions.

Hi, first time reviewing for JOSS. To clarify this process: any comments I have that prevent me from checking off something in the checklist should be raised as an issue in the CEGO repository and not here, right?

And if I have multiple comments, is there a preference for separating them into multiple issues, or combining all comments into one large issue?

Small comments, especially anything stylistic or specific to the paper, are fine here. More significant issues, especially those that should have a definite resolution, are better filed as separate issues with the CEGO repository.

A few small comments so far @ianhbell:

• The version in setup.py is still listed as 0.0.0 rather than 1.0.0
• Installation worked fine through both methods, but I initially tried to install the listed dependencies separately before later realizing that they are automatically installed already. Might be worth mentioning in the Readme?
• the doi for the 2006 paper is missing in the .bib file: 10.1145/1143997.1144142

Comments:
I am not sure from the README.md or the software paper what are the full capabilities of this software. There are some great examples, also provided in Jupyter notebooks, of a MINLP and some highly non convex functions of real-valued variables, but I don't have a clear picture of exactly what CEGO's scope does and doesn't contain.

To fix this, I would like to see:

1. Better API documentation. Please tell me if I'm not seeing something, but is there no reference description of the individual functions/classes/return types/arguments?
2. In the software paper, it would help to include a clear description of exactly what types of constraints and what types of objective functions CEGO can handle. Can CEGO exploit derivative information of objectives/constraints when derivatives exist and are available? Most critically, and what I don't know from the examples, can CEGO handle arbitrary analytic or black-box constraints, or can CEGO only handle bound constraints and integer constraints?
3. I'm unsure if what is provided count as "automated tests". Yes, I was available to get them to work, but do they count as automatic? This is more of a technicality than anything else.
4. Continuing on point 3, the output in the paper for the shown example (the HundredDigitPlus example) does not match what is obtained from multiple runs, almost certainly due to randomization. Although many runs find the global minimum, some runs fail to find the global minimum within 1000 evaluations. Something needs to be said in the software paper and possibly the documentation about this (expected and totally reasonable) behavior.

Two smaller comments:

1. Is the amount of Python code shown in the software paper acceptable by JOSS standards?
2. Echoing a previous comment that the DOI is missing for the 2006 ALPS paper.

To everyone, sorry for the delay in dealing with this review...

@sjvrijn I will, as I have done for other projects, bump the revision to 1.0.0 once the review is complete, and at that time I will mint a DOI, and push to PyPI. Thanks for the pointer about the DOI (fixed). Didn't realize that the conference paper had a DOI.

Sorry for the delay everyone...

@sjvrijn Thanks for the pointer about the DOI; I fixed that, didn't know that the conference paper had a DOI. I plan to bump the version to 1.0.0 once the review is complete, mint a DOI for the release, and push to PYPI.

@mmenickelly

1. Good point regarding docs. I have set up to use doxygen to generate docs, and have committed a PDF of the documentation. It's not a big file.
2. It's strictly unconstrained optimization. Or rather, strictly softly constrained optimization with the ability to enforce integer constraints on values. So no derivatives (which are a major obtain to generate in the general case). I'll update the README.
3. You're right. I thought I had set up Travis tests for CEGO, but I guess that was another project. I'll do so and get them to fire on every commit.
4. That's odd. For me, it yielded the global optimum on every run. So I came in and ran it a few times, and indeed, it didn't always get there. So I tweaked the control parameters a bit and added a caveat notice to each notebook along the lines of your comment

A1. I think so, I did something similar for ChebTools. Do you think it is too much or too little?
A2. Fixed, see above

I have set up the Travis tests (https://travis-ci.org/usnistgov/CEGO), and pushed all changes to the repo. Awaiting round number 2!

Is this ready for the reviewers—@sarats, @sjvrijn, @mmenickelly—to take a second look?

@ianhbell Could you add the installing of the Python interface to the Travis setup? It currently does not fail if this does not work for some reason. Also, which Python versions is it intended/tested for?

I have fixed the build issues with binder and now build the Python wrapper in TravisCI as well. All's well at the moment, ready for another look-see.

A few more remarks:

• Could you expand the Community Guidelines a bit more? They do mention contributions/bugs, but regular help with using the package is not addressed

• Notebooks seem to work as intended, but are not consistent in their presentation. I don't mind it that much, but at least explicitly listing the intended global optimum and a reference for the used function would be nice

  • SpringOptimization.ipynb
    • no source for problem is listed
    • it is unclear to me what the four printed values below cell 2 mean
    • the result using pyswarms in cell 4 seems to be better than yours. Is that expected behavior?
  • NonIntegerConstraing.ipynb
    • add a link to the given source
  • Griewank.ipynb
    • list expected optimum to be 0

PS: the use of for i in range(len(x)) in the Griewank definition is not wrong, but more 'pythonic' would be:

    for i, xi in enumerate(x):
        sum1 += pow(xi.as_double(), 2)/4000.0
        prod1 *= cos(xi.as_double()/sqrt(i+1))

🙂

Thanks @sjvrijn -- those were all great recommendations. I have made all them and am ready for another look-see. Some comments:

0. I wasn't entirely clear what you had in mind here, so I added To get started, you should check out the Jupyter notebooks in the notebooks folder; they demonstrate some of the capabilities of CEGO.
1. The call to pyswarms doesn't satisfy the constraints since pyswarms only does continuous optimization
2. Thanks for the pythonic suggestion. It's a pity that they don't do a better job talking about the enumerate function in the docs of Python. It's only much later on I learned about this very useful function. Indexing x twice is not a huge penalty time-wise, but I agree your version is nicer to read and saves two characters per line

Glad you found them useful!

RE: 0., Based on https://joss.readthedocs.io/en/latest/review_criteria.html#community-guidelines, your community guidelines explicitly covers the first two (contributions and reporting issues) but not yet the third: seeking support. A quick line on what they can do in that case would complete it for me. It could e.g. be raising an issue here on Github, sending you an email, joining a chatroom, etc

Other than that, I have no further comments :)

Great, I added

If you want to discuss or request assistance, please open an issue.

@sarats and @mmenickelly, you still have a few unchecked boxes in your reviews. Can you update us on their status and whether there are remaining issues to resolve?

@whedon generate pdf

Attempting PDF compilation. Reticulating splines etc...

@whedon check references

Attempting to check references...

OK DOIs

- 10.1145/1143997.1144142 is OK
- 10.1145/1569901.1570011 is OK
- 10.1109/MCSE.2007.53 is OK
- 10.1115/1.2912596 is OK
- 10.1023/A:1008202821328 is OK

MISSING DOIs

- None

INVALID DOIs

- None

👉 Check article proof 📄 👈

Looks good to me!

Thanks. Please update the author info to remove my name.

@whedon set 10.5281/zenodo.2649254 as archive

OK. 10.5281/zenodo.2649254 is the archive.

Hmm - where is your name listed as author?

In the Zenodo archive; follow DOI link above or direct here: https://zenodo.org/record/2649254

@whedon accept

Attempting dry run of processing paper acceptance...

Check final proof 👉 openjournals/joss-papers#640

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#640, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

OK DOIs

- 10.1145/1143997.1144142 is OK
- 10.1145/1569901.1570011 is OK
- 10.1109/MCSE.2007.53 is OK
- 10.3233/978-1-61499-649-1-87 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1115/1.2912596 is OK
- 10.1023/A:1008202821328 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@openjournals/joss-eics We're ready for you.

Thanks for the reviews, @sarats, @sjvrijn, @mmenickelly, and for the editing, @jedbrown

@whedon accept deposit=true

Doing it live! Attempting automated processing of paper acceptance...

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 openjournals/joss-papers#642
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01147
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

👋 @arfon - can you check on this?

There seems to be a problem with the paper, specifically https://www.theoj.org/joss-papers/joss.01147/10.21105.joss.01147.pdf isn't there.

really? When I try it on a wifi network and on my phone, I don't see it on either

The DOI resolves correctly, and the landing page for the paper is fine, but the PDF of the paper is 404 for me

Link works for me!

I can see the pdf from that link

weird...

@arfon, any thoughts on this before I close the issue? I'll also try again in 2 hours from another location and see if it works there

Now it’s working for me

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.01147/status.svg)](https://doi.org/10.21105/joss.01147)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01147">
  <img src="http://joss.theoj.org/papers/10.21105/joss.01147/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01147/status.svg
   :target: https://doi.org/10.21105/joss.01147

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-joss

Now it’s working for me

The PDFs are cached in Cloudflare which means that if you click the link to the PDF before GitHub pages has had a chance to build (and deploy) then you can get a 404. Your browser (and Cloudflare) can then sometimes cache the 404 response (I think).