Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW]: HPX - The C++ Standard Library for Parallelism and Concurrency #2352

Open
whedon opened this issue Jun 17, 2020 · 44 comments
Open

[REVIEW]: HPX - The C++ Standard Library for Parallelism and Concurrency #2352

whedon opened this issue Jun 17, 2020 · 44 comments
Assignees
Labels

Comments

@whedon
Copy link
Collaborator

@whedon whedon commented Jun 17, 2020

Submitting author: @diehlpk (Patrick Diehl)
Repository: https://github.com/STEllAR-GROUP/hpx
Version: 1.4.1
Editor: @danielskatz
Reviewer: @bhatele, @davidbeckingsale
Archive: Pending

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca"><img src="https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca/status.svg)](https://joss.theoj.org/papers/022e5917b95517dff20cd3742ab95eca)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@bhatele & @davidbeckingsale, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.

Please try and complete your review in the next six weeks

Review checklist for @bhatele

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@diehlpk) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @davidbeckingsale

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@diehlpk) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @bhatele, @davidbeckingsale it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Important

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

PDF failed to compile for issue #2352 with the following error:

Can't find any papers to compile :-(

@danielskatz
Copy link

@danielskatz danielskatz commented Jun 17, 2020

@whedon generate pdf from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

@danielskatz
Copy link

@danielskatz danielskatz commented Jun 17, 2020

@whedon check references from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Attempting to check references... from custom branch JOSS
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Reference check summary:

OK DOIs

- 10.5281/zenodo.598202 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/3152041.3152084 is OK

MISSING DOIs

- https://doi.org/10.1145/3337821.3337915 may be missing for title: Runtime Adaptive Task Inlining on Asynchronous Multitasking Runtime Systems
- https://doi.org/10.1109/espm2.2018.00009 may be missing for title: Asynchronous Execution of Python Code on Task-Based Runtime Systems
- https://doi.org/10.1109/mlhpc49564.2019.00009 may be missing for title: Scheduling Optimization of Parallel Linear Algebra Algorithms using Supervised Learning
- https://doi.org/10.1145/3078155.3078187 may be missing for title: Using SYCL as an implementation framework for hpx. compute
- https://doi.org/10.1109/espm2.2018.00006 may be missing for title: Integration of CUDA Processing within the C++ Library for Parallelism and Concurrency (HPX)
- https://doi.org/10.14529/jsfi150305 may be missing for title: An autonomic performance environment for exascale
- https://doi.org/10.1109/ipdrm49579.2019.00008 may be missing for title: Assessing the Performance Impact of using an Active Global Address Space in HPX: A Case for AGAS
- https://doi.org/10.1109/icppw.2009.14 may be missing for title: ParalleX: An Advanced Parallel Execution Model for Scaling-impaired Applications
- https://doi.org/10.1145/1964218.1964232 may be missing for title: Preliminary design examination of the ParalleX system from a Software and Hardware Perspective
- https://doi.org/10.1177/1094342018819744 may be missing for title: Harnessing Billions of Tasks for a Scalable Portable Hydrodynamic Simulation of the Merger of two Stars
- https://doi.org/10.1109/ipdpsw.2018.00173 may be missing for title: Methodology for Adaptive Active Message Coalescing in Task Based Runtime Systems
- https://doi.org/10.2172/1614897 may be missing for title: Implementing Software Resiliency in HPX for Extreme Scale Computing

INVALID DOIs

- http://dx.doi.org/10.1007/978-3-540-87475-1_39 is INVALID because of 'https://doi.org/' prefix
@danielskatz
Copy link

@danielskatz danielskatz commented Jun 17, 2020

👋 @bhatele & @davidbeckingsale - thanks for agreeing to review this submission. Please be sure to read the comments above, and let me know if you have any questions. Basically, your job is to check the article proof and repository and check items off your checklist above. (And note that you will have to accept the invitation before you will be able to do so.)

If you see small problems that need to be discussed, feel free to discuss them here. But if you can, create a new issue in the [target repository]](https://github.com/STEllAR-GROUP/hpx) and link to this review thread in that issue to create a corresponding breadcrumb trail here.

I look forward to seeing how this review goes

@danielskatz
Copy link

@danielskatz danielskatz commented Jun 17, 2020

👋 @diehlpk - note that whedon suggests changes in your references too, though whedon is not always correct, so please check and make the appropriate changes.

You can rerun the commands above to ask whedon to regenerate the pdf and check the references again as needed.

@davidbeckingsale
Copy link
Collaborator

@davidbeckingsale davidbeckingsale commented Jun 17, 2020

@diehlpk - can you offer a quick bit of rationale behind author selection? A couple of contributors are not on the author list, but have more contributions (in terms of commits/loc) than some of the other authors.

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 17, 2020

@davidbeckingsale We sent some an email and one reminder to our mailing lists and invited people to contribute to the paper before the deadline. The current list of authors followed the invitation and contributed to the paper. However, we could not contact over 100 contributors in person.

@davidbeckingsale
Copy link
Collaborator

@davidbeckingsale davidbeckingsale commented Jun 17, 2020

@davidbeckingsale We sent some an email and one reminder to our mailing lists and invited people to contribute to the paper before the deadline. The current list of authors followed the invitation and contributed to the paper. However, we could not contact over 100 contributors in person.

Perfect. Sorry for being pedantic, I am just trying to make sure that the rationale for each of the checklist points is document somewhere 😄

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 17, 2020

@whedon check references from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Attempting to check references... from custom branch JOSS
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Reference check summary:

OK DOIs

- 10.1145/3337821.3337915 is OK
- 10.1109/espm2.2018.00009 is OK
- 10.1109/mlhpc49564.2019.00009 is OK
- 10.5281/zenodo.598202 is OK
- 10.1145/3078155.3078187 is OK
- 10.1109/espm2.2018.00006 is OK
- 10.14529/jsfi150305 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdrm49579.2019.00008 is OK
- 10.1109/icppw.2009.14 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/1964218.1964232 is OK
- 10.1007/978-3-540-87475-1_39 is OK
- 10.1109/ipdpsw.2018.00173 is OK
- 10.1145/3152041.3152084 is OK
- 10.2172/1614897 is OK

MISSING DOIs

- https://doi.org/10.1177/1094342018819744 may be missing for title: Harnessing Billions of Tasks for a Scalable Portable Hydrodynamic Simulation of the Merger of two Stars

INVALID DOIs

- None
@diehlpk
Copy link

@diehlpk diehlpk commented Jun 17, 2020

@whedon generate pdf from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...
@diehlpk
Copy link

@diehlpk diehlpk commented Jun 17, 2020

Attempting to check references... from custom branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 17, 2020

@whedon generate pdf from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 17, 2020

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 18, 2020

@whedon check references from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 18, 2020

Attempting to check references... from custom branch JOSS
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 18, 2020

Reference check summary:

OK DOIs

- 10.1145/3337821.3337915 is OK
- 10.1109/espm2.2018.00009 is OK
- 10.1109/mlhpc49564.2019.00009 is OK
- 10.5281/zenodo.598202 is OK
- 10.1145/3078155.3078187 is OK
- 10.1109/espm2.2018.00006 is OK
- 10.14529/jsfi150305 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdrm49579.2019.00008 is OK
- 10.1109/icppw.2009.14 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/1964218.1964232 is OK
- 10.1007/978-3-540-87475-1_39 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdpsw.2018.00173 is OK
- 10.1145/3152041.3152084 is OK
- 10.2172/1614897 is OK

MISSING DOIs

- None

INVALID DOIs

- None
@diehlpk
Copy link

@diehlpk diehlpk commented Jun 19, 2020

@danielskatz We fixed all issues with the DOI reported by whedon.

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 19, 2020

@whedon generate pdf from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 19, 2020

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 19, 2020

@davidbeckingsale
Copy link
Collaborator

@davidbeckingsale davidbeckingsale commented Jun 23, 2020

@diehlpk - I don't feel that the paper addresses the following review point: "State of the field: Do the authors describe how this software compares to other commonly-used packages?"

Could you please add a brief overview of related work and compare hpx to these other packages?

@danielskatz
Copy link

@danielskatz danielskatz commented Jun 26, 2020

👋 @diehlpk - This is just to make sure you've seen the comments and are working on responding to them...

@danielskatz
Copy link

@danielskatz danielskatz commented Jun 26, 2020

👋 @bhatele - have you had a chance to start your review? Can you let us know your status?

@hkaiser
Copy link

@hkaiser hkaiser commented Jun 26, 2020

👋 @diehlpk - This is just to make sure you've seen the comments and are working on responding to them...

We're working on them. Thanks.

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 26, 2020

@whedon check references from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 26, 2020

Attempting to check references... from custom branch JOSS
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 26, 2020

Reference check summary:

OK DOIs

- 10.1145/3337821.3337915 is OK
- 10.1109/espm2.2018.00009 is OK
- 10.1109/mlhpc49564.2019.00009 is OK
- 10.5281/zenodo.598202 is OK
- 10.1145/3078155.3078187 is OK
- 10.1109/espm2.2018.00006 is OK
- 10.14529/jsfi150305 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdrm49579.2019.00008 is OK
- 10.1109/icppw.2009.14 is OK
- 10.5281/zenodo.1409043 is OK
- 10.1145/1964218.1964232 is OK
- 10.1007/978-3-540-87475-1_39 is OK
- 10.1177/1094342018819744 is OK
- 10.1109/ipdpsw.2018.00173 is OK
- 10.1145/3152041.3152084 is OK
- 10.2172/1614897 is OK
- 10.1109/HPDC.2000.868632 is OK
- 10.1177/1094342007078442 is OK
- 10.1016/j.jpdc.2014.07.003 is OK
- 10.1109/SC.2012.71 is OK
- 10.1109/MCSE.2013.98 is OK

MISSING DOIs

- None

INVALID DOIs

- None
@diehlpk
Copy link

@diehlpk diehlpk commented Jun 26, 2020

@whedon generate pdf from branch JOSS

@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 26, 2020

Attempting PDF compilation from custom branch JOSS. Reticulating splines etc...
@whedon
Copy link
Collaborator Author

@whedon whedon commented Jun 26, 2020

@diehlpk
Copy link

@diehlpk diehlpk commented Jun 26, 2020

@davidbeckingsale

We highlighted the review paper

Thoman, P., Dichev, K., Heller, T. et al. A taxonomy of task-based parallel programming technologies for high-performance > computing. J Supercomput 74, 1422–1434 (2018). https://doi.org/10.1007/s11227-018-2238-4

which is a very nice overview of AMTS. In addition, we added a paragraph to the paper and highlighted that the main features of HPX is the distributed computing capability with some other notable AMTS. Among this, the future-proof C++ standard conforming API is the major difference.

I am sorry for the delay, but I had two important deadlines this week.

@danielskatz
Copy link

@danielskatz danielskatz commented Jul 5, 2020

👋 @bhatele & @davidbeckingsale - How are your reviews coming? Can you let us know your status?

@davidbeckingsale
Copy link
Collaborator

@davidbeckingsale davidbeckingsale commented Jul 6, 2020

@danielskatz - my checklist shows the tasks I have completed. I had some other priorities last week but will make more review progress this week.

@danielskatz
Copy link

@danielskatz danielskatz commented Jul 12, 2020

👋 @bhatele - When will you be able to start this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.