Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @FelixHenninger, @JonathanReardon it looks like you're currently assigned to review this paper 🎉.

⭐️ Important ⭐️

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

Attempting PDF compilation. Reticulating splines etc...

👉 Check article proof 📄 👈

Hi @adswa @FelixHenninger @JonathanReardon 👋 the actual review will take place in here

Hello, (1) could the list of dependencies be added to the README.md? It states that all will be installed with pip, but it might be better to explicitly state what those dependencies are. (2) Could the formatting of the top (author/affiliation etc.) section of paper.md be improved for clarity -- perhaps less whitespace, and no or less line-wrapping to make the information easier to read. (3) I don't see any guidelines for third parties wishing to contribute/report/seek support, if I've missed it could you direct me to it?

The documentation is very good, and the "example computation" is particularly useful (https://multimatch.readthedocs.io/en/latest/example.html). Do you state anywhere what you used to overlay the data onto movie frame image (Figure 1 and 2)? if not, I think it would be a useful information to add.

Hey @JonathanReardon,
Thanks for these helpful remarks.

The dependencies are now explicitly stated in the README and in the docs.

As for formatting the paper.md header section I'm unfortunately unsure how to do that, at the moment. @kyleniemeyer, is it possible to line-break those affiliations or similar?

Indeed, the "how to contribute" section was only a minor section in the README.md, and I have now added a CONTRIBUTING.md file (with guidelines on contributing, seeking help, and a code of conduct).

I have linked the script used to overlay gaze onto the stimulus content (its publicly available as part of the studyforrest publication) in the example in the docs - I agree that this is helpful information. Also, many thanks for reading the documentation so attentively!

The changes made so far are in this PR. I'll try to also add a header improvement to it, and will make a note here when I merge them into master.

Hi @adswa -- thank you for the swift response and the changes! I have now finished my part of the review (all boxes ticked). I thoroughly checked the tool and I found it very easy to use, it works exactly as stated. Docs/instructions were also thorough and very clear, which I very much appreciate. Note: I tested it on Linux Mint (19.1) -- I'm not sure what systems you've tested it on but it might be useful to know that it works on this distribution too. It's great to see another useful tool re-implemented for the open source world, congrats (to all involved)!

I'm not sure how it works with respect to the other reviewer, but from my end, I'm happy to approve for publication.

@whedon generate pdf

Attempting PDF compilation. Reticulating splines etc...

👉 Check article proof 📄 👈

@adswa @JonathanReardon if you mean the affiliation info in the PDF linked in the last comment, the format is part of the JOSS template. The Markdown file is just the original source, so don't worry about how things look in that.

(also, one comment for @adswa: could you add more detail to the affiliation info for your colleague Yaroslav? That one should have the same info, like department, city, state, country.)

@kyleniemeyer Yes -- I did mean the markdown file, but of course how raw md looks isn't very important! Sorry @adswa, that may have come across as incredibly picky (you can ignore that comment), the reality is, I forgot what I was looking at for a second :) thanks all!

@whedon generate pdf

Attempting PDF compilation. Reticulating splines etc...

👉 Check article proof 📄 👈

Thanks @kyleniemeyer, the affiliation is updated, and I merged all changes in this commit. No worries, @JonathanReardon! It did not come across in a bad way at all!

Have a good weekend everyone!

Thanks for your feedback @JonathanReardon!

@FelixHenninger have you had a chance to start your review?

Are there any updates yet, @kyleniemeyer? I just want to make sure I haven't missed a notification.

@adswa we are just waiting on @FelixHenninger's review now; I was in touch via email a few days ago and got a confirmation that it should be completed soon.

Thanks a lot!

Hi @FelixHenninger, just wanted to check in our your review—thanks!

Hej everyone, and sincerest apologies for the delay, which has been due to me being out of office in in July, followed by health issues on my part over the past ten days -- I'm very sorry for the trouble! My brief notes follow:

This project has been a genuine pleasure to review as the installation and use work flawlessly, and as a result I have only minor points to add to @JonathanReardon's excellent (and already-addressed) comments above. I emphatically agree that the documentation is fantastic, and also found the references describing the package's inner workings very clear and helpful when looking through the functionality. I had a couple of things that were unclear to me at first, but in every case it turned out that I hadn't read the docs carefully enough. 🙈

I found the manuscript very helpful and accessible, with a clear motivation and concise description of the package. I have only two minor formatting-related points that might be worth a second look:

• On lines 54/55, I think the citations might work better in the in-text format rather than in brackets.
• In the bibliography, I think that the PDF generation might have garbled the capitalization for Behavior Research Methods. There are also some minor inconsistencies with the DOI formatting, and some of the links don't work as a result (Burmeister & Mast, Dewhurst et al. and Johansson et al.).

There is one minor point I encountered that I think is the scope of the JOSS review, so please consider this as suggestion only (if they aren't covered elsewhere already -- I'd be happy to make an issue out of it if that's a better place for the information):

• It's really great to see CI testing in a project like this! 😍 One of the things I had missed initially is the automated testing, and thus I had tried running the test suite locally, which isn't (as far as I can see) documented -- it might be helpful for future contributors to briefly note how to trigger a test run manually.

Ok, I hope this makes sense -- again, thanks for putting together this super-neat package, and doing it with such care and visible attention to detail. If you, @adswa, could take another look at the manuscript formatting, I think this is a go from my perspective. 👍 Sorry again for the delay!

@adswa my only editorial request for the paper is to add

A summary describing the high-level functionality and purpose of the software for a diverse, non-specialist audience.

(This is from our documentation on what papers should contain.)

This could take multiple forms, but the simplest is probably to add a sentence or two explaining the application to a general reader at the very beginning.

Thanks a lot, @kyleniemeyer, I've included a short description at the very beginning into the introduction in this PR:

multimatch-gaze is a Python package for computing the
similarity of scan paths from eye gaze data on five different dimensions.

Let me know if that suffices at as a high-level functionality description, or if that is not detailed enough. Thanks!

@adswa this is better, but I think it still needs a bit more general explanation for non-experts—for example, I don't really know what scan paths are or what they are used for. Could you add a sentence explaining that?

Thanks a lot for the feedback, @kyleniemeyer! I made another attempt in this PR

@whedon generate pdf

Attempting PDF compilation. Reticulating splines etc...

👉 Check article proof 📄 👈

@adswa that looks great! I just made a minor edit in this PR, if you could merge: adswa/multimatch_gaze#39

Once that is complete, please archive the entire repository (e.g., using Zenodo) and let me know the DOI here.

All done, it is 10.5281/zenodo.3369531.

@whedon generate pdf

Attempting PDF compilation. Reticulating splines etc...

👉 Check article proof 📄 👈

@adswa sorry, one last thing—can you edit the title of the Zenodo archive to match the title of the paper?

oh sure! sorry, I wasn't aware. Should be done now.

@whedon set 10.5281/zenodo.3369531 as archive

OK. 10.5281/zenodo.3369531 is the archive.

@openjournals/joss-eics ok, this submission is ready to accept!

@whedon accept

Attempting dry run of processing paper acceptance...

Check final proof 👉 openjournals/joss-papers#905

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#905, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

👋 @kyleniemeyer - I think @openjournals/joss-eics members can do this themselves, even if not on duty - at least @arfon and I have in the past. There's certainly no requirement to, but it's an option if you want to.

@whedon accept deposit=true

Doing it live! Attempting automated processing of paper acceptance...

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 openjournals/joss-papers#906
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01525
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

Thanks to @FelixHenninger and @JonathanReardon for reviewing and @kyleniemeyer for editing!

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01525/status.svg)](https://doi.org/10.21105/joss.01525)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01525">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01525/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01525/status.svg
   :target: https://doi.org/10.21105/joss.01525

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-joss

Thanks a lot to everyone involved! This has been a great experience! 🎉