Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign up[REVIEW]: Automated Sleep Stage Scoring Using k-Nearest Neighbors Classifier #2377
Comments
|
Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @Emma-k-ward, @sbuergers it looks like you're currently assigned to review this paper Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post. If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews To fix this do the following two things:
For a list of things I can do to help you, just type: For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type: |
|
Hey @Emma-k-ward, @sbuergers, and @trisbek this is where the main review occurs, if you need to open something very technical as an issue please do so as an issue at the original repo but all other comments — so most, if not all, of the review — should be here. Hope this is all clear but please ask me any questions you may have! |
whedon
assigned
Emma-k-ward,
oliviaguest and
sbuergers and unassigned
oliviaguest,
sbuergers and
Emma-k-ward
|
OK, @trisbek is now a reviewer |
|
Hey @teamPSG, This is a nicely written and well documented toolbox, well done! I used Matlab 2019b for testing and the SoftwareVerification ran without problems. From what I could gather there are only a few points to consider: Contribution and authorship: From the commit history it is unclear who, apart from Tamás Kiss, contributed to the software. Can you include a statement of contributions for the other authors? References: The references in the main text do not render properly (e.g. [@STEPHENSON2009263,@BASTIANINI2014277...]). It might be that this can simply be fixed by putting a semicolon instead of a comma in-between authors, see this guide. Community guidelines: I did not see clear guidelines for third-parties wishing to: i.) Contribute to the software, ii.) Report issues or problems with the software (other than contacting the author, but it would be useful to have some pointers for how to use github for this). There are plenty of examples here on github that do this well, for instance here. @oliviaguest, thanks for setting us up! I take it these comments should not go in the issues of the project? Cheers, |
|
@sbuergers thank you for your feedback so far! These seem to be appropriate to stay here — although feel free to talk to @teamPSG on how to organise these, of course! |
|
@whedon generate pdf |
|
Hi @sbuergers, Thanks for your feedback and really fast action! Glad you like the toolbox and thanks for the suggestions. Contributions and authorship: I included a section on who did what at the end of the manuscript. References: thanks for the hint, it worked, references are now rendered properly (during writing I checked the ms offline using the local compiler and it looked good there -- some components of the compilation workflow might have changed in the course of development...) Community guidelines: good point, I missed this one. I added a CONTRIBUTING.md to the repo. I guess the comments are fine right here. Thanks for the review! Best, |
|
Hi @teamPSG, thanks for implementing the feedback! This looks good to me now. Best, |
|
Hey @Emma-k-ward and @trisbek can you give me a rough ETA for your reviews? I ask not to hurry you but just to be organised. Thank you. |
|
Hi @oliviaguest, have you received feedback on ETA from @Emma-k-ward and/or @trisbek? (The text editor doesn't auto-complete the mention for_trisbek_ for me -- does it mean anything?) Thanks! |
|
I have not been able to run anything on my house PC so I will check with
Samika later (she is in californian time) and let you know/
tris
El lun., 6 jul. 2020 a las 9:42, teamPSG (<notifications@github.com>)
escribió:
… Hi @oliviaguest <https://github.com/oliviaguest>, have you received
feedback on ETA from @Emma-k-ward <https://github.com/Emma-k-ward> and/or
@trisbek <https://github.com/trisbek>? (The text editor doesn't
auto-complete the mention for_trisbek_ for me -- does it mean anything?)
Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2377 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQCEF2ANW6UI6AEQT5ITMSTR2GE6NANCNFSM4OETIXNQ>
.
|
|
@trisbek great! Please tag Samika's username so we can give due credit and assign as reviewer if they so wish, thanks. |
|
@whedon generate pdf |
|
@teamPSG are you making any edits at the moment for this? |
|
@oliviaguest no, I have not touched the ms since my response to Steffen (17 days ago according to GitHub) |
|
OK, no problem and let us wait for the reviewers to get back to us. If they don't give a vague ETA within a week or so, I'll email them. |
|
Samika ran the code using Matlab 2018B on my behalf. The Software Verification scripts run smoothly and quickly. Function Library scripts look very clearly documented. Functionality/Performance: The paper states the algorithm's potential to stage sleep across species and genetic/drug manipulations for large animal cohorts, and it states the algorithm has been used already on mice, rats, and non-human primates. But the paper (and Github) only include summary results for 6 knockout mice. If it's already available, it would be nice to see the prediction accuracy output for a larger sample size or at least on other species/conditions to confirm the algorithm's versatility. (Optional) The algorithm has a nice "deflated single model" to account for imbalanced data (e.g., too few REM epochs). Compared to the original single model, this deflated model results in improved REM prediction accuracy but slightly decreased Wake/NREM accuracy. If I am interested in achieving the greatest accuracy for all sleep stages, it's not obvious to me how I would determine which model to use in the end. It might be helpful to have a small script that outputs which model maximizes the true positive rate for most subjects. |
|
I have added comments but was unable to tick the boxes on read the PDF today
El lun., 13 jul. 2020 a las 11:50, Olivia Guest (<notifications@github.com>)
escribió:
… OK, no problem and let us wait for the reviewers to get back to us. If
they don't give a vague ETA within a week or so, I'll email them.
|
|
The reviewer already has a pending invite. @trisbek please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations |
|
Invite me again?
[image: image.png]
El mar., 14 jul. 2020 a las 14:58, whedon (<notifications@github.com>)
escribió:
… The reviewer already has a pending invite.
@trisbek <https://github.com/trisbek> please accept the invite by
clicking this link:
https://github.com/openjournals/joss-reviews/invitations
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2377 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQCEF2G4TZ6SEQKVROZ4KADR3RQBRANCNFSM4OETIXNQ>
.
|
|
@trisbek I cannot AFAIK, you have to be logged in and use the link above. |
|
Perhaps another @openjournals/joss-editors can lend me a helping hand, please? I assume that @whedon is correct and all @trisbek needs to do is click that link? |
|
right - clicking the link should work |
|
Also @trisbek (sorry I keep asking!) can you clarify if Samika wishes to be included as a reviewer? How much of this code and paper have they reviewed, and do they want credit attributed to them for their work? |
|
Hi,
it does not let me accept and it does not let me see the pdf
[image: image.png]
Samika said, next time she will upload her scripts and OSF and the lot to a
new account of github but this time she is happy for me to copypaste her
comments.
El mar., 14 jul. 2020 a las 15:57, Olivia Guest (<notifications@github.com>)
escribió:
… Also @trisbek <https://github.com/trisbek> (sorry I keep asking!) can you
clarify if Samika wishes to be included as a reviewer? How much of this
code and paper have they reviewed, and do they want credit attributed to
them for their work?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2377 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQCEF2DNX4RHN2HBPXG7NXLR3RW7BANCNFSM4OETIXNQ>
.
|
|
The pdf file is visible without being logged in. @trisbek did you follow https://github.com/openjournals/joss-papers/blob/joss.02377/joss.02377/10.21105.joss.02377.pdf for the pdf? |
|
@trisbek We have discussed this case (the whole editorial board) as this has not happened at JOSS before (that a reviewer copy/pastes another reviewer's words). We have decided that Samika needs to make a GitHub account (if she doesn't have one already) so that she can receive credit (this should take a few minutes only). Samika does not have to write a separate review since she already did, she merely has to tick the boxes at the top of this issue. This should be very quick and easy — we apologise for bringing this up now but we believe it's open and transparent to do the above. We will email her a link to this GitHub issue so she can comment here with her username. Thank you! |
|
Hi all - apologies for the confusion! I've reviewed the scripts and paper, and my comments are above in Tristan's message. I'm happy to tick the boxes once I am added. Thanks. |
Submitting author: @teamPSG (Tamas Kiss)
Repository: https://github.com/teamPSG/kNN_Sleep_Scorer_kNNSS
Version: v1.0.0
Editor: @oliviaguest
Reviewers: @Emma-k-ward, @sbuergers, @trisbek
Archive: Pending
Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@Emma-k-ward & @sbuergers, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @oliviaguest know.
Review checklist for @Emma-k-ward
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
Review checklist for @sbuergers
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
Review checklist for @trisbek
Conflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper