Stubbing out code for editor votes #764

arfon · 2020-07-01T17:47:35Z

This PR implements a possible way for the JOSS editorial team to make decisions about editorial scope. Our current approach for flagging/discussing whether a paper is in scope is very noisy for editors (messages posted to the Google group), hard to keep track of, and prone to errors.

The idea is that any pre-review or review issue labeled with query-scope will automatically be included in the twice-weekly emails to all editors to prompt them to vote.

Specifically this PR implements

A UI addition to provide a way for editors to vote on whether or not a submission is in scope:

Vote summaries added to the dashboard views:

A summary email that is sent to editors on Mondays and Thursdays:


        Stubbing out code for editor votes


        Use the right label

arfon · 2020-07-01T17:51:00Z

app/controllers/votes_controller.rb

+      flash[:notice] = "Vote recorded"
+      redirect_to paper_path(@paper)
+    # Can't vote on the same item twice
+    rescue ActiveRecord::RecordNotUnique => e


Each editor can only vote on each paper once 😬

arfon · 2020-07-01T17:51:00Z

app/helpers/home_helper.rb

@@ -37,6 +37,9 @@ def pretty_labels_for(paper)
          concat content_tag(:span, label, style: "padding: 3px; margin-right: 3px; border-radius: 2px; background-color: ##{colour}; color: white;")
        elsif label == "recommend-accept"
          concat content_tag(:span, label, style: "padding: 3px; margin-right: 3px; border-radius: 2px; background-color: ##{colour}; color: #000000;")
+        elsif label == "query-scope"


This change means that papers labeled with query-scope on the joss-reviews repository will show the label in the dashboard.

arfon · 2020-07-01T17:51:00Z

lib/tasks/editorials.rake

+  task send_query_scope_email: :environment do
+    if Time.now.monday? || Time.now.thursday?
+      reviews_repo = Rails.application.settings["reviews"]
+      review_issues = ReviewIssue.download_review_issues(reviews_repo, 'query-scope')


Again, query-scope is the label we're including in the summary emails.

arfon · 2020-07-01T17:51:00Z

spec/models/vote_spec.rb

+require 'rails_helper'
+
+RSpec.describe Vote, type: :model do
+  let(:vote) { create(:vote) }


#TODO write some tests...

arfon · 2020-07-01T17:52:53Z

/ cc @openjournals/joss-eics @openjournals/joss-editors

danielskatz · 2020-07-01T18:10:21Z

Do we want a whedon command to turn on the query-scope label? Maybe not...

danielskatz · 2020-07-01T18:10:41Z

otherwise, I like the idea of this

terrytangyuan

Love this. The mailing list can be much less verbose!

kthyng · 2020-07-01T19:35:20Z

I like this a lot! Small thing: could the scope emojis only show up in the dashboard if the label has been activated for that submissions? Might make it cleaner and easier to see which are being discussed. Also, would there be a place that aggregates all of the submissions with this label (other than the email)?

eloisabentivegna · 2020-07-01T19:36:46Z

Brilliant feature!

terrytangyuan · 2020-07-01T19:40:16Z

Two questions

How often do we receive emails like this from noreply@joss?
Maybe I missed it somewhere but where can we make comments instead of or along with voting given that the email we receive cannot be replied?

arfon · 2020-07-01T19:51:49Z

Do we want a whedon command to turn on the query-scope label? Maybe not...

Something like @whedon flag scope or something like that? This might be a good way to ensure the correct label is applied, plus one could imagine using this in combination with asking the author to say more about the software on the pre-review issue.

arfon · 2020-07-01T19:54:11Z

How often do we receive emails like this from noreply@joss?

@terrytangyuan - the idea would be that these summary emails would be sent twice per week. Mondays and Thursdays and would only include those papers that have been flagged as possibly out of scope.

Maybe I missed it somewhere but where can we make comments instead of or along with voting given that the email we receive cannot be replied?

Yes! Just zooming in a little on one of the screenshots from above, the idea would be that you can include a short (or long) comment with your vote in the text box:

arfon · 2020-07-01T19:54:59Z

I like this a lot! Small thing: could the scope emojis only show up in the dashboard if the label has been activated for that submissions? Might make it cleaner and easier to see which are being discussed. Also, would there be a place that aggregates all of the submissions with this label (other than the email)?

Yes, that's a good idea. Perhaps we should just have a dedicated tab on the dashboard for these submissions?

labarba · 2020-07-01T19:59:00Z

the idea would be that these summary emails would be sent twice per week. Mondays and Thursdays and would only include those papers that have been flagged as possibly out of scope.

I really think once per week is fine. No one, NO ONE, reacts to submissions at such a fast clip as we do. We are creating a monster.

jgostick · 2020-07-01T20:15:31Z

It seems to me, in my brief 2 weeks under the deluge of these email conversations, that both out of scope and lack of size are issues that need input. In principle it might be possible to answer the size question quantitatively (like comparing the LOC to other projects in the same language), but this is probably most easily done just by asking someone familiar with the language. In other words, should this feature do double duty for scope and size?

gkthiruvathukal · 2020-07-01T20:30:25Z

@jgostick In a significant number of cases, I would think "lack of size" is also "lack of scope". Most of these end up being minor utilities and, therefore, not in scope.

This is a welcome addition, @arfon!


        Send on a Wednesday only

danielskatz · 2020-07-01T21:52:33Z

lib/tasks/editorials.rake

@@ -23,9 +23,9 @@ namespace :editorials do
  end

  desc "Send twice-weekly emails with possible submission rejections"
-  # Send on Monday and Thursday
+  # Send on Wednesday


Let's talk about this tomorrow - I would like to not only do this once a week, just because of the backlog that could build up and be carried over from one AEiC to another. At least, we should think about the schedule of this with respect to AEiC rotation

Sorry, we forgot to talk about this today.

My rationale for picking Wednesday was that this would be a couple of days after the start of an EiC rotation so potentially the incoming EiC would have had a chance to review the new submissions and flag any that potentially might be out of scope.

One alternative might be to send this on Sunday at the end of a rotation?


        Fixing the build

danielskatz · 2020-07-02T15:17:07Z

Once we do merge this in, we should check for open issues such as #709 that this PR might enable us to close


        show vote summary only for papers labeled as 'query-scope' (#765)


        Basic unit tests for voting

arfon added 2 commits Jul 1, 2020

Stubbing out code for editor votes

Verified

This commit was signed with a verified signature.

arfon Arfon Smith

GPG key ID: 29DFFF5280423804 Learn about signing commits

Loading status checks…

a88dc53

Use the right label

Verified

This commit was signed with a verified signature.

arfon Arfon Smith

GPG key ID: 29DFFF5280423804 Learn about signing commits

Loading status checks…

736be20

arfon reviewed Jul 1, 2020

View changes

terrytangyuan reviewed Jul 1, 2020

View changes

Send on a Wednesday only

Verified

This commit was signed with a verified signature.

arfon Arfon Smith

GPG key ID: 29DFFF5280423804 Learn about signing commits

Loading status checks…

21dfabb

danielskatz reviewed Jul 1, 2020

View changes

Fixing the build

Verified

This commit was signed with a verified signature.

arfon Arfon Smith

GPG key ID: 29DFFF5280423804 Learn about signing commits

Loading status checks…

a86314b

xuanxu and others added 2 commits Jul 2, 2020

show vote summary only for papers labeled as 'query-scope' (#765)

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

e1a05c2

Basic unit tests for voting

Verified

This commit was signed with a verified signature.

arfon Arfon Smith

GPG key ID: 29DFFF5280423804 Learn about signing commits

Loading status checks…

7ed605c

openjournals / joss

Stubbing out code for editor votes #764

Stubbing out code for editor votes #764

arfon commented Jul 1, 2020 •

edited

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

arfon commented Jul 1, 2020

danielskatz commented Jul 1, 2020

danielskatz commented Jul 1, 2020

terrytangyuan left a comment

kthyng commented Jul 1, 2020

eloisabentivegna commented Jul 1, 2020

terrytangyuan commented Jul 1, 2020 •

edited

arfon commented Jul 1, 2020

arfon commented Jul 1, 2020

arfon commented Jul 1, 2020

labarba commented Jul 1, 2020

jgostick commented Jul 1, 2020

gkthiruvathukal commented Jul 1, 2020

This comment has been minimized.

This comment has been minimized.

danielskatz commented Jul 2, 2020

openjournals / joss

Sponsor openjournals/joss

Join GitHub today

Stubbing out code for editor votes #764

Stubbing out code for editor votes #764

Conversation

arfon commented Jul 1, 2020 • edited

This comment has been minimized.

arfon Jul 1, 2020

This comment has been minimized.

arfon Jul 1, 2020

This comment has been minimized.

arfon Jul 1, 2020

This comment has been minimized.

arfon Jul 1, 2020

arfon commented Jul 1, 2020

danielskatz commented Jul 1, 2020

danielskatz commented Jul 1, 2020

terrytangyuan left a comment

kthyng commented Jul 1, 2020

eloisabentivegna commented Jul 1, 2020

terrytangyuan commented Jul 1, 2020 • edited

arfon commented Jul 1, 2020

arfon commented Jul 1, 2020

arfon commented Jul 1, 2020

labarba commented Jul 1, 2020

jgostick commented Jul 1, 2020

gkthiruvathukal commented Jul 1, 2020

This comment has been minimized.

danielskatz Jul 1, 2020

This comment has been minimized.

arfon Jul 2, 2020

danielskatz commented Jul 2, 2020

arfon commented Jul 1, 2020 •

edited

terrytangyuan commented Jul 1, 2020 •

edited