Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update jobs to use aut-0.60.0. #387

Merged
merged 1 commit into from Apr 16, 2020
Merged

Update jobs to use aut-0.60.0. #387

merged 1 commit into from Apr 16, 2020

Conversation

@ruebot
Copy link
Member

ruebot commented Apr 16, 2020

  • Resolves #386
  • Move the faux txt derivatives to what they actually are; csv.
  • Update Spark job to use DataFrames
  • Update auk documentation and lessons with correct file extension
    (s/txt/csv)
  • Data migration needs to be completed on prod
    • rename full-text and full-domains
      • s/.txt/.csv/g
    • on all -fullurls.txt
      • remove the first and last character on each line. ( )

@SamFritz @ianmilligan1 can you review the text/documentation changes? We should be good to go, but just want to make sure. Once I get approval from you two, I'll merge this, deploy it to production, and do the data migration.

- Resolves #386
- Move the faux txt derivatives to what they actually are; csv.
- Update Spark job to use DataFrames
- Update auk documentation and lessons with correct file extension
(s/txt/csv)
- Data migration needs to be completed on prod
  - rename full-text and full-domains
    - s/.txt/.csv/g
  - on all -fullurls.txt
    - remove the first and last character on each line. ( )
- TravisCI should only test Ruby 2.6.5
- Update tests to reflect changes
- Rename text fixtures
@ruebot ruebot force-pushed the issue-386 branch from 95dd90e to 5dd2b18 Apr 16, 2020
@codecov

This comment has been minimized.

Copy link

codecov bot commented Apr 16, 2020

Codecov Report

Merging #387 into master will increase coverage by 0.25%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #387      +/-   ##
==========================================
+ Coverage   89.15%   89.40%   +0.25%     
==========================================
  Files          35       35              
  Lines         636      623      -13     
==========================================
- Hits          567      557      -10     
+ Misses         69       66       -3     
Impacted Files Coverage Δ
app/helpers/wasapi_files_helper.rb 87.50% <ø> (-0.50%) ⬇️
app/helpers/collections_helper.rb 100.00% <100.00%> (ø)
test/helpers/collections_helper_test.rb 100.00% <100.00%> (ø)
app/mailers/user_mailer.rb 100.00% <0.00%> (ø)
app/controllers/dashboards_controller.rb 39.70% <0.00%> (+1.13%) ⬆️
app/helpers/users_helper.rb 91.66% <0.00%> (+7.05%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e1e713d...5dd2b18. Read the comment docs.

Copy link
Member

ianmilligan1 left a comment

Tiny change to consider, but otherwise looks great!

@@ -55,7 +55,7 @@
<p class="top_toc"><%=link_to('Back to Top', anchor: '#toc-top') %></p>

<h3 class="about_h3" id="5toc">The domain derivative download</h3>
<p class="about_p">By clicking on the "Domains" button, users are provided with a text file that will be named something like 7485-fullurls.txt. You can open this file up in your text editor of choice. By default on MacOS, it will open in TextEdit and look like this:</p>
<p class="about_p">By clicking on the "Domains" button, users are provided with a text file that will be named something like 7485-fullurls.csv. You can open this file up in your text editor of choice. By default on MacOS, it will open in TextEdit and look like this:</p>

This comment has been minimized.

Copy link
@ianmilligan1

ianmilligan1 Apr 16, 2020

Member

text file -> comma-separated value file
it will open in TextEdit and look like this -> it will open in a spreadsheet program (Excel or Numbers) and look like this

@@ -55,7 +55,7 @@
<p class="top_toc"><%=link_to('Back to Top', anchor: '#toc-top') %></p>

<h3 class="about_h3" id="5toc">The domain derivative download</h3>
<p class="about_p">By clicking on the "Domains" button, users are provided with a text file that will be named something like 7485-fullurls.txt. You can open this file up in your text editor of choice. By default on MacOS, it will open in TextEdit and look like this:</p>
<p class="about_p">By clicking on the "Domains" button, users are provided with a text file that will be named something like 7485-fullurls.csv. You can open this file up in your text editor of choice. By default on MacOS, it will open in TextEdit and look like this:</p>
<%= image_tag("Tutorial_domain_derivative_file.png", alt: "Screenshot of the domain derivative file in MacOS TextEdit", class:"body_img")%>

This comment has been minimized.

Copy link
@ianmilligan1

ianmilligan1 Apr 16, 2020

Member

let's swap out the screenshot? If you think this is a good idea I can send it to you via Slack @ruebot (or you can grab it from GitHub below)

alt text: "Screenshot of the domain derivative file in Microsoft Excel"

Screen Shot 2020-04-16 at 3 29 26 PM

@ruebot ruebot merged commit d09cf48 into master Apr 16, 2020
2 checks passed
2 checks passed
codecov/patch 100.00% of diff hit (target 89.15%)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@ruebot ruebot deleted the issue-386 branch Apr 16, 2020
ianmilligan1 added a commit that referenced this pull request Apr 16, 2020
ruebot pushed a commit that referenced this pull request Apr 16, 2020
* Minor tweaks to domain page (follow up on #387)
* Suggest using "Import"; text -> CSV
* Updating image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.