Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Spark Job with new WriteGraph; Resolves #211 #212

Merged
merged 4 commits into from Aug 26, 2019

Conversation

@ianmilligan1
Copy link
Member

commented Nov 28, 2018

This should not be merged until AUT 0.18 is released with the new WriteGraph functionality

GitHub issue(s):

#211

What does this Pull Request do?

This is a relatively straight-forward pull request that changes the syntax in the Spark Job from the older WriteGraphML to the newer WriteGraph.asGraphml. This has the primary effect of turning node IDs for the networks from long hashes to simple numbers.

It also updates the application.yml.example as if users are using a version before 0.18.0 the command won't work.

I figured this should be done on a branch as I wanted to particularly make sure that the ensuing networks work well in Gephi. They appear to do so.

How should this be tested?

I have tested locally but basically we will want to see that (a) the standard suite of derivatives are generated; and (b) the ensuing network files work. The two screenshots show some local testing, showing how the new graphs render well after GraphPass and also work well in Gephi.

screen shot 2018-11-28 at 11 14 28 am

screen shot 2018-11-28 at 11 18 10 am

Additional Notes:

As noted above, only merge once new AUT is released. This should not require new documentation.

Interested parties

@ruebot @greebie

ianmilligan1 added some commits Nov 28, 2018

@ianmilligan1 ianmilligan1 requested a review from ruebot Nov 28, 2018

@codecov-io

This comment has been minimized.

Copy link

commented Nov 28, 2018

Codecov Report

Merging #212 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #212   +/-   ##
======================================
  Coverage    89.4%   89.4%           
======================================
  Files          35      35           
  Lines         623     623           
======================================
  Hits          557     557           
  Misses         66      66

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e523b6...5a3df25. Read the comment docs.

@ruebot ruebot added the on hold label Dec 11, 2018

ruebot added some commits Aug 26, 2019

@ruebot

ruebot approved these changes Aug 26, 2019

@ruebot ruebot merged commit a10db14 into master Aug 26, 2019

3 checks passed

codecov/patch Coverage not affected when comparing 5e523b6...5a3df25
Details
codecov/project 89.4% remains the same compared to 5e523b6
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@ruebot ruebot deleted the issue-211 branch Aug 26, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.