Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upAdd binary extraction DataFrames to PySpark. #350
+77
−4
Conversation
ruebot
requested a review
from ianmilligan1
Aug 20, 2019
This comment has been minimized.
This comment has been minimized.
codecov
bot
commented
Aug 20, 2019
•
Codecov Report
@@ Coverage Diff @@
## master #350 +/- ##
==========================================
- Coverage 75.52% 74.76% -0.77%
==========================================
Files 39 39
Lines 1373 1387 +14
Branches 265 265
==========================================
Hits 1037 1037
- Misses 220 234 +14
Partials 116 116 |
This comment has been minimized.
This comment has been minimized.
|
Here's an even better test: https://github.com/archivesunleashed/aut/wiki/Using-AUT-with-PySpark (Just swap out 0.18.0 with the path to |
lintool
reviewed
Aug 21, 2019
This comment has been minimized.
This comment has been minimized.
|
Comments about naming df's if it's not too late... |
ianmilligan1
approved these changes
Aug 21, 2019
|
Ran through the PySpark documentation with the Jupyter command in this PR. All worked perfectly! |
ianmilligan1
merged commit eda185b
into
master
Aug 21, 2019
1 check passed
continuous-integration/travis-ci/pr
The Travis CI build passed
Details
ianmilligan1
deleted the
images-pyspark
branch
Aug 21, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
ruebot commentedAug 20, 2019
GitHub issue(s):
What does this Pull Request do?
Add binary extraction DataFrames to PySpark.
How should this be tested?
Then do this for each of the additions, and make sure it works:
Additional Notes:
Let's see what happens with the test coverage. I assume I'm going to have to add some things to
src/test/scala/io/archivesunleashed/df/DataFrameLoaderTest.scala