New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix image documentation to make prefix clear #87

Closed
ianmilligan1 opened this Issue Jan 9, 2019 · 0 comments

Comments

Projects
None yet
1 participant
@ianmilligan1
Member

ianmilligan1 commented Jan 9, 2019

From Slack @ruebot noted that if you say

import io.archivesunleashed._
import io.archivesunleashed.df._
val df = RecordLoader.loadArchives("/tuna1/scratch/nruest/geocites/warcs/9/image_test/*", sc).extractImageDetailsDF();
val res = df.select($"bytes").orderBy(desc("bytes")).saveToDisk("bytes", "/tuna1/scratch/nruest/geocites/images/")

Then in the images directory you get a bunch of files with the preceding -.

We should refine and then document this example so that it is say

val res = df.select($"bytes").orderBy(desc("bytes")).saveToDisk("bytes", "/tuna1/scratch/nruest/geocites/images/geo")

where each image would then be geo-HASH.gif etc.

Opening an issue just so I don't forget to do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment