Skip to content

/ aut-docs

Permalink
Tree: 36d86baa59
Commits on Feb 10, 2020

  1. Documentation update for archivesunleashed/aut#422 (#47)

    @ruebot
    ruebot committed Feb 10, 2020
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    36d86ba

  2. Update to 0.50.0 and DataFrames where applicable. (#44)

    @ruebot
    ruebot committed Feb 10, 2020 
    * Update to 0.50.0 and DataFrames where applicable.
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    928523f

  3. Add DataFrame schemas; resolves #45. (#46)

    @ruebot
    ruebot committed Feb 10, 2020 
    * Add DataFrame schemas; resolves #45.
* Add 0.50.0 version, and rename to imagegraph in current
* remove redundant doc
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    c7b99dd
Commits on Feb 6, 2020

  1. Document keepValidPagesDF() (#43)

    @ruebot
    ruebot committed Feb 6, 2020 
    * Document keepValidPagesDF()
* Words are hard.
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    0ffe446
Commits on Feb 5, 2020

  1. 0.50.0 release

    @ruebot
    ruebot committed Feb 5, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    3c8630f

  2. Code formatting and code consistency review. (#42)

    @ruebot
    ruebot committed Feb 5, 2020 
    * Code formatting and code consistency review.

- Resolves #29

* review
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    245aab4

  3. Document .all(); resolves #26. (#41)

    @ruebot
    ruebot committed Feb 5, 2020
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    9acbe3e
Commits on Feb 3, 2020

  1. Update udf names for binary analysis. (#40)

    @ruebot
    ruebot committed Feb 3, 2020
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    536f277
Commits on Jan 20, 2020

  1. Update documentation for archivesunleashed/aut#372 (#39)

    @ruebot
    ruebot committed Jan 20, 2020 
    * Add DF results Python section
* Add won't implement language to binary analysis.
* Add won't implement language to standard derivatives.
* Remove index, fix ToC in setting up.
* Update README, add scala df to link analys, add TSV rdd results
* text-analysis scala df
* Add to be implemented
* Resolves #22
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    4ce3cb5
Commits on Jan 15, 2020

  1. Add discardMimeTypesTika, and tweak other MIME type examples. (#38)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 15, 2020 
    - Resolves #32
- Addresses archivesunleashed/aut#223
- Addresses archivesunleashed/aut#401
- Addresses archivesunleashed/aut#396
- Addresses archivesunleashed/aut#399
    45de55b
Commits on Jan 13, 2020

  1. More DF filter updates. (#37)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 13, 2020 
    - Addresses #32
- Addresses archivesunleashed/aut#223
- Addresses archivesunleashed/aut#401
- Addresses archivesunleashed/aut#396
- Addresses archivesunleashed/aut#399
    8ed292b

  2. Scala DF filter updates for a few aut PRs. (#36)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 13, 2020 
    - Addresses #32
- Addresses archivesunleashed/aut#223
- Addresses archivesunleashed/aut#401
- Addresses archivesunleashed/aut#396
- Addresses archivesunleashed/aut#399
    a59904a
Commits on Jan 8, 2020

  1. Add extract-simple-site-link-structure DF example. (#35)

    @ruebot
    ruebot committed Jan 8, 2020 
    * Add extract-simple-site-link-structure DF example.
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    4186cd9
Commits on Dec 18, 2019

  1. Update filters documentation for https://github.com/archivesunleashed… (

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Dec 18, 2019 
    #33)

* Update filters documentation for archivesunleashed/aut#391

- Add ToC
- Add Scala RDD, Scala DF, and Python DF sections

* review
    19d8607
Commits on Dec 17, 2019

  1. Change extractValidPagesDF -> webpages (#31)

    @ianmilligan1 @ruebot
    ianmilligan1 authored and ruebot committed Dec 17, 2019
    c2e6956
Commits on Dec 5, 2019

  1. Updates for archivesunleashed/aut#387 (#30)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Dec 5, 2019 
    * Updates for archivesunleashed/aut#387

* Missed some in #24
    955f7bd
Commits on Nov 26, 2019

  1. Add "Find Images Shared Between Domains" section. (#27)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 26, 2019 
    * Add "Find Images Shared Between Domains" section.

- Resolves archivesunleashed/aut#237

* review
    e774c6c
Commits on Nov 22, 2019

  1. Add example for Scala DF version of "Extract Most Frequent Images MD5… (

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 22, 2019 
    #28)

* Add example for Scala DF version of "Extract Most Frequent Images MD5 Hash".

- See archivesunleashed/aut#382

* rename
    9cffe13
Commits on Nov 21, 2019

  1. Rename pages() to webpages(). (#25)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 21, 2019 
    - See: archivesunleashed/aut#384
    e162384
Commits on Nov 19, 2019

  1. Documentation update for archivesunleashed/aut#381 (#24)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 19, 2019
    fdbd1be

  2. Move cookbook to standard derivatives guide (#21)

    @ruebot
    ruebot committed Nov 19, 2019 
    - Update all current cookbook examples to follow documentation style
guide
- Add Parquet, CSV, S3, and Python DF examples
- Update index
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    d8e71e1
Commits on Nov 12, 2019

  1. Add example for archivesunleashed/aut#377 / h… (#20)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 12, 2019 
    * Add example for archivesunleashed/aut#377 / archivesunleashed/aut#238

* review
    930f8c8
Commits on Nov 7, 2019

  1. Updates for changing RemoveHttpHeader to RemoveHTTPHeader. (#19)

    @SinghGursimran @ruebot
    SinghGursimran authored and ruebot committed Nov 7, 2019 
    - Add ScalaDF example for: Extract Plain Text Without HTTP Headers
- See also:
   - archivesunleashed/aut#368
   - archivesunleashed/aut#374
   - archivesunleashed/aut#370
    4f73504
Commits on Nov 6, 2019

  1. Better definition of what AUT is. (#18)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 6, 2019
    f40abd4
Commits on Nov 5, 2019

  1. Update NER documentation for archivesunleashed/aut#361 (#17)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 5, 2019
    10c2be5
Commits on Oct 28, 2019

  1. Incorporate PySpark setup into overall documentation. (#16)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Oct 28, 2019 
    * Incorporate PySpark setup into overall documentation.

- Removes standalone PySpark documentation.
- Incorporates PySpark setup into getting started documentation.
- Incorporates PySpark examples into overall documentation.
- Breaks out scaling documentation to it's own documentation.
- Removes cruft.
- Renames files so they're all lowercase now.
- Updates README ToC
    4cc429a
Commits on Oct 26, 2019

  1. Update to ToC to be more consistent after #11. (#13)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Oct 26, 2019
    48c1331

  2. Add binary analysis (#11)

    @ruebot
    ruebot committed Oct 26, 2019 
    - Add documentation for binary analysis and extraction in Scala DF and
Python DF
- Add Scala DF and Python DF version of extractImageLinks
- Update main ToC
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    978d0ab
Commits on Oct 25, 2019

  1. Fix link-analysis ToC links. (#12)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Oct 25, 2019
    53e0f63
Commits on Oct 23, 2019

  1. Fixed Table of Content on Current Doc README (#10)

    @ianmilligan1
    ianmilligan1 committed Oct 23, 2019 
    * Fixing table of content links

* Adding more relative links

* Adding image (fixing existing markdown)

* Removing image on seeing it rendered
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    4a8955e
Commits on Oct 21, 2019

  1. Adds ToC subheadings; changes collection analysis headings (#9)

    @ianmilligan1
    ianmilligan1 committed Oct 21, 2019
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    5173748

  2. Changed text-analysis.md to use consistent phrasing (#8)

    @lintool
    lintool committed Oct 21, 2019 
    Heading changed to verb phrases so it fits with "How do I..."
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    5416f8f

  3. Delete unneeded files (#7)

    @lintool
    lintool committed Oct 21, 2019
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    5112ffa

  4. Refactoring Documentation for Explanations and Consistent Structure (#5)

    @ianmilligan1 @ruebot
    ianmilligan1 authored and ruebot committed Oct 21, 2019 
    - Flesh out root README with a site-wide table of contents;
- Provide some basic introduction;
- Provide some context on RDD/DF; and
- Break the large "getting started and overview" document into at least two parts.
    95a7559

  5. Wrote up beginnings of 'what to do with results' (#3)

    @lintool
    lintool committed Oct 21, 2019
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    200b4f1
Older
You can’t perform that action at this time.