Skip to content
Please note that GitHub no longer supports your web browser.

We recommend upgrading to the latest Google Chrome or Firefox.

Learn more

/ aut-docs

Permalink
Tree: 47a96a8301
Commits on Jan 17, 2020

  1. review

    @ruebot
    ruebot committed Jan 17, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    47a96a8

  2. more clean-up

    @ruebot
    ruebot committed Jan 17, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    d9441a9

  3. Add to be implemented; #22

    @ruebot
    ruebot committed Jan 17, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    9c34b18

  4. text-analysis scala df

    @ruebot
    ruebot committed Jan 17, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    9b6fe02

  5. Update README, add scala df to link analys, add TSV rdd results

    @ruebot
    ruebot committed Jan 17, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    b489e55
Commits on Jan 16, 2020

  1. Remove index, fix ToC in setting up.

    @ruebot
    ruebot committed Jan 16, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    3475498

  2. Add won't implement language to standard derivatives.

    @ruebot
    ruebot committed Jan 16, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    9dab703

  3. Add won't implement language to binary analysis.

    @ruebot
    ruebot committed Jan 16, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    5089a03

  4. Add DF results Python section

    @ruebot
    ruebot committed Jan 16, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    a7bccf8

  5. Address archivesunleashed/aut#372 - DRAFT

    @ruebot
    ruebot committed Jan 16, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    88789a9
Commits on Jan 15, 2020

  1. Add discardMimeTypesTika, and tweak other MIME type examples. (#38)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 15, 2020 
    - Resolves #32
- Addresses archivesunleashed/aut#223
- Addresses archivesunleashed/aut#401
- Addresses archivesunleashed/aut#396
- Addresses archivesunleashed/aut#399
    45de55b
Commits on Jan 13, 2020

  1. More DF filter updates. (#37)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 13, 2020 
    - Addresses #32
- Addresses archivesunleashed/aut#223
- Addresses archivesunleashed/aut#401
- Addresses archivesunleashed/aut#396
- Addresses archivesunleashed/aut#399
    8ed292b

  2. Scala DF filter updates for a few aut PRs. (#36)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 13, 2020 
    - Addresses #32
- Addresses archivesunleashed/aut#223
- Addresses archivesunleashed/aut#401
- Addresses archivesunleashed/aut#396
- Addresses archivesunleashed/aut#399
    a59904a
Commits on Jan 8, 2020

  1. Add extract-simple-site-link-structure DF example. (#35)

    @ruebot
    ruebot committed Jan 8, 2020 
    * Add extract-simple-site-link-structure DF example.
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    4186cd9
Commits on Dec 18, 2019

  1. Update filters documentation for https://github.com/archivesunleashed… (

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Dec 18, 2019 
    #33)

* Update filters documentation for archivesunleashed/aut#391

- Add ToC
- Add Scala RDD, Scala DF, and Python DF sections

* review
    19d8607
Commits on Dec 17, 2019

  1. Change extractValidPagesDF -> webpages (#31)

    @ianmilligan1 @ruebot
    ianmilligan1 authored and ruebot committed Dec 17, 2019
    c2e6956
Commits on Dec 5, 2019

  1. Updates for archivesunleashed/aut#387 (#30)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Dec 5, 2019 
    * Updates for archivesunleashed/aut#387

* Missed some in #24
    955f7bd
Commits on Nov 26, 2019

  1. Add "Find Images Shared Between Domains" section. (#27)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 26, 2019 
    * Add "Find Images Shared Between Domains" section.

- Resolves archivesunleashed/aut#237

* review
    e774c6c
Commits on Nov 22, 2019

  1. Add example for Scala DF version of "Extract Most Frequent Images MD5… (

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 22, 2019 
    #28)

* Add example for Scala DF version of "Extract Most Frequent Images MD5 Hash".

- See archivesunleashed/aut#382

* rename
    9cffe13
Commits on Nov 21, 2019

  1. Rename pages() to webpages(). (#25)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 21, 2019 
    - See: archivesunleashed/aut#384
    e162384
Commits on Nov 19, 2019

  1. Documentation update for archivesunleashed/aut#381 (#24)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 19, 2019
    fdbd1be

  2. Move cookbook to standard derivatives guide (#21)

    @ruebot
    ruebot committed Nov 19, 2019 
    - Update all current cookbook examples to follow documentation style
guide
- Add Parquet, CSV, S3, and Python DF examples
- Update index
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    d8e71e1
Commits on Nov 12, 2019

  1. Add example for archivesunleashed/aut#377 / h… (#20)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 12, 2019 
    * Add example for archivesunleashed/aut#377 / archivesunleashed/aut#238

* review
    930f8c8
Commits on Nov 7, 2019

  1. Updates for changing RemoveHttpHeader to RemoveHTTPHeader. (#19)

    @SinghGursimran @ruebot
    SinghGursimran authored and ruebot committed Nov 7, 2019 
    - Add ScalaDF example for: Extract Plain Text Without HTTP Headers
- See also:
   - archivesunleashed/aut#368
   - archivesunleashed/aut#374
   - archivesunleashed/aut#370
    4f73504
Commits on Nov 6, 2019

  1. Better definition of what AUT is. (#18)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 6, 2019
    f40abd4
Commits on Nov 5, 2019

  1. Update NER documentation for archivesunleashed/aut#361 (#17)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Nov 5, 2019
    10c2be5
Commits on Oct 28, 2019

  1. Incorporate PySpark setup into overall documentation. (#16)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Oct 28, 2019 
    * Incorporate PySpark setup into overall documentation.

- Removes standalone PySpark documentation.
- Incorporates PySpark setup into getting started documentation.
- Incorporates PySpark examples into overall documentation.
- Breaks out scaling documentation to it's own documentation.
- Removes cruft.
- Renames files so they're all lowercase now.
- Updates README ToC
    4cc429a
Commits on Oct 26, 2019

  1. Update to ToC to be more consistent after #11. (#13)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Oct 26, 2019
    48c1331

  2. Add binary analysis (#11)

    @ruebot
    ruebot committed Oct 26, 2019 
    - Add documentation for binary analysis and extraction in Scala DF and
Python DF
- Add Scala DF and Python DF version of extractImageLinks
- Update main ToC
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    978d0ab
Commits on Oct 25, 2019

  1. Fix link-analysis ToC links. (#12)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Oct 25, 2019
    53e0f63
Commits on Oct 23, 2019

  1. Fixed Table of Content on Current Doc README (#10)

    @ianmilligan1
    ianmilligan1 committed Oct 23, 2019 
    * Fixing table of content links

* Adding more relative links

* Adding image (fixing existing markdown)

* Removing image on seeing it rendered
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    4a8955e
Commits on Oct 21, 2019

  1. Adds ToC subheadings; changes collection analysis headings (#9)

    @ianmilligan1
    ianmilligan1 committed Oct 21, 2019
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    5173748

  2. Changed text-analysis.md to use consistent phrasing (#8)

    @lintool
    lintool committed Oct 21, 2019 
    Heading changed to verb phrases so it fits with "How do I..."
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    5416f8f

  3. Delete unneeded files (#7)

    @lintool
    lintool committed Oct 21, 2019
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    5112ffa

  4. Refactoring Documentation for Explanations and Consistent Structure (#5)

    @ianmilligan1 @ruebot
    ianmilligan1 authored and ruebot committed Oct 21, 2019 
    - Flesh out root README with a site-wide table of contents;
- Provide some basic introduction;
- Provide some context on RDD/DF; and
- Break the large "getting started and overview" document into at least two parts.
    95a7559
Older
You can’t perform that action at this time.