Skip to content

/ aut

Permalink
Tree: 99a4e8b283
Commits on Mar 26, 2020

  1. Alphabetical sort udfs

    @ruebot
    ruebot committed Mar 26, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    99a4e8b
Commits on Mar 25, 2020

  1. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Mar 25, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    0da3d37
Commits on Mar 23, 2020

  1. Tweak hasDate to handle Seq. (#430)

    @ruebot
    ruebot committed Mar 23, 2020
    Loading status checks…
     
    Tweak hasDate to handle Seq.
- Addresses #425
- Add test for hasDate
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    92b5f2d
Commits on Mar 19, 2020

  1. Add setup; might not need it in the end :shrug:

    @ruebot
    ruebot committed Mar 19, 2020
    Loading status checks…
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    3eb9393

  2. Merge branch 'master' of github.com:archivesunleashed/aut into issue-409

    @ruebot
    ruebot committed Mar 19, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    3707731
Commits on Mar 18, 2020

  1. Restyle keep/discard filter UDFs in the context of DataFrames (#429)

    @ruebot
    ruebot committed Mar 18, 2020
    Loading status checks…
     
    Co-authored-by: g285sing <g285sing@student.cs.uwaterloo.ca> (@SinghGursimran)

- Resolves #425
- Replace all keep/discard DF udfs with `hasXYZ()`
- Update tests
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    e1d908b
Commits on Feb 20, 2020

  1. Merge branch 'issue-409' of github.com:archivesunleashed/aut into iss…

    @ruebot
    ruebot committed Feb 20, 2020 
    …ue-409
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    0b032d7

  2. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Feb 20, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    fd94139

  3. Update Spark and Hadoop versions. (#426)

    @ruebot
    ruebot committed Feb 20, 2020
    Loading status checks…
     
    - Update Spark to 2.4.5
- Update Hadoop to 2.7.4 (for RADOS/S3 support)
- Tweak README
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    9ce73a8
Commits on Feb 18, 2020

  1. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Feb 18, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    a55b4c6
Commits on Feb 12, 2020

  1. Add logic so UDFs that filter on url should also filter on src (#424).

    @SinghGursimran @ruebot
    SinghGursimran and ruebot committed Feb 12, 2020
    Loading status checks…
     
    - Resolves #418 
- Update tests

Co-authored-by: Nick Ruest <ruestn@gmail.com>
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    ebb5298
Commits on Feb 11, 2020

  1. [skip travis] Add pre-print link to README. (#423)

    @ruebot
    ruebot committed Feb 11, 2020 
    * [skip travis] Add pre-print link to README.
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    9474a09
Commits on Feb 10, 2020

  1. Add img alt text to imagegraph(); resolves #420. (#422)

    @ruebot
    ruebot committed Feb 10, 2020
    Loading status checks…
     
    - Update ExtractImageLinksRDD to grab alt text
- Add alt_text column to imagegraph
- Update tests
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    8f1a9f1

  2. Rename imageLinks to imagegraph; resolves #419 (#421)

    @ruebot
    ruebot committed Feb 10, 2020
    Loading status checks…
     
    * Rename imageLinks to imagegraph; resolves #419
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    87c9734

  3. Python work check-in

    @ruebot
    ruebot committed Feb 10, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    f237c35
Commits on Feb 6, 2020

  1. Need --repositories flag with --packages. (#417)

    @ruebot
    ruebot committed Feb 6, 2020
    Loading status checks…
     
    - Fully resolves this issue archivesunleashed/docker-aut#19
- archivesunleashed/docker-aut@37ce4e2
- archivesunleashed/docker-aut@082907a
- archivesunleashed/docker-aut@baee431
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    ae2f097

  2. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Feb 6, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    054c6b7

  3. [maven-release-plugin] prepare for next development iteration

    @ruebot
    ruebot committed Feb 5, 2020
    Loading status checks…
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    e297369
Commits on Feb 5, 2020

  1. [maven-release-plugin] prepare release aut-0.50.0

    @ruebot
    ruebot committed Feb 5, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    4d1dcc9
Commits on Jan 23, 2020

  1. Start adding filters; keep_valid_pages.

    @ruebot
    ruebot committed Jan 23, 2020
    Loading status checks…
     
    - TODO, make it object oriented
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    5255c10

  2. clean-up

    @ruebot
    ruebot committed Jan 23, 2020
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    5803b4d
Commits on Jan 22, 2020

  1. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Jan 22, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    3b16fd4
Commits on Jan 21, 2020

  1. Clean up test descriptions, addresses #372. (#416)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 21, 2020
    Loading status checks…
     
    - Clean up test descriptions
- Rename typo filename
    ffef735

  2. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Jan 21, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    e8bf94a

  3. Add ExtractImageDetailsDF. (#415)

    @SinghGursimran @ruebot
    SinghGursimran authored and ruebot committed Jan 21, 2020
    Loading status checks…
     
    - Add test
- Addresses #223
    71b459c

  4. - Remove order udfs alphabetically

    @ruebot
    ruebot committed Jan 21, 2020
    Loading status checks…
     
    - Get detect_language setup
- ComputeSHA1 and MD5 need some work?
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    825a933

  5. Merge branch 'issue-409' of github.com:archivesunleashed/aut into iss…

    @ruebot
    ruebot committed Jan 21, 2020 
    …ue-409
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    7060f5f

  6. Setup external lib packaging for Python!!

    @ruebot
    ruebot committed Jan 21, 2020 
    Add remove_html udf
Rename remove_http_header to remove_http_headers
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    0a094cc
Commits on Jan 19, 2020

  1. Matchbox-pyhton1

    g285sing
    g285sing committed Jan 19, 2020
    Loading status checks…
    2
    5830ab9

  2. computeMD5

    g285sing
    g285sing committed Jan 19, 2020
    Loading status checks…
    e310611
Commits on Jan 18, 2020

  1. Merge branch 'master' into issue-409

    @ruebot
    ruebot committed Jan 18, 2020
    Loading status checks…
    Verified
    This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
    GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
    d5a4f06

  2. Add crawl_date to binary DataFrames and imageLinks. (#414)

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 18, 2020
    Loading status checks…
     
    - Resolves #413
- Update tests where necessary
    9e357cc

  3. rename links to webgraph

    @ruebot
    ruebot committed Jan 18, 2020
    Loading status checks…
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    af2bc5a

  4. Add some PySpark udfs

    @ruebot
    ruebot committed Jan 18, 2020
    Loading status checks…
     
    - Add remove_http_header, remove_prefix_www
- Rename extract_domain_func to extract_domain
- Formatting updates
- Addresses #409
    Verified
    This commit was signed with a verified signature.
    ruebot Nick Ruest
    GPG key ID: 417FAF1A0E1080CD Learn about signing commits
    0e8489b
Commits on Jan 17, 2020

  1. Various DataFrame implementation updates for documentation clean-up; …

    @ruebot @ianmilligan1
    ruebot authored and ianmilligan1 committed Jan 17, 2020
    Loading status checks…
     
    …Addresses #372.

- .all() column HttpStatus to http_status_code
- Adds archive_filename to .all()
- Significant README updates for setup
- See also: archivesunleashed/aut-docs#39
    9277e68
Older
You can’t perform that action at this time.