archivesunleashed/aut

…Record and WARCRecord variables. Also removes unused headerResponseFormat variable. (#316)

- Exclude slf4j binding logback-classic
(mojohaus/cobertura-maven-plugin#6 (comment))

- CVE-2018-8024 https://nvd.nist.gov/vuln/detail/CVE-2018-8024
- CVE-2018-1334 https://nvd.nist.gov/vuln/detail/CVE-2018-1334
- CVE-2018-17190 https://nvd.nist.gov/vuln/detail/CVE-2018-17190
- CVE-2018-11770 https://nvd.nist.gov/vuln/detail/CVE-2018-11770

* Log opening and closing of archive files as per #156
* Remove redundant log message. Spark already logs the file that is to be read when an executor computes an RDD.

* Remove deprecated ExtractGraph.scala file.
* Remove deprecated ExtractGraphTest.scala file.

…& #164 (#292)

* Resolves #198
* Resolves #164
* Add getHttpStatus to ArchiveRecord class & trait
  - add .getHttpStatus to potential outputs
  - add tests for .getHttpStatus calls
  - improve ArchiveRecord testing overall.
* Add .getArchiveFile feature to ArchiveRecordImpl.
  - add getArchiveFile to trait
  - add getArchiveFile for ArchiveRecordImpl
  - add tests for getArchiveFile.
* Other code style fixes.
* Include updates to tests.

…zipWithUniqueIds() (#289)

* Resolves #243 
* Create GEXF with proper ids instead of hash to avoid collisions.
* Add WriteGEXF files.
* Add WriteGraph file and test.
* Add test for Graphml output.
* Add xml escaping for edges.
* Add test case for non-escaped edges.
* Add additional tests to cover for more potential cases of graphml and gexf files.
* Coverage for null cases in urls.

- CVE-2018-1338
- CVE-2018-11762
- CVE-2018-11761
- CVE-2016-6809
- CVE-2018-1339
- CVE-2018-11796
- CVE-2016-4434
- CVE-2018-1335

* Fix backslash in URL
* Add backslash test in ExtractDomainTest

- Follow on to
72cb5e2
- https://nvd.nist.gov/vuln/detail/CVE-2018-7489

- CVE-2017-752
- See also: https://nvd.nist.gov/vuln/detail/CVE-2017-7525

…les. (#272)

* Fix exception when processing corrupted ARC files
* Filter out non-empty archive files in loadArchives()
* Fix archive files path pattern
* Resolves #246
* Resolves #271 
* Resolves #258

* ExtractBoilerpipeText now removes headers.

)

- Adds:
  - retweet_count
  - favorite_count
  - in_reply_to_status_id_str
  - in_reply_to_user_id_str
  - in_reply_to_screen_name
  - source
  - user.protected
  - user.profile_image_url
  - user.description
  - user.location
  - user.name
  - user.url
  - user.time_zone
- Updates some doc comments
- Updates tests

…). (#249)

* Fully resolves #196 
* Resolves #212

* Deal with wildcard import lint issues.
* Fix some magic numbers & duplicate string runs.
* Lint fixes, mostly explicit import warnings.
* All other scalastyle issues require refactoring.

…ssue 203 (#245)

* pom.xml change for GraphX
* Changes for GraphXSLS
* Changes for SLS graph
* Changes for GraphX
* Changes for converting WARC RDD to GraphX object
* Rename extractor to ExtractGraphX
* Various lint fixes (usually Magic Numbers)
* Remove illegal imports from scala style (we use wildcard imports a lot)
* Add WriteGraphXMLTest.

* Make the TravisCI build less verbose since we're hitting the 4MB log limit.
* Pin site.plugin and project-info-reports.plugin so mvn site builds.
  - See:
    - https://stackoverflow.com/questions/51091539/maven-site-plugins-3-3-java-lang-classnotfoundexception-org-apache-maven-doxia
    - https://travis-ci.org/archivesunleashed/aut/jobs/408259462#L3201-L3202