push old code to github

cloudyr · Nov 4, 2018 · 65c7a7ec9a8c88106f2c45ad2684e9f69dc986e4 · 65c7a7e
commit 65c7a7ec9a8c88106f2c45ad2684e9f69dc986e4
diff --git a/.Rbuildignore b/.Rbuildignore
@@ -0,0 +1,13 @@
+.travis.yml
+CONTRIBUTING.md
+README.Rmd
+README.html
+Makefile
+drat.sh
+knitreadme.sh
+^revdep$
+^man-roxygen$
+^.*\.Rproj$
+^\.Rproj\.user$
+^\.github.?
+^revdep.?
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -0,0 +1,29 @@
+Contributions to **aws.athena** are welcome from anyone and are best sent as pull requests on [the GitHub repository](https://github.com/cloudyr/aws.athena/). This page provides some instructions to potential contributors about how to add to the package.
+
+ 1. Contributions can be submitted as [a pull request](https://help.github.com/articles/creating-a-pull-request/) on GitHub by forking or cloning the [repo](https://github.com/cloudyr/aws.athena/), making changes and submitting the pull request.
+ 
+ 2. The cloudyr project follows [a consistent style guide](http://cloudyr.github.io/styleguide/index.html) across all of its packages. Please refer to this when editing package code.
+ 
+ 3. Pull requests should involve only one commit per substantive change. This means if you change multiple files (e.g., code and documentation), these changes should be committed together. If you don't know how to do this (e.g., you are making changes in the GitHub web interface) just submit anyway and the maintainer will clean things up.
+ 
+ 4. All contributions must be submitted consistent with the package license ([GPL-2](http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html)).
+ 
+ 5. Non-trivial contributions need to be noted in the `Authors@R` field in the [DESCRIPTION](https://github.com/cloudyr/aws.athena/blob/master/DESCRIPTION). Just follow the format of the existing entries to add your name (and, optionally, email address). Substantial contributions should also be noted in [`inst/CITATION`](https://github.com/cloudyr/aws.athena/blob/master/inst/CITATION).
+ 
+ 6. The cloudyr project use royxgen code and documentation markup, so changes should be made to roxygen comments in the source code `.R` files. If changes are made, roxygen needs to be run. The easiest way to do this is a command line call to: `Rscript -e devtools::document()`. Please resolve any roxygen errors before submitting a pull request.
+ 
+ 7. Please run `R CMD BUILD aws.athena` and `R CMD CHECK aws.athena_VERSION.tar.gz` before submitting the pull request to check for any errors.
+ 
+Some specific types of changes that you might make are:
+
+ 1. Bug fixes. Great!
+ 
+ 2. Documentation-only changes (e.g., to Rd files, README, vignettes). This is great! All contributions are welcome.
+ 
+ 3. New functionality. This is fine, but should be discussed on [the GitHub issues page](https://github.com/cloudyr/aws.athena/issues) before submitting a pull request.
+ 
+ 3. Changes requiring a new package dependency should also be discussed on [the GitHub issues page](https://github.com/cloudyr/aws.athena/issues) before submitting a pull request.
+ 
+ 4. Message translations. These are very appreciated! The format is a pain, but if you're doing this I'm assuming you're already familiar with it.
+
+Any questions you have can be opened as GitHub issues or directed to thosjleeper (at) gmail.com.
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md
@@ -0,0 +1,25 @@
+Please specify whether your issue is about:
+
+ - [ ] a possible bug
+ - [ ] a question about package functionality
+ - [ ] a suggested code or documentation change, improvement to the code, or feature request
+
+If you are reporting (1) a bug or (2) a question about code, please supply:
+
+ - [a fully reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) using a publicly available dataset (or provide your data)
+ - if an error is occurring, include the output of `traceback()` run immediately after the error occurs
+ - the output of `sessionInfo()`
+
+Put your code here:
+
+```R
+## load package
+library("aws.athena")
+
+## code goes here
+
+
+## session info for your system
+sessionInfo()
+```
+
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,9 @@
+Please ensure the following before submitting a PR:
+
+ - [ ] if suggesting code changes or improvements, [open an issue](https://github.com/cloudyr/aws.athena/issues/new) first
+ - [ ] for all but trivial changes (e.g., typo fixes), add your name to [DESCRIPTION](https://github.com/cloudyr/aws.athena/blob/master/DESCRIPTION)
+ - [ ] for all but trivial changes (e.g., typo fixes), document your change in [NEWS.md](https://github.com/cloudyr/aws.athena/blob/master/NEWS.md) with a parenthetical reference to the issue number being addressed
+ - [ ] if changing documentation, edit files in `/R` not `/man` and run `devtools::document()` to update documentation
+ - [ ] add code or new test files to [`/tests`](https://github.com/cloudyr/aws.athena/tree/master/tests/testthat) for any new functionality or bug fix
+ - [ ] make sure `R CMD check` runs without error before submitting the PR
+
diff --git a/.gitignore b/.gitignore
@@ -0,0 +1,17 @@
+.Renviron
+.Rproj.user
+.Rhistory
+.RData
+*.Rproj
+README.html
+/inst/doc/*.log
+/vignettes/*.log
+/vignettes/*.aux
+/vignettes/*.pdf
+/vignettes/*.sty
+/vignettes/*.tex
+/vignettes/*.out
+/vignettes/*.blg
+/vignettes/figure/
+/vignettes/*.bib.bak
+/revdep/
diff --git a/.travis.yml b/.travis.yml
@@ -0,0 +1,27 @@
+language: r
+sudo: false
+cache: packages
+matrix:
+  include:
+  - os: linux
+    dist: trusty
+    sudo: required
+    env: DRAT_DEPLOY=true
+  - os: osx
+    osx_image: xcode9.2
+  - os: osx
+    osx_image: xcode7.3
+    r_build_args: "--no-build-vignettes"
+    r_check_args: "--ignore-vignettes --no-examples"
+r_packages:
+- knitr
+- covr
+- drat
+after_success:
+- R -q -e 'library("covr");codecov()'
+- test $TRAVIS_PULL_REQUEST == "false" && test $TRAVIS_BRANCH == "master" && test
+  $DRAT_DEPLOY == "true" && bash drat.sh
+- R -q -e "travisci::restart_last_build('cloudyr/awspack')"
+env:
+  global:
+    secure: hcNZH6aCEbgJ2ziIRICqvm1w5Hs90uxJcxAdwyh4dsWm/g/ZUr1rAdHf5P/eWpF1pWZJT+UQVJqMCWiAKNgpajn2qDtWAdd7PfCLeQ+xcqjF9nNzRv30uI5C0676l8S/CI90febh7/y4GF+vCcVep998NcJaXAas0Mujd+CMZ80TWNSpDsoEs3sxm0IwuCFmN2tPr2aTx+PBx6lVSJ8fAmzQ9J3L4Lh6yQx9hhkIcsaINOJb1TUEGpOMrsLmJp1TPw1WYNNf7Pd4OyADeIBhCXDZ8uQMcj8uI7aC8yDHUxOC0NXLbmp7/1Ht5F/797wI8DB+sULkQdapUkPrRtxtNe3GwTAdOQ/tGVf9qeyyaMqd8TIyOB2uSZIgbUqOhJl+Y4g+C41PmA509hmzHKdVTRTGGcSloS/vTQ2r5xhW5fkfxRQL6VKOpj9Tl4EsdbcNyRS6Zfcaubh5yfDs10W0CHX1YMpMXUGRWPqh4cmruO6jcRX2PcAOixTjxZ04q4s/OGXuyNSfF6IguBKWzCbDYVVJ3gXkocePGxHJQZz4iD/w6dmgyabqUfSkRKEAm6mZqg1rsvptrGtFsDORsQcK+bizKK7KClbJgSisbxO+CpZXi9gMzuoNgLxevJlCY6euara5TinAFVXyZSoyrwryAXEmnmUUAnfU9NEqoF5BO4o=
diff --git a/DESCRIPTION b/DESCRIPTION
@@ -0,0 +1,19 @@
+Package: aws.athena
+Type: Package
+Title: 'AWS Athena' Client Package
+Version: 0.1.1
+Date: 2018-08-17
+Authors@R: c(person("Thomas J.", "Leeper", role = c("aut", "cre"),
+                    email = "thosjleeper@gmail.com",
+                    comment = c(ORCID = "0000-0003-4097-6326")))
+Description: Client package for 'AWS Athena' <https://aws.amazon.com/athena/>, an interactive query service that makes it easy to analyze data in 'Amazon S3' using standard 'SQL'.
+License: GPL (>= 2)
+URL: https://github.com/cloudyr/aws.athena
+BugReports: https://github.com/cloudyr/aws.athena/issues
+Imports:
+    httr,
+    aws.signature (>= 0.3.7),
+    aws.s3
+Suggests:
+    testthat
+RoxygenNote: 5.0.1
diff --git a/Makefile b/Makefile
@@ -0,0 +1,25 @@
+pkg = $(shell basename $(CURDIR))
+
+all: build
+
+NAMESPACE: R/*
+	Rscript -e "devtools::document()"
+
+README.md: README.Rmd
+	Rscript -e "knitr::knit('README.Rmd')"
+
+README.html: README.md
+	pandoc -o README.html README.md
+
+../$(pkg)*.tar.gz: DESCRIPTION NAMESPACE README.md
+	cd ../ && R CMD build $(pkg)
+
+build: ../$(pkg)*.tar.gz
+
+check: ../$(pkg)*.tar.gz
+	cd ../ && R CMD check $(pkg)*.tar.gz
+	rm ../$(pkg)*.tar.gz
+
+install: ../$(pkg)*.tar.gz
+	cd ../ && R CMD INSTALL $(pkg)*.tar.gz
+	rm ../$(pkg)*.tar.gz
diff --git a/NEWS.md b/NEWS.md
@@ -0,0 +1,3 @@
+# aws.athena 0.1.1
+
+* Initial release.
diff --git a/R/athenaHTTP.R b/R/athenaHTTP.R
@@ -0,0 +1,92 @@
+#' @title API Requests
+#' @description Amazon Athena HTTP Requests
+#' @param action A character string specifying an API endpoint.
+#' @param query An optional named list containing query string parameters and their character values.
+#' @param headers A list of headers to pass to the HTTP request.
+#' @param body A request body
+#' @param verbose A logical indicating whether to be verbose. Default is given by \code{options("verbose")}.
+#' @param region A character string specifying an AWS region. See \code{\link[aws.signature]{locate_credentials}}.
+#' @param key A character string specifying an AWS Access Key. See \code{\link[aws.signature]{locate_credentials}}.
+#' @param secret A character string specifying an AWS Secret Key. See \code{\link[aws.signature]{locate_credentials}}.
+#' @param session_token Optionally, a character string specifying an AWS temporary Session Token to use in signing a request. See \code{\link[aws.signature]{locate_credentials}}.
+#' @param ... Additional arguments passed to \code{\link[httr]{GET}}.
+#' @return If successful, a named list. Otherwise, a data structure of class \dQuote{aws-error} containing any error message(s) from AWS and information about the request attempt.
+#' @details This function constructs and signs an Athena API request and returns the results thereof, or relevant debugging information in the case of error.
+#' @author Thomas J. Leeper
+#' @import httr
+#' @importFrom jsonlite fromJSON toJSON
+#' @importFrom aws.signature signature_v4_auth
+#' @export
+athenaHTTP <- 
+function(
+  action,
+  headers = list(),
+  query = list(),
+  body = NULL,
+  verbose = getOption("verbose", FALSE),
+  region = Sys.getenv("AWS_DEFAULT_REGION", "us-east-1"), 
+  key = NULL, 
+  secret = NULL, 
+  session_token = NULL,
+  ...
+) {
+    # locate and validate credentials
+    credentials <- aws.signature::locate_credentials(key = key, secret = secret, session_token = session_token, region = region, verbose = verbose)
+    key <- credentials[["key"]]
+    secret <- credentials[["secret"]]
+    session_token <- credentials[["session_token"]]
+    region <- credentials[["region"]]
+    
+    # generate request signature
+    d_timestamp <- format(Sys.time(), "%Y%m%dT%H%M%SZ", tz = "UTC")
+    url <- paste0("https://athena.",region,".amazonaws.com")
+    Sig <- aws.signature::signature_v4_auth(
+           datetime = d_timestamp,
+           region = region,
+           service = "athena",
+           verb = "POST",
+           action = "/",
+           query_args = query,
+           canonical_headers = list(host = paste0("athena.",region,".amazonaws.com"),
+                                    "x-amz-date" = d_timestamp,
+                                    "X-Amz-Target" = paste0("AmazonAthena.", action),
+                                    "Content-Type" = "application/x-amz-json-1.1"),
+           request_body = if (is.null(body)) "" else jsonlite::toJSON(body, auto_unbox = TRUE),
+           key = key, 
+           secret = secret,
+           session_token = session_token,
+           verbose = verbose)
+    # setup request headers
+    headers[["X-Amz-Target"]] <- paste0("AmazonAthena.", action)
+    headers[["Content-Type"]] <- "application/x-amz-json-1.1"
+    headers[["x-amz-date"]] <- d_timestamp
+    headers[["x-amz-content-sha256"]] <- Sig$BodyHash
+    headers[["Authorization"]] <- Sig[["SignatureHeader"]]
+    if (!is.null(session_token) && session_token != "") {
+        headers[["x-amz-security-token"]] <- session_token
+    }
+    H <- do.call(httr::add_headers, headers)
+    
+    # execute request
+    if (length(query)) {
+        r <- httr::POST(url, H, query = query, body = body, encode = "json", ...)
+    } else {
+        r <- httr::POST(url, H, body = body, encode = "json", ...)
+    }
+    
+    if (httr::http_error(r)) {
+        x <- jsonlite::fromJSON(httr::content(r, "text", encoding = "UTF-8"))
+        httr::warn_for_status(r)
+        h <- httr::headers(r)
+        out <- structure(x, headers = h, class = "aws_error")
+        attr(out, "request_canonical") <- Sig$CanonicalRequest
+        attr(out, "request_string_to_sign") <- Sig$StringToSign
+        attr(out, "request_signature") <- Sig$SignatureHeader
+    } else {
+        out <- try(jsonlite::fromJSON(httr::content(r, "text", encoding = "UTF-8")), silent = TRUE)
+        if (inherits(out, "try-error")) {
+            out <- structure(httr::content(r, "text", encoding = "UTF-8"), "unknown")
+        }
+    }
+    return(out)
+}
diff --git a/R/athena_executions.R b/R/athena_executions.R
@@ -0,0 +1,95 @@
+#' @rdname executions
+#' @title AWS Athena Executions
+#' @description Get, Start, and Stop Athena Query Executions
+#' @param id A character string containing an Amazon Athena Query Executionn ID. A vector of IDs can also be specified to request multiple query executions.
+#' @param n An integer specifying the maximum number of results to return.
+#' @param token A character string specifying a continuation token, for pagination.
+#' @param \dots Additional arguments passed to \code{\link{athenaHTTP}}
+#' @references
+#'  \href{https://docs.aws.amazon.com/athena/latest/APIReference/API_ListQueryExecutions.html}{API Reference: ListQueryExecutions}
+#'  \href{https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryExecution.html}{API Reference: GetQueryExecution}
+#'  \href{https://docs.aws.amazon.com/athena/latest/APIReference/API_BatchGetQueryExecution.html}{API Reference: BatchGetQueryExecution}
+#' @examples
+#' \dontrun{
+#' require("aws.s3")
+#' b <- aws.s3::put_bucket("aws-athena-r-demo")
+#' 
+#' # create an Athena database
+#' id <- start_athena_execution(
+#'   query = "create database DEMO",
+#'   output = "s3://aws-athena-r-demo"
+#' )
+#' 
+#' # create a table in the database
+#' 
+#' }
+#' @export
+list_athena_executions <-
+function(
+  n = 50,
+  token = NULL,
+  ...
+) {
+    bod <- list()
+    bod$MaxResults <- n
+    if (!is.null(token)) {
+        bod$NextToken <- token
+    }
+    res <- athenaHTTP("ListQueryExecutions", body = bod, ...)
+    res$NamedQueryIds
+}
+
+#' @rdname executions
+#' @export
+get_athena_execution <-
+function(
+  id,
+  ...
+) {
+    bod <- list()
+    if (length(id) > 1L) {
+        bod$QueryExecutionIds <- id
+        res <- athenaHTTP("BatchGetQueryExecution", body = bod, ...)
+    } else {
+        bod$QueryExecutionId <- id
+        res <- athenaHTTP("GetQueryExecution", body = bod, ...)
+    }
+    res
+}
+
+#' @rdname executions
+#' @importFrom aws.s3 get_bucketname
+#' @export
+start_athena_execution <-
+function(
+  database = NULL,
+  query,
+  output,
+  encryption = NULL,
+  ...
+) {
+    bod <- list()
+    if (!is.null(database)) {
+        bod$QueryExecutionContext <- list(Database = database)
+    }
+    bod$QueryString <- query
+    bod$ResultConfiguration <- list(OutputLocation = output)
+    if (!is.null(encryption)) {
+        bod$ResultConfiguration$EncryptionConfiguration <- encryption
+    }
+    res <- athenaHTTP("StartQueryExecution", body = bod, ...)
+    res
+}
+
+#' @rdname executions
+#' @export
+stop_athena_execution <-
+function(
+  id,
+  ...
+) {
+    bod <- list()
+    bod$QueryExecutionId <- id
+    res <- athenaHTTP("StopQueryExecution", body = bod, ...)
+    res
+}