Http

You’ve experienced an HTTP redirect (or URL redirect, or URL forwarding) even if you haven’t noticed. We all use browsers (I assume, since you are reading this), either on a phone or laptop/desktop computer. Browsers don’t show all the HTTP requests going on in the background, some of which are redirects. Redirection is used for various reasons, including to prevent broken links when web pages are moved, for privacy protection, to allow multiple domains to refer to a single web page, and more. ...

I was listening to a Bike Shed podcast episode 189, “It’s Gonna Work, Definitely, No Problems Whatsoever”, and starting at 27:44 there was a conversation about exception handling. Specifically it was about exception handling in control flow when doing web API requests. This topic piqued my interest straight away as I do a lot of API stuff (making and wrapping). The part of the conversation that I want to address is their conclusion that exceptions in control flow are an anti-pattern. Seems this is a general pattern in programming languages, e.g., this SO thread. But on the contrary there are some languages in which exceptions in control flow are considered normal behavior; e.g., Python (this, this). ...

request is DSL for http requests for R, and is inspired by the CLI tool httpie. It’s built on httr. The following were driving principles for this package: The web is increasingly a JSON world, so we assume applications/json by default, but give back other types if not The workflow follows logically, or at least should, from, hey, I got this url, to i need to add some options, to execute request - and functions support piping so that you can execute functions in this order Whenever possible, we transform output to data.frame’s - facilitating downstream manipulation via dplyr, etc. We do GET requests by default. Specify a different type if you don’t want GET. Given GET by default, this client is optimized for consumption of data, rather than creating new things on servers You can use non-standard evaluation to easily pass in query parameters without worrying about &’s, URL escaping, etc. (see api_query()) Same for body params (see api_body()) The following is a brief demo of some of the package functionality: ...

Inspired by httpie, a Python command line client as a sort of drop in replacement for curl, I am playing around with something similar-ish in R, at least in spirit. I started a little R pkg called httsnap with the following ideas: The web is increasingly a JSON world, so set content-type and accept headers to applications/json by default The workflow follows logically, or at least should, from, hey, I got this url, to i need to add some options, to execute request Whenever possible, transform output to data.frame’s - facilitating downstream manipulation via dplyr, etc. Do GET requests by default. Specify a different type if you don’t want GET. Some functionality does GET by default, though in some cases you need to specify GET You can use non-standard evaluation to easily pass in query parameters without worrying about &’s, URL escaping, etc. (see Query()) Same for body params (see Body()) Install Install and load httsnap ...

I’ve been working on a little thing called httping - a small R package that started as a pkg to Ping urls and time requests. It’s a port of the Ruby gem httping. The httr package is in Depends in this package, so its functions can be called directly, without having to load httr explicitly yourself. In addition to timing requests, I’ve been tinkering with how to make http requests, with curl options accepting and returning the same object so they can be chained together, and then that object passed to a http verb like GET. Maybe this is a bad idea, but maybe not. ...

We’ve (ropensci) been working on an R client for interacting with Elasticsearch for a while now, first commit was November 2013. Elasticsearch is a document database built on the JVM. elastic interacts with the Elasticsearch HTTP API, and includes functions for setting connection details to Elasticsearch instances, loading bulk data, searching for documents with both HTTP query variables and JSON based body requests. In addition, elastic provides functions for interacting with APIs for indices, documents, nodes, clusters, an interface to the cat API, and more. ...

At rOpenSci we occasssionally hear from our users that they run into an error like: Error in function (type, msg, asError = TRUE) : easy handled already used in multi handle This error occurs in the httr package that we use to do http requests to sources of data on the web. It happens when e.g., you make a lot of requests to a resource, then it gets interrupted somehow - then you make another call, and you get the error above. Let’s try it with the an version of httr (v0.5): ...

Recently noticed a little Python library called httpcode that does a simple thing: gives information on http codes in the CLI. I thought this could maybe potentially be useful for R. So I made an R version. Installation devtools::install_github("sckott/httpcode") library("httpcode") Search by http code http_code(100) #> <Status code: 100> #> Message: Continue #> Explanation: Request received, please continue http_code(400) #> <Status code: 400> #> Message: Bad Request #> Explanation: Bad request syntax or unsupported method http_code(503) #> <Status code: 503> #> Message: Service Unavailable #> Explanation: The server cannot process the request due to a high load http_code(999) #> Error: No description found for code: 999 Fuzzy code search http_code('1xx') #> [[1]] #> <Status code: 100> #> Message: Continue #> Explanation: Request received, please continue #> #> [[2]] #> <Status code: 101> #> Message: Switching Protocols #> Explanation: Switching to new protocol; obey Upgrade header #> #> [[3]] #> <Status code: 102> #> Message: Processing #> Explanation: WebDAV; RFC 2518 http_code('3xx') #> [[1]] #> <Status code: 300> #> Message: Multiple Choices #> Explanation: Object has several resources -- see URI list #> #> [[2]] #> <Status code: 301> #> Message: Moved Permanently #> Explanation: Object moved permanently -- see URI list #> #> [[3]] #> <Status code: 302> #> Message: Found #> Explanation: Object moved temporarily -- see URI list #> #> [[4]] #> <Status code: 303> #> Message: See Other #> Explanation: Object moved -- see Method and URL list #> #> [[5]] #> <Status code: 304> #> Message: Not Modified #> Explanation: Document has not changed since given time #> #> [[6]] #> <Status code: 305> #> Message: Use Proxy #> Explanation: You must use proxy specified in Location to access this resource. #> #> [[7]] #> <Status code: 306> #> Message: Switch Proxy #> Explanation: Subsequent requests should use the specified proxy #> #> [[8]] #> <Status code: 307> #> Message: Temporary Redirect #> Explanation: Object moved temporarily -- see URI list #> #> [[9]] #> <Status code: 308> #> Message: Permanent Redirect #> Explanation: Object moved permanently http_code('30[12]') #> [[1]] #> <Status code: 301> #> Message: Moved Permanently #> Explanation: Object moved permanently -- see URI list #> #> [[2]] #> <Status code: 302> #> Message: Found #> Explanation: Object moved temporarily -- see URI list http_code('30[34]') #> [[1]] #> <Status code: 303> #> Message: See Other #> Explanation: Object moved -- see Method and URL list #> #> [[2]] #> <Status code: 304> #> Message: Not Modified #> Explanation: Document has not changed since given time Search by text message http_search("request") #> [[1]] #> <Status code: 100> #> Message: Continue #> Explanation: Request received, please continue #> #> [[2]] #> <Status code: 200> #> Message: OK #> Explanation: Request fulfilled, document follows #> #> [[3]] #> <Status code: 202> #> Message: Accepted #> Explanation: Request accepted, processing continues off-line #> #> [[4]] #> <Status code: 203> #> Message: Non-Authoritative Information #> Explanation: Request fulfilled from cache #> #> [[5]] #> <Status code: 204> #> Message: No Content #> Explanation: Request fulfilled, nothing follows #> #> [[6]] #> <Status code: 306> #> Message: Switch Proxy #> Explanation: Subsequent requests should use the specified proxy #> #> [[7]] #> <Status code: 400> #> Message: Bad Request #> Explanation: Bad request syntax or unsupported method #> #> [[8]] #> <Status code: 403> #> Message: Forbidden #> Explanation: Request forbidden -- authorization will not help #> #> [[9]] #> <Status code: 408> #> Message: Request Timeout #> Explanation: Request timed out; try again later. #> #> [[10]] #> <Status code: 409> #> Message: Conflict #> Explanation: Request conflict. #> #> [[11]] #> <Status code: 413> #> Message: Request Entity Too Large #> Explanation: Entity is too large. #> #> [[12]] #> <Status code: 414> #> Message: Request-URI Too Long #> Explanation: URI is too long. #> #> [[13]] #> <Status code: 416> #> Message: Requested Range Not Satisfiable #> Explanation: Cannot satisfy request range. #> #> [[14]] #> <Status code: 503> #> Message: Service Unavailable #> Explanation: The server cannot process the request due to a high load #> #> [[15]] #> <Status code: 505> #> Message: HTTP Version Not Supported #> Explanation: Cannot fulfill request. http_search("forbidden") #> [[1]] #> <Status code: 403> #> Message: Forbidden #> Explanation: Request forbidden -- authorization will not help http_search("too") #> [[1]] #> <Status code: 413> #> Message: Request Entity Too Large #> Explanation: Entity is too large. #> #> [[2]] #> <Status code: 414> #> Message: Request-URI Too Long #> Explanation: URI is too long. http_search("birds") #> Error: No status code found for search: : birds