openadds talks to Openaddresses.io. a run down of its things: Install devtools::install_github("sckott/openadds") library("openadds") List datasets Scrapes links to datasets from the openaddresses site dat <- oa_list() dat[2:6] #> [1] "https://data.openaddresses.io.s3.amazonaws.com/20150511/au-tas-launceston.csv" #> [2] "https://s3.amazonaws.com/data.openaddresses.io/20141127/au-victoria.zip" #> [3] "https://data.openaddresses.io.s3.amazonaws.com/20150511/be-flanders.zip" #> [4] "https://data.openaddresses.io.s3.amazonaws.com/20150417/ca-ab-calgary.zip" #> [5] "https://data.openaddresses.io.s3.amazonaws.com/20150511/ca-ab-grande_prairie.zip" Search for datasets Uses oa_list() internally, then searches through columns requested. oa_search(country = "us", state = "ca") #> Source: local data frame [68 x 5] #> #> country state city ext #> 1 us ca san_mateo_county .zip #> 2 us ca alameda_county .zip #> 3 us ca alameda_county .zip #> 4 us ca amador .zip #> 5 us ca amador .zip #> 6 us ca bakersfield .zip #> 7 us ca bakersfield .zip #> 8 us ca berkeley .zip #> 9 us ca berkeley .zip #> 10 us ca butte_county .zip #> .. ... ... ... ... #> Variables not shown: url (chr) Get data Passing in a URL ...

geojsonio converts geographic data to GeoJSON and TopoJSON formats - though the focus is mostly on GeoJSON For those not familiar GeoJSON it is a format for encoding a variety of geographic data structures. GeoJSON supports the following geometry types: Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, and GeometryCollection. These geometry types are also found in well known text (WKT), and have equivalents in R’s spatial classes. Read the spec for more detailed information. ...

Inspired by httpie, a Python command line client as a sort of drop in replacement for curl, I am playing around with something similar-ish in R, at least in spirit. I started a little R pkg called httsnap with the following ideas: The web is increasingly a JSON world, so set content-type and accept headers to applications/json by default The workflow follows logically, or at least should, from, hey, I got this url, to i need to add some options, to execute request Whenever possible, transform output to data.frame’s - facilitating downstream manipulation via dplyr, etc. Do GET requests by default. Specify a different type if you don’t want GET. Some functionality does GET by default, though in some cases you need to specify GET You can use non-standard evaluation to easily pass in query parameters without worrying about &’s, URL escaping, etc. (see Query()) Same for body params (see Body()) Install Install and load httsnap ...

With the help of user input, I’ve tweaked solr just a bit to make things faster using default setings. I imagine the main interface for people using the solr R client is via solr_search(), which used to have wt=json by default. Changing this to wt=csv gives better performance. And it sorta makes sense to use csv, as the point of using an R client is probably do get data eventually into a data.frame, so it makes sense to go csv format (Already in tabular format) if it’s faster too. ...

It would be nice to easily push each row or column of a data.frame into CouchDB instead of having to prepare them yourself into JSON, then push in to couch. I recently added ability to push data.frame’s into couch using the normal PUT /{db} method, and added support for the couch bulk API. Install install.packages("devtools") devtools::install_github("sckott/sofa") library("sofa") PUT /db You can write directly from a data.frame, either by rows or columns. First, rows: ...

CSL (Citation Style Language) is used quite widely now to specify citations in a standard fashion. csl is an R client for exploring CSL styles, and is inspired by the Ruby gem csl. For example, csl is given back in the PLOS Lagotto article level metric API (follow https://alm.plos.org/api/v5/articles?ids=10.1371%252Fjournal.pone.0025110&info=detail&source_id=crossref). Let me know if you have any feedback at the repo https://github.com/ropensci/csl Install install.packages("devtools") devtools::install_github("ropensci/csl") library("csl") Load CSL style from a URL You can load CSL styles from either a URL or a local file on your machine. Firt, from a URL. In this case from the Zotero style repository, for the American Journal or Political Science. ...

setup backup curl -XPUT 'http://localhost:9200/_snapshot/my_backup/' -d '{ "type": "fs", "settings": { "location": "/Users/sacmac/esbackups/my_backup", "compress": true } }' create backup http PUT "localhost:9200/_snapshot/my_backup/snapshot_2?wait_for_completion=true" get info on snapshot http "localhost:9200/_snapshot/my_backup/snapshot_2" restore curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_2/_restore" partial restore, including various options that can be used curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_2/_restore" -d '{ "indices": "index_1,index_2", "ignore_unavailable": "true", "include_global_state": false, "rename_pattern": "index_(.+)", "rename_replacement": "restored_index_$1" }'

Recently I spun up a box on a cloud hosting provider planning to make a tens of thousdands of queries to an Elasticsearch instance on the same box. I could have done this on my own machine, but didn’t want to take up compute resources. I installed R and Elasticsearch on the box, then went about doing my thang. A day later when things were still running, the hosting provider sent me a message that apparently my box had been serving up a DDoS attack. ...

I’ve been working on a little thing called httping - a small R package that started as a pkg to Ping urls and time requests. It’s a port of the Ruby gem httping. The httr package is in Depends in this package, so its functions can be called directly, without having to load httr explicitly yourself. In addition to timing requests, I’ve been tinkering with how to make http requests, with curl options accepting and returning the same object so they can be chained together, and then that object passed to a http verb like GET. Maybe this is a bad idea, but maybe not. ...

We’ve (ropensci) been working on an R client for interacting with Elasticsearch for a while now, first commit was November 2013. Elasticsearch is a document database built on the JVM. elastic interacts with the Elasticsearch HTTP API, and includes functions for setting connection details to Elasticsearch instances, loading bulk data, searching for documents with both HTTP query variables and JSON based body requests. In addition, elastic provides functions for interacting with APIs for indices, documents, nodes, clusters, an interface to the cat API, and more. ...