download.file() from a multilingual website

Question

When I navigate to this page (http://hk.ishares.com/product_info/fund/overview/SEHK/2836.htm) and click the "NAV/Index History" link (under "Market Information") to download a CSV file, the contents of that file is in English. However if I try and do that same via R with the following commands, the contents are in Chinese:

download.file("http://hk.ishares.com/product_info/fund/excel_histoverview.htm?ticker=2836", destfile="2836.csv")
a <- read.csv("2836.csv", skip = 5)
> head(a)
            日期 指數收市水平..HKD. 單位資產淨值 總派息.每股...HKD.
1 2013年03月18日          3666.9390      15.7774                  0
2 2013年03月15日          3701.3143      15.9145                  0
3 2013年03月14日          3709.7446      15.9484                  0
4 2013年03月13日          3668.3178      15.8762                  0
5 2013年03月12日          3707.0364      15.9726                  0
6 2013年03月11日          3716.4011      16.0521                  0

Presumably it's because the website is recognising a cookie in my browser and delivering me the English version of the file, but not when I go through R.

Is there a way to get around this? It is causing me some difficulties to convert the CSV file into an XTS because I don't know how to convert the Chinese dates into a Date object.

Thank you.

Answer 1

Supposing you have the httr library installed, try this:

library(httr)

# Get cookies from English page
en_page <- GET("http://hk.ishares.com/product_info/fund/overview/SEHK/2836.htm?ls=true&l=en")

# Get the data
data <- GET("http://hk.ishares.com/product_info/fund/excel_histoverview.htm?ticker=2836",
        set_cookies(en_page$cookies[[1]]))

# Load into a data.frame
a <- read.csv(textConnection(content(data)), skip = 5)

head(a)

Basically, we get the cookies from the English page and we send the request with those.