Operations and Data Import

Overview

Loading Packages
Understand data types
Load a CSV file
Organize an analysis project

A quick note on R packages

Packages are essentially free and open source add-ons for R
There are over 7,000 packages available for R that add all sorts of functionality
A few examples
Additional graphics capabilities from, the ggplot2 and the lattice packages
Advanced regression techniques from the lme4 and nlme packages for mixed effects (hierarchical linear models, multilevel models, etc.)
3d graphics from the scatterplot3d package (also webGL)
Text mining analytics with tm
Predictive modeling frameworks with caret
Interfaces to other programming languages like Python, Java, and C and C++
A web server: Rserve
And Minesweeper from the fun package

How to download/update packages?

# You can find and install packages within R
install.packages("sos") # Name must be in quotes
install.packages(c("sos","dplyr","ggplot2"))
# Packages get updated FREQUENTLY
update.packages() # will update them all

Note, on Windows Vista and later R either needs to be installed (run as an administrator to install) as an administrator to install packages
Packages are stored in the folder called the library which is just a collection of packages
- Packages and libraries are used for each other
Loading a package couldn’t be easier library(ggplot2) or require(ggplot2). require() function calls library() function

Finding Packages

Official packages are found on CRAN (Comprehensive R A Network)
Unofficial packages or beta versions of packages are found on RForge and GitHub
To find out what packages are out there that do a specific function, try:
Google “doing X in R package”
Look at CRAN taskviews
CRAN taskviews are great to find a bunch of packages related to a problem you are trying to solve

Some Must Have Packages

plyr ggplot2 lme4 sp knitr sos forecast quantmod stringr XML and much more based on the task(s)!

These packages come with data, code, functions, and utilities that you can access
If you need help run help(package=PackageName)
Some packages also have vignettes which walk you through use cases for the package
caret has a great example

Data Management

Data or objects in R can be listed by the ls() command
Workspace tab in RStudio (right upper window) have a complete list of the objects in R’s memory that is accessible to you
All objects have names
All object names are unique (not strictly so, but let’s not violate this)
To reference items within an object type we need to give it an address like in mydata$thingIwant or mydata@thingIwant
The $ and @ distinction depends on whether this is an S3 or an S4 class
R will warn you if you do it wrong, so just remember when $ doesn’t work, use @

The Working Directory

The working directory, lovingly denoted by wd is both your friend and enemy
R needs to know where it is in your file system to be able to access data and write output
Check this by typing getwd()
R work is best done in a selfcontained directory, like C:/Users/My Documents/My Project/ which is then set as the working directory
How to set the working directory? The setwd() command: setwd("PATH/TO/MY PROJECT/")

Preliminaries

Get and set the working directory
Understand file system paths
Understand relative and full paths
When you do an R project, it really helps to put it into a self-contained folder so you can reference everything you need easily within the file system
Create reproducible documents (aka, rmd files) for each project

Manipulating Project/files Paths

Paths are strings that tell R where to find files on your computer
R uses both full and relative paths
When we want to call files from different directories or write to different directories, we can use relative paths within the project
Relative paths look like /data or ~data which means, look for the data folder in our current working directory
Full paths work differently, they specify the exact location on the hard drive of the machine like C:/Path/To/My/Data or usr/home/jaredrocks

When to use a full path

Never
It breaks the ability to pass a project directory to a coauthor/collaborator
But when you must…
When you have something on a whole different drive or network store
When you don’t think you’ll migrate your code to a new operating system or a different machine/network environment
When the project is simple

Ground Rules

Get used to plain text input files
R can handle other formats, but your error rate increases as does the tweaking necessary
R has a limited set of special characters (symbols) you cannot use in your data input to be translated correctly
These symbols are reserved and will be interpreted in strange ways if you include them in your plain text data file
Most of them are fairly obvious operators, see Paul Murrell’s excellent summary. But to get help pages try ?">, ?"%*%, ?"&&" or ?"|".

Missing Data Symbols

Missing data has the symbols NA or NaN or NULL depending on the context.
Consider:

a<-c(1,2,3)  # a is a vector with three elements
# Ask R for element 4
print(a[4]) # or simply a[4]

## [1] NA

But what is the difference between NA and NULL?

a<-c(a,NULL) # Append NULL onto a
print(a)

## [1] 1 2 3

length(a)

## [1] 3

# Notice no change
a<-c(a,NA)
print(a)

## [1]  1  2  3 NA

length(a)

## [1] 4

NA can hold a place, NULL cannot

What the heck is Not a Number?

NaN is even more special, and only holds things like imaginary numbers
NaN stands for “Not a Number”

b<-1
b<-sqrt(-b)

## Warning in sqrt(-b): NaNs produced

print(b)

## [1] NaN

pi/0

## [1] Inf

Inf is a special case as well representing an infinite value
Just for fun sin(Inf) = NaN

Organization of a analysis/project is key

Separate data from scripts
Separate automatic scripts from interactive scripts
Put figures apart from both of these
Always keep your raw data
Always write comment(s) in script file

Read in Data (aka import data)

Reading in data is one of the trickiest issues for R
This is because R is incredibly flexible and can handle data in almost any form including .csv .dta .sas .spss .dat and even .xls and .xlsx with some care
So we have to carefully specify the data types to R so it can understand what form the data needs to take
Even we can read data on the web, import tables from html pages. See WebTechnologies task view.

CSV is Our Friend

The easiest data type is .csv though Excel files can be read as well

#Load some data
oil<-read.csv("http://yunus.hacettepe.edu.tr/~iozkan/data/oilcsv.csv", header=T, sep=";")
# Note if we don't assign data to 'oil' using `<-` operator
# R just prints contents of table

Let’s Check What We Got

## 'data.frame':    7142 obs. of  4 variables:
##  $ Date : Factor w/ 7142 levels "01.02.1988","01.02.1989",..: 216 449 1150 1386 1625 1865 2103 2818 3055 3293 ...
##  $ Crude: num  25.6 26 26.5 25.9 25.9 ...
##  $ Brent: num  NA NA NA NA NA NA NA NA NA NA ...
##  $ X    : logi  NA NA NA NA NA NA ...

Always Check Your Data

A few great commands:

  dim(oil)

## [1] 7142    4

summary

  summary(oil[,1:3])

##          Date          Crude            Brent       
##  01.02.1988:   1   Min.   : 10.25   Min.   :  9.10  
##  01.02.1989:   1   1st Qu.: 18.92   1st Qu.: 18.08  
##  01.02.1990:   1   Median : 25.32   Median : 25.23  
##  01.02.1991:   1   Mean   : 40.69   Mean   : 42.28  
##  01.02.1993:   1   3rd Qu.: 62.05   3rd Qu.: 63.12  
##  01.02.1994:   1   Max.   :145.31   Max.   :143.95  
##  (Other)   :7136   NA's   :98       NA's   :410

Checking your data II

names

  names(oil)

## [1] "Date"  "Crude" "Brent" "X"

attributes and class

  names(attributes(oil))

## [1] "names"     "class"     "row.names"

  class(oil)

## [1] "data.frame"

str which lists all data elements in an object and their type

Another Example From About Getting Data From Internet

Here is the example code from UCLA introduction to R slides. xlsx package depends on rJava and xlsxjars. If you have diffuculties to get Java installed on your computer try another packages (or better use csv files instead of xls-xlsx files. Many computer do show an error message error: JAVA_HOME cannot be determined from the Registry).

require(foreign)

## Loading required package: foreign

# SPSS files
dat.spss <- read.spss("http://www.ats.ucla.edu/stat/data/hsb2.sav", to.data.frame=TRUE)
# Stata files
dat.dta <- read.dta("http://www.ats.ucla.edu/stat/data/hsb2.dta")
#

require(readxl)

## Loading required package: readxl

## Warning: package 'readxl' was built under R version 3.1.3

# these two steps only needed to read excel files from the internet
f <- tempfile("hsb2", fileext=".xls")

# 1- Download file to a temporary file -- here it is f
download.file("http://www.ats.ucla.edu/stat/data/hsb2.xls", f, mode="wb")

# 2- read xls file from this temporary file
dat.xls <- read_excel(f, sheet=1)

Other References

Session Info

It is good to include the session info, e.g. this document is produced with knitr version 1.8.2. Here is my session info:

print(sessionInfo(), locale=FALSE)

## R version 3.1.1 (2014-07-10)
## Platform: i386-w64-mingw32/i386 (32-bit)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] readxl_0.1.0   foreign_0.8-61
## 
## loaded via a namespace (and not attached):
##  [1] digest_0.6.4    evaluate_0.5.5  formatR_1.0     htmltools_0.2.6
##  [5] knitr_1.8.2     Rcpp_0.11.5     rmarkdown_0.6.1 stringr_0.6.2  
##  [9] tools_3.1.1     yaml_2.1.13

Introduction to R

Click here for A Brief Introduction to R.

Attribution and License

This work (R Tutorial for Education, by Jared E. Knowles), in service of the Wisconsin Department of Public Instruction, is free of known copyright restrictions. Some pages were deleted/inserted by I. Ozkan to make more suitable to the Economics Students.