Attempting to data clean and merge Actilife and GPS data files

Loading packages

Our intention is to use packages from the tidyverse and other tidy packages to provide a constistant framework.

library(DT)
library(plotly)

Loading required package: ggplot2
package 㤼㸱ggplot2㤼㸲 was built under R version 3.5.1
Attaching package: 㤼㸱plotly㤼㸲

The following object is masked from 㤼㸱package:ggplot2㤼㸲:

    last_plot

The following object is masked from 㤼㸱package:stats㤼㸲:

    filter

The following object is masked from 㤼㸱package:graphics㤼㸲:

    layout

library(tidyverse) # data manipulation and visualization - loads several packages

-- Attaching packages --------------------------------------- tidyverse 1.2.1 --
-- Conflicts ------------------------------------------ tidyverse_conflicts() --
x dplyr::filter() masks plotly::filter(), stats::filter()
x dplyr::lag()    masks stats::lag()

library(readr) # reading data files
library(lubridate) # working with dates

Attaching package: 㤼㸱lubridate㤼㸲

The following object is masked from 㤼㸱package:base㤼㸲:

    date

library(tibbletime)

package 㤼㸱tibbletime㤼㸲 was built under R version 3.5.1
Attaching package: 㤼㸱tibbletime㤼㸲

The following object is masked from 㤼㸱package:stats㤼㸲:

    filter

library(tidyquant)

package 㤼㸱tidyquant㤼㸲 was built under R version 3.5.1Loading required package: PerformanceAnalytics
package 㤼㸱PerformanceAnalytics㤼㸲 was built under R version 3.5.1Loading required package: xts
package 㤼㸱xts㤼㸲 was built under R version 3.5.1Loading required package: zoo
package 㤼㸱zoo㤼㸲 was built under R version 3.5.1
Attaching package: 㤼㸱zoo㤼㸲

The following objects are masked from 㤼㸱package:base㤼㸲:

    as.Date, as.Date.numeric


Attaching package: 㤼㸱xts㤼㸲

The following objects are masked from 㤼㸱package:dplyr㤼㸲:

    first, last

Version 0.4-0 included new data defaults. See ?getSymbols.
Learn from a quantmod author: https://www.datacamp.com/courses/importing-and-managing-financial-data-in-r

library(sf)

Linking to GEOS 3.6.1, GDAL 2.2.3, proj.4 4.9.3

library(tmap)
library(crosstalk)

Data munging

Load and clean ActiLife Data

Read the file and create new column which encodes data-time as a single variable.

Using tibbletime commands, we will make this dataframe “time aware”.

We are basing these manipulations off the tutorial http://www.business-science.io/code-tools/2017/10/26/demo_week_tibbletime.html .

df=read_csv("Autumn ActiLife Data.csv",skip=10)
df <- df %>% mutate(Date=mdy(Date)) %>% mutate(date_time=ymd_hms(paste(Date,Time))) %>%
  janitor::clean_names() %>%
  as_tbl_time(index = date_time)
str(df)

Classes ‘tbl_time’, ‘tbl_df’, ‘tbl’ and 'data.frame':   3468 obs. of  13 variables:
 $ date                 : Date, format: "2018-06-29" "2018-06-29" "2018-06-29" ...
 $ time                 : 'hms' num  09:15:00 09:16:00 09:17:00 09:18:00 ...
  ..- attr(*, "units")= chr "secs"
 $ axis1                : int  0 14 0 1025 449 136 1128 0 0 0 ...
 $ axis2                : int  18 9 0 1066 417 275 537 34 80 0 ...
 $ axis3                : int  0 3 0 2275 1532 706 1798 0 19 0 ...
 $ steps                : int  0 1 0 15 9 4 8 0 0 0 ...
 $ lux                  : int  0 0 0 0 0 0 0 0 0 0 ...
 $ inclinometer_off     : int  44 0 0 0 0 0 0 0 0 0 ...
 $ inclinometer_standing: int  0 0 0 60 60 56 60 60 56 59 ...
 $ inclinometer_sitting : int  0 55 60 0 0 4 0 0 4 1 ...
 $ inclinometer_lying   : int  16 5 0 0 0 0 0 0 0 0 ...
 $ vector_magnitude     : num  18 16.9 0 2713.4 1650 ...
 $ date_time            : POSIXct, format: "2018-06-29 09:15:00" "2018-06-29 09:16:00" "2018-06-29 09:17:00" ...
 - attr(*, "index_quo")= language ~date_time
  ..- attr(*, ".Environment")=<environment: 0x0000000022777db8> 
 - attr(*, "index_time_zone")= chr "UTC"

Load and clean GPS data

First we load as a data frame and create a date-time variable.

df2 <- read_csv("Autumn 7-1-18.csv") %>% mutate(Date=mdy(Date)) %>% rename(Time='Time (local)') %>%
  mutate(date_time=ymd_hms(paste(Date,Time))) %>% janitor::clean_names()

Parsed with column specification:
cols(
  Date = col_character(),
  `Time (local)` = col_time(format = ""),
  Latitude = col_double(),
  Longitude = col_double(),
  `Altitude (feet)` = col_double(),
  `Speed (mi/hr)` = col_double()
)

We convert do a tibbletime object and change the periodicity to 1 minute intervals.

We chose to average all data from time xx:xx:00-xx:xx:59, and then round down to the nearest minute

df3_tt=df2 %>%
    as_tbl_time(index = date_time) %>% 
  as_period("minutes",side="start" ) %>% mutate(date_time=floor_index(date_time,unit="minutes"))

Merge the two data frames

We will merge all of the data, recognizing that not all times exist in both data sets, but we also create dfg, which restricts to those points with gps data

dffull=full_join(df,df3_tt,by="date_time") 
dfg=right_join(df,df3_tt,by="date_time") 

Some visualizations using ggplot2

dfg %>% ggplot( aes(x=date_time, y= longitude,col=speed_mi_hr)) +geom_point(aes())

dfg %>% ggplot( aes(x=date_time, y= latitude,col=speed_mi_hr)) +geom_point(aes())

dfg %>% ggplot( aes(x=longitude, y= latitude,col=steps)) +geom_point(aes())

Creating sf object and looking at Maps

We create a sf simple features object in an xy coordinate system and plot using ggplot.

dfg1=st_as_sf(dfg, coords=c("longitude" ,"latitude"))
#dfg1 %>% ggplot(aes(col=steps)) + geom_sf() +theme_minimal()+ scale_colour_viridis_c()

We also plot this data on a tmap plot:

tmap_mode("view")

tmap mode set to interactive viewing

#tmap_mode("plot")
tm_shape(dfg1) + 
  tm_dots("inclinometer_standing")+ 
  tm_view(alpha = 1,basemaps.alpha = 1)+
  tm_basemap(c(Canvas = "Esri.WorldGrayCanvas", Imagery = "Esri.WorldImagery",OpenStreet="OpenStreetMap"))

Currect projection of shape dfg1 unknown. Long-lat (WGS84) is assumed.

Can we do a multiview plot with crosstalk?

sd=SharedData$new(dfg1)
p1=tm_shape(sd$data()) + 
  tm_dots("speed_mi_hr")+ 
  tm_view(alpha = 1,basemaps.alpha = 1)+
  tm_basemap(c(Canvas = "Esri.WorldGrayCanvas", Imagery = "Esri.WorldImagery"))
lf <- tmap_leaflet(p1)

Currect projection of shape sd$data() unknown. Long-lat (WGS84) is assumed.

p2=datatable(sd)
p3=sd %>% ggplot(aes(x=date_time,y=steps))+geom_point() 
ggplotly(p3) %>%   highlight("plotly_selected", dynamic = TRUE)

Adding more colors to the selection color palette
Setting the `off` event (i.e., 'plotly_deselect') to match the `on` event (i.e., 'plotly_selected'). You can change this default via the `highlight()` function.
Setting the `off` event (i.e., 'plotly_deselect') to match the `on` event (i.e., 'plotly_selected'). You can change this default via the `highlight()` function.

p1=leaflet(sd) %>% addProviderTiles("OpenStreetMap") %>% addCircles()
p1

# bscols(p1,datatable(sd))
bscols(p1,ggplotly(p3) %>%   highlight("plotly_selected", dynamic = TRUE))

Adding more colors to the selection color palette
Setting the `off` event (i.e., 'plotly_deselect') to match the `on` event (i.e., 'plotly_selected'). You can change this default via the `highlight()` function.
Setting the `off` event (i.e., 'plotly_deselect') to match the `on` event (i.e., 'plotly_selected'). You can change this default via the `highlight()` function.

#bscols( lf, ggplotly(p3) %>% highlight(color="red"))
#bscols( lf, datatable(sd))
#bscols( ggplotly(p3), datatable(sd))