Details for plot-overland.ipynb

Published by gedankenstuecke

Description

Use GPS data collected with Overland on an iOS device and plot it on a map. The notebook demonstrates both how to use a data set and make a map with all the data points and gives examples on how to focus on certain regions.

0

Tags & Data Sources

GPS location data location movement maps mapping Overland connection

Comments

Please log in to comment.

Notebook
Last updated 3 years, 11 months ago

Analyzing your GPS data from Overland

This notebook requires you to use the Overland connection to get your GPS data from your iPhone into Open Humans.

This notebook then uses the GPS data to plot your personal movement history using R.

For a start let's load our required packages:

also installing the dependencies ‘sp’, ‘RgoogleMaps’, ‘png’, ‘rjson’, ‘jpeg’

Updating HTML index of packages in '.Library'
Making 'packages.html' ... done

Attaching package: ‘purrr’

The following object is masked from ‘package:jsonlite’:

    flatten

With that out of the way we can access our Overland data from our Open Humans account. As the GPS records are can grow pretty large, each Year-Month will get it's own file. The code below will randomly use one of your files.

If you want to adjust it to a specific month you can change overland-data to something like overland-data-2018-12 following the overland-data-YYYY-MM format. By default it uses 2019-06 as the data points

Now we can start downloading the data:

[1] "read first month"
longitudelatitudeactivityaltitudebattery_levelbattery_statedeferreddesired_accuracyhorizontal_accuracymotionpausessignificant_changespeedtimestampvertical_accuracywifivelocitydatelonlat
2.37397 48.85856 other 97 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:00:45Z10 pguyon 0 2020-04-01T00:00:45Z2.37397 48.85856
2.37397 48.85856 other 90 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:15:45Z10 pguyon 0 2020-04-01T00:15:45Z2.37397 48.85856
2.37397 48.85856 other 90 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:30:50Z10 pguyon 0 2020-04-01T00:30:50Z2.37397 48.85856
2.37397 48.85856 other 90 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:45:50Z10 pguyon 0 2020-04-01T00:45:50Z2.37397 48.85856
2.37397 48.85856 other 88 1 full 0 100 2500 stationary False 0 0 2020-04-01T01:00:50Z10 pguyon 0 2020-04-01T01:00:50Z2.37397 48.85856
2.37397 48.85856 other 89 1 full 0 100 2500 stationary False 0 0 2020-04-01T01:15:50Z10 pguyon 0 2020-04-01T01:15:50Z2.37397 48.85856

Now we can look into how our data looks like:

Getting started with the data processing

Now that we have our fitbit data stored in loc we can start to work with that data. Much of this notebook is adapted code from Shirin Glander's excellent blogpost, which is based on Google Location History data but the same principles apply for our Overland data.

Analyzing the data

Let's install the required ggmap package and load some more data visualization tools:

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date


Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Google's Terms of Service: https://cloud.google.com/maps-platform/terms/.
Please cite ggmap if you use it! See citation("ggmap") for details.

Let's make a map

The global view

We will start by looking at a map that will include all GPS points recorded for the data set you are looking at:

Below we define our boundaries that we want to plot by giving the minimum and maximum latitude/longitude of the map we want to see. By default we use the minimum/maximum values observed in our data.

Source : http://tile.stamen.com/terrain/14/8299/5633.png
Source : http://tile.stamen.com/terrain/14/8300/5633.png
Source : http://tile.stamen.com/terrain/14/8301/5633.png
Source : http://tile.stamen.com/terrain/14/8302/5633.png
Source : http://tile.stamen.com/terrain/14/8299/5634.png
Source : http://tile.stamen.com/terrain/14/8300/5634.png
Source : http://tile.stamen.com/terrain/14/8301/5634.png
Source : http://tile.stamen.com/terrain/14/8302/5634.png
Source : http://tile.stamen.com/terrain/14/8299/5635.png
Source : http://tile.stamen.com/terrain/14/8300/5635.png
Source : http://tile.stamen.com/terrain/14/8301/5635.png
Source : http://tile.stamen.com/terrain/14/8302/5635.png
Source : http://tile.stamen.com/terrain/14/8299/5636.png
Source : http://tile.stamen.com/terrain/14/8300/5636.png
Source : http://tile.stamen.com/terrain/14/8301/5636.png
Source : http://tile.stamen.com/terrain/14/8302/5636.png

Now that we have the map downloaded we can start plotting our own movements on that map:

Each location point is put on the map, with the color denoting the type of travel that Overland has tried to identify for the given movement. Let's now zoom in into a given part of the map. For this we can redownload a map with the coordinates that we want to use. You can adjust the boundaries in the cell below:

And now we can plot that data:

Warning message:
“Removed 25660 rows containing missing values (geom_point).”

Now we can see on a smaller level where and how we've moved!

This makes for a much more useful map in many cases. Getting the Latitude / Longitude parameters in the cells above right can take some experimentation, to zoom in on the right point of the map. Give it a try!

Notebook
Last updated 3 years, 11 months ago

Analyzing your GPS data from Overland

This notebook requires you to use the Overland connection to get your GPS data from your iPhone into Open Humans.

This notebook then uses the GPS data to plot your personal movement history using R.

For a start let's load our required packages:

In [1]:
library(httr)
library(jsonlite)
library(ggplot2)
library(devtools)
install.packages('ggmap')
library(purrr)
also installing the dependencies ‘sp’, ‘RgoogleMaps’, ‘png’, ‘rjson’, ‘jpeg’

Updating HTML index of packages in '.Library'
Making 'packages.html' ... done

Attaching package: ‘purrr’

The following object is masked from ‘package:jsonlite’:

    flatten

With that out of the way we can access our Overland data from our Open Humans account. As the GPS records are can grow pretty large, each Year-Month will get it's own file. The code below will randomly use one of your files.

If you want to adjust it to a specific month you can change overland-data to something like overland-data-2018-12 following the overland-data-YYYY-MM format. By default it uses 2019-06 as the data points

In [8]:
month <- '2020-04'

Now we can start downloading the data:

In [9]:
access_token <- Sys.getenv("OH_ACCESS_TOKEN")
url <- paste("https://www.openhumans.org/api/direct-sharing/project/exchange-member/?access_token=",access_token,sep="")
resp <- GET(url)
user <- content(resp, "parsed")

user_data <- user$data

while (!is.null(user$`next`)) {
    resp <- GET(user$`next`)
    user <- content(resp, "parsed")
    user_data <- append(user_data, user$data)
}

for (data_source in user_data){
    if (grepl(paste('overland-data',month,sep='-'), data_source$basename)){
        print('read first month')
        loc <- read.csv(url(data_source$download_url))
    }
}

loc$velocity <- loc$speed
loc$date <- loc$timestamp
loc$lon <- loc$longitude
loc$lat <- loc$latitude

head(loc)
[1] "read first month"
longitudelatitudeactivityaltitudebattery_levelbattery_statedeferreddesired_accuracyhorizontal_accuracymotionpausessignificant_changespeedtimestampvertical_accuracywifivelocitydatelonlat
2.37397 48.85856 other 97 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:00:45Z10 pguyon 0 2020-04-01T00:00:45Z2.37397 48.85856
2.37397 48.85856 other 90 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:15:45Z10 pguyon 0 2020-04-01T00:15:45Z2.37397 48.85856
2.37397 48.85856 other 90 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:30:50Z10 pguyon 0 2020-04-01T00:30:50Z2.37397 48.85856
2.37397 48.85856 other 90 1 full 0 100 2500 stationary False 0 0 2020-04-01T00:45:50Z10 pguyon 0 2020-04-01T00:45:50Z2.37397 48.85856
2.37397 48.85856 other 88 1 full 0 100 2500 stationary False 0 0 2020-04-01T01:00:50Z10 pguyon 0 2020-04-01T01:00:50Z2.37397 48.85856
2.37397 48.85856 other 89 1 full 0 100 2500 stationary False 0 0 2020-04-01T01:15:50Z10 pguyon 0 2020-04-01T01:15:50Z2.37397 48.85856

Now we can look into how our data looks like:

Getting started with the data processing

Now that we have our fitbit data stored in loc we can start to work with that data. Much of this notebook is adapted code from Shirin Glander's excellent blogpost, which is based on Google Location History data but the same principles apply for our Overland data.

Analyzing the data

Let's install the required ggmap package and load some more data visualization tools:

In [10]:
library(lubridate)
library(zoo)
# set up plotting theme
library(ggplot2)
library(ggmap)

my_theme <- function(base_size = 12, base_family = "sans"){
  theme_grey(base_size = base_size, base_family = base_family) +
  theme(
    axis.text = element_text(size = 12),
    axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1),
    axis.title = element_text(size = 14),
    panel.grid.major = element_line(color = "grey"),
    panel.grid.minor = element_blank(),
    panel.background = element_rect(fill = "aliceblue"),
    strip.background = element_rect(fill = "lightgrey", color = "grey", size = 1),
    strip.text = element_text(face = "bold", size = 12, color = "navy"),
    legend.position = "right",
    legend.background = element_blank(),
    panel.margin = unit(.5, "lines"),
    panel.border = element_rect(color = "grey", fill = NA, size = 0.5)
  )
}
Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date


Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Google's Terms of Service: https://cloud.google.com/maps-platform/terms/.
Please cite ggmap if you use it! See citation("ggmap") for details.

Let's make a map

The global view

We will start by looking at a map that will include all GPS points recorded for the data set you are looking at:

Below we define our boundaries that we want to plot by giving the minimum and maximum latitude/longitude of the map we want to see. By default we use the minimum/maximum values observed in our data.

In [11]:
loc2 <- loc
boundary_west=min(loc2$lon)
boundary_east=max(loc2$lon)
boundary_south=min(loc2$lat)
boundary_north=max(loc2$lat)
loc2 <- loc2[which(!is.na(loc2$velocity)), ]


my_map <- get_map(location=c(boundary_west,
                               boundary_south,
                               boundary_east,
                               boundary_north), source = "stamen",maptype='toner-lite')
Source : http://tile.stamen.com/terrain/14/8299/5633.png
Source : http://tile.stamen.com/terrain/14/8300/5633.png
Source : http://tile.stamen.com/terrain/14/8301/5633.png
Source : http://tile.stamen.com/terrain/14/8302/5633.png
Source : http://tile.stamen.com/terrain/14/8299/5634.png
Source : http://tile.stamen.com/terrain/14/8300/5634.png
Source : http://tile.stamen.com/terrain/14/8301/5634.png
Source : http://tile.stamen.com/terrain/14/8302/5634.png
Source : http://tile.stamen.com/terrain/14/8299/5635.png
Source : http://tile.stamen.com/terrain/14/8300/5635.png
Source : http://tile.stamen.com/terrain/14/8301/5635.png
Source : http://tile.stamen.com/terrain/14/8302/5635.png
Source : http://tile.stamen.com/terrain/14/8299/5636.png
Source : http://tile.stamen.com/terrain/14/8300/5636.png
Source : http://tile.stamen.com/terrain/14/8301/5636.png
Source : http://tile.stamen.com/terrain/14/8302/5636.png

Now that we have the map downloaded we can start plotting our own movements on that map:

In [15]:
loc2$time <-strptime(loc2$timestamp,"%Y-%m-%dT%H:%M:%S")
ggmap(my_map) + 
    geom_point(data = subset(loc2, as.Date(loc2$date) > as.Date('2018-10-10')), 
                aes(x = lon, 
                    y = lat),color='red') +
                    
  theme(legend.position = "right") + 
  labs(x = "Longitude", y = "Latitude", 
       title = "My Location History",
       subtitle = "")

Each location point is put on the map, with the color denoting the type of travel that Overland has tried to identify for the given movement. Let's now zoom in into a given part of the map. For this we can redownload a map with the coordinates that we want to use. You can adjust the boundaries in the cell below:

In [21]:
boundary_west=7.616
boundary_east=7.63 # for those I wouldn't have needed to ask Google thanks to the Prime Meridian in Greenwich :D
boundary_south=51.96
boundary_north=51.97

my_map <- get_map(location=c(boundary_west,
                               boundary_south,
                               boundary_east,
                               boundary_north), source = "stamen",maptype='toner')

And now we can plot that data:

In [22]:
ggmap(my_map) + 
    geom_point(data = subset(loc2), 
                aes(x = lon, 
                    y = lat, 
                    color = motion)) +
  theme(legend.position = "right")
Warning message:
“Removed 25660 rows containing missing values (geom_point).”

Now we can see on a smaller level where and how we've moved!

This makes for a much more useful map in many cases. Getting the Latitude / Longitude parameters in the cells above right can take some experimentation, to zoom in on the right point of the map. Give it a try!

In [ ]: