quantified-flu-fitbit-intraday.ipynb
A notebook that visualizes your resting heart rate before & after flu/cold events. For more experimental use as the data collection isn't as easy and the runtime of the notebook is underwhelming at this point.
A Fitbit allows you to measure vital signs during sleep such as resting heart rate. Here we will use this data to see if there are any patterns that emerge before falling sick. This is part of the #quantifiedflu
project that came out of one of the Open Humans community calls!
This notebook is a bit messier to use, as you will need to use both the regular Fitbit connection on Open Humans as well as the Intraday Fitbit connection. We will need to use both as the intraday one lacks the sleep times, while the regular one does not provide high resolution heart rate data. Long story short, you'll need to set up both connections through the links above to make this work.
There are three parameters you can easily change for this notebook in the cell below:
sick_dates
, is a list of dates on which you fell sick. Enter dates as YYYY-MM-DD
format, e.g. 2019-11-29
for the 29th of November 2019. You can provide as many sick dates as you remember as a comma-separated ist.weeks_before_sick
, decides how many weeks before the actual falling sick you want to have shown in your graphs. By default it is 3
.weeks_after_sick
, decides how many weeks after the event you want to see in your graph. By default it's 1
.The code below will now get your data from Open Humans and create a table with all the data points in the date ranges you specified above:
There's no way around it. This step feels like it takes forever because of how the data is structured. There's probably more clever ways to access the data, but for now it needs to check most of the records to find out where the actual data of our data ranges is by looping over it. Please be patient with this step
Can you observe clear patterns? Is there something unexpected in your data? You can publish your own notebook after running it via the menu above: Click File
-> Share Notebook
-> Upload to Open Humans
to share it with others.
A Fitbit allows you to measure vital signs during sleep such as resting heart rate. Here we will use this data to see if there are any patterns that emerge before falling sick. This is part of the #quantifiedflu
project that came out of one of the Open Humans community calls!
This notebook is a bit messier to use, as you will need to use both the regular Fitbit connection on Open Humans as well as the Intraday Fitbit connection. We will need to use both as the intraday one lacks the sleep times, while the regular one does not provide high resolution heart rate data. Long story short, you'll need to set up both connections through the links above to make this work.
There are three parameters you can easily change for this notebook in the cell below:
sick_dates
, is a list of dates on which you fell sick. Enter dates as YYYY-MM-DD
format, e.g. 2019-11-29
for the 29th of November 2019. You can provide as many sick dates as you remember as a comma-separated ist.weeks_before_sick
, decides how many weeks before the actual falling sick you want to have shown in your graphs. By default it is 3
.weeks_after_sick
, decides how many weeks after the event you want to see in your graph. By default it's 1
.import os
import requests
import tempfile
import json
import arrow
import json
import glob
import pandas as pd
sick_dates = ['2018-12-31']
weeks_before_sick = 2
weeks_after_sick = 1
fitbit_data = requests.get('http://195.201.146.207/STATIC/qf/fitbit-data.json').json()
The code below will now get your data from Open Humans and create a table with all the data points in the date ranges you specified above:
from ohapi import api
import os
import requests
import tempfile
import arrow
"""
user_details = api.exchange_oauth2_member(os.environ.get('OH_ACCESS_TOKEN'))
for i in user_details['data']:
if i['basename'] == 'fitbit-data.json' and i['source'] == 'direct-sharing-102':
fitbit_data = requests.get(i['download_url']).json()
"""
sd_dict = {}
for sd in sick_dates:
sdd = arrow.get(sd)
period_start = sdd.shift(weeks=weeks_before_sick*-1).format('YYYY-MM-DD')
period_end = sdd.shift(weeks=weeks_after_sick).format('YYYY-MM-DD')
sd_dict[sd] = {'period_start': period_start, 'period_end': period_end, 'sleep_start_dates': [], 'sleep_amount_minutes': []}
### YEP, THIS IS PROBABLY ALL AS COMPLICATED AS I COULD MAKE IT, BUT I FEEL A BIT BRAIN DEAD TODAY.
### PLEASE CLEAN UP THE MESS BELOW IF YOU FEEL CAPABLE OF IT!
for p in sd_dict.keys():
for year in fitbit_data['sleep-start-time']:
for entry in fitbit_data['sleep-start-time'][year]['sleep-startTime']:
sdate = arrow.get(entry['dateTime'])
if (sdate >= arrow.get(sd_dict[p]['period_start'])) and (sdate <= arrow.get(sd_dict[p]['period_end'])):
sd_dict[p]['sleep_start_dates'].append(entry)
for p in sd_dict.keys():
for year in fitbit_data['sleep-minutes']:
for entry in fitbit_data['sleep-minutes'][year]['sleep-minutesAsleep']:
sdate = arrow.get(entry['dateTime'])
if (sdate >= arrow.get(sd_dict[p]['period_start'])) and (sdate <= arrow.get(sd_dict[p]['period_end'])):
sd_dict[p]['sleep_amount_minutes'].append(entry)
for p in sd_dict.keys():
sd_dict[p]['entries'] = {}
for entry in sd_dict[p]['sleep_start_dates']:
sd_dict[p]['entries'][entry['dateTime']] = {'start': arrow.get(entry['dateTime'] + " " + entry['value'])}
for entry in sd_dict[p]['sleep_amount_minutes']:
sd_dict[p]['entries'][entry['dateTime']]['end'] = sd_dict[p]['entries'][entry['dateTime']]['start'].shift(minutes=int(entry['value']))
for k in sd_dict.keys():
del sd_dict[k]['sleep_start_dates']
del sd_dict[k]['sleep_amount_minutes']
There's no way around it. This step feels like it takes forever because of how the data is structured. There's probably more clever ways to access the data, but for now it needs to check most of the records to find out where the actual data of our data ranges is by looping over it. Please be patient with this step
files = ['fitbit-intraday-2018-12.json', 'fitbit-intraday-2019-01.json']
hr_data = []
for file in files:
data = requests.get(f'http://195.201.146.207/STATIC/qf/{file}').json()
hr_data += data['activities-heart-intraday']
import json
period = []
timestamp = []
heart_rate = []
for p in sd_dict.keys():
"""
for i in user_details['data']:
if i['source'] == 'direct-sharing-191' and i['basename'] == 'fitbit-intraday-{}.json'.format(sd_dict[p]['period_start'][:7]):
data = json.loads(requests.get(i['download_url']).content)
hr_data = data['activities-heart-intraday']
if sd_dict[p]['period_start'][:7] != sd_dict[p]['period_end'][:7]:
for i in user_details['data']:
if i['source'] == 'direct-sharing-191' and i['basename'] == 'fitbit-intraday-{}.json'.format(sd_dict[p]['period_end'][:7]):
data = json.loads(requests.get(i['download_url']).content)
hr_data = hr_data + data['activities-heart-intraday']
"""
for entry_date,entry in sd_dict[p]['entries'].items():
for hr_point in hr_data:
hr_date = hr_point['date']
if (arrow.get(hr_date) >= arrow.get(sd_dict[p]['period_start'])) & (arrow.get(hr_date) <= arrow.get(sd_dict[p]['period_end'])):
for hr_datapoint in hr_point['dataset']:
hr_timestamp = hr_date + ' ' + hr_datapoint['time']
hr_timestamp = arrow.get(hr_timestamp)
if (hr_timestamp >= entry['start']) and (hr_timestamp <= entry['end']):
period.append(p)
timestamp.append(hr_timestamp)
heart_rate.append(hr_datapoint['value'])
dataframe = pd.DataFrame(
data = {
'period': period,
'timestamp': timestamp,
'heart_rate': heart_rate,
}
)
# Convert hr_data to pandas dataframe
rows = []
for hr_point in hr_data:
hr_date = hr_point['date']
for hr_datapoint in hr_point['dataset']:
rows.append((hr_date + ' ' + hr_datapoint['time'], hr_datapoint['value']))
hr_dataframe = pd.DataFrame(rows, columns=['timestamp', 'heart_rate'])
hr_dataframe['timestamp'] = pd.to_datetime(hr_dataframe['timestamp'], utc=True)
# combine with sick days data
dataframes = []
for p in sd_dict.keys():
for entry_date,entry in sd_dict[p]['entries'].items():
#
start = entry['start'].datetime
end = entry['end'].datetime
filtered = hr_dataframe[(hr_dataframe['timestamp'] >= start) & (hr_dataframe['timestamp'] <= end)].copy()
filtered['period'] = p
dataframes.append(filtered)
dataframe_new = pd.concat(dataframes).reset_index(drop=True)
%load_ext rpy2.ipython
%%R -i dataframe -w 15 -h 4 --units in -r 200
library(ggplot2)
ggplot(subset(dataframe,dataframe$heart_rate > 0), aes(x=as.Date(timestamp),y=heart_rate,color=period)) +
geom_vline(aes(xintercept=as.Date(period)),alpha=0.6,color='red') +
geom_point(alpha=0.1) +
geom_smooth(method='loess') + theme_minimal() + scale_x_date('timestamp') +
facet_grid(. ~ period, scales='free')
Can you observe clear patterns? Is there something unexpected in your data? You can publish your own notebook after running it via the menu above: Click File
-> Share Notebook
-> Upload to Open Humans
to share it with others.