Modernizations

.gitignore

@@ -1,3 +1,5 @@
 *.gpx
 *.csv
 *.fit
+__pycache__/
+*.lock

.python-version

@@ -0,0 +1 @@
+3.13

ReadMe.md

@@ -1,52 +1,67 @@
 # PythonHeatmap
 
-This is a simple way to visualize GPS data from Strava/Garmin/Polar in either the csv, fit, and/or gpx formats on an attractive and interactive interface.
-This was a side-project while I helped organize the McGill Physics Hackathon.
+PythonHeatmap is a tool for visualizing GPS data from Strava, Garmin, Polar, and other fitness platforms in CSV, FIT, and/or GPX formats as interactive heatmaps.
 
-If you are interested in physics or programming, hackathons are a great idea. If you're curious about more of my work, the linked pages at mrhheffernan.github.io provide links to more information.
+## Features
 
-An additional feature is present to color the lines by heart rate, which can be found in the hr_color branch by tjrademaker. The code in that branch also incoporates tcx file support. All original code there is written by tjrademaker and is offered under the MIT License.
+- Supports GPX, and FIT file formats
+- Interactive HTML heatmaps using Folium
+- CLI arguments for customization
+- Configurable timezone handling
 
 ## Getting Started
 
-Download your data as a gpx, csv, or fit file from your provider of choice. For advanced users, `selenium_downloader.py` is provided to automate this process. These users will have to specify some paths and have selenium/chromium configured before running the script. Additionally, they will have to supply a file called `login_info.secret`. This file should contain `username,password,athlete_id` and will be read in by `selenium_downloader.py`. Currently, `selenium_downloader.py` may not export all data, but is intended for use for the past 12 months of activities. It sometimes downloads more.
+### Prerequisites
 
-Most users can simply request their data as a download from Strava.
+Python 3.13 or higher is required. Dependencies are managed automatically by uv - no installation required.
 
-Note that extra python packages may be required if you have fit files, as the binary files are not easily readable on all systems. Just download the python files here and run them! This is also written in to be compatible with Python 3.7, certain rewrites will be necessary if using Python2.
+### Obtaining Data
 
-I'm Montreal based, so the map is currently designed to center on Montreal. To correct for this, change "Montreal Quebec" to your location!
+Download GPS data from your fitness platform. For Strava, bulk exports are available under account settings. For Garmin Connect, compressed `.fit.gz` files may need to be extracted:
 
+```bash
+gunzip *.fit.gz
+```
 
-### Prerequisites
+For automated downloading, `selenium_downloader.py` is provided. This requires:
+- Selenium and Chromium configured
+- A `login_info.secret` file containing `username,password,athlete_id`
 
-Certain Python modules are required. They are: numpy, pandas, geopy, folium, gpxpy, fitparse, and pytz. To download any and all of these in one fell swoop, the below code is provided.
+Note: The selenium downloader may not export all historical data and is best suited for the most recent 12 months of activities. This script is not maintained or tested with updates to Strava's UI and may require adjustments to work with current Strava versions.
 
-```
-pip install numpy pandas geopy folium gpxpy fitparse pytz
-```
+### Running
 
-Also required prerequisites are GPS tracks. On Strava, these are available for bulk download under settings. If files have been uploaded via Garmin Connect, there may be compressed .fit files in .fit.gz format. To unzip these (at least in linux/unix-based systems):
-```
-gunzip *.fit.gz
-```
+Run from the directory containing your GPS files:
 
-## Running the tests
+```bash
+uv run personal_heatmap.py
+```
 
-The heatmap will be output in a html file, which is viewable in a web browser. Currently, there is no native folium support for image exports, so screenshots of relevant areas is the recommended strategy.
+#### CLI Options
 
-The Python is designed to run in the same directory as the GPS files, so make sure this is the case.
+- `--dir`: Directory containing .fit and .gpx files (default: current directory)
+- `--timezone`: Timezone for timestamps, e.g., 'US/Pacific' (default: 'US/Pacific')
+- `--output_path`: Path for the output heatmap HTML file (default: 'heatmap.html')
 
-To run:
+For FIT to CSV conversion:
 
+```bash
+uv run fit_to_csv.py --dir /path/to/files --timezone US/Pacific
 ```
-python personal_heatmap.py
+
+- `--overwrite`: Overwrite existing CSV files
+
+For the simple matplotlib-based heatmap:
+
+```bash
+uv run simple_heatmap.py --dir /path/to/files --output_path output.png
 ```
 
-## License
+## Output
 
-Map tiles by Stamen Design, under CC BY 3.0. Data by OpenStreetMap, under ODbL.
+The heatmap is generated as an HTML file viewable in any web browser. Use the interactive map controls to navigate and zoom to desired areas.
 
+## License
 Original Python Copyright 2018 Matthew Heffernan
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
@@ -55,21 +70,8 @@ The above copyright notice and this permission notice shall be included in all c
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
-This code is by Matthew Heffernan.  As long as you retain this notice you
-can do whatever you want with this stuff, subject to the conditions above.
-If we meet some day, and you think this stuff is worth it, you can buy me a beer
-in return.   - Matthew Heffernan
-
 ## Acknowledgements
-This code is built with a combination of original and unlicensed code. Special thanks are due to the developers working to make the FIT file format more accessible, especially Max Candocia whose fit_to_csv code is instrumental and included here. Source: https://maxcandocia.com/article/2017/Sep/22/converting-garmin-fit-to-csv/
-
-Additional thanks are due to the McGill Physics Hackathon 2018, during which I wrote this code while assisting many capable hackers visualize physics concepts. Their dedication and the unlimited coffee were inspirational to the development of this project.
-
-## simple_heatmap.py
-This is a simple heatmap which does not superimpose the tracks on a map, but does provide a simple playground for plotting tracks. This reproduces much of the functionality of some prominent Strava apps, but full resolution is gained for free and is more customizable with matplotlib. Enjoy! This script will additionally required the matplotlib module.
 
-This doesn't automatically center, but the native zooming interface will allow you to better crop the heatmap for use on social media. The GUI save feature is recommended.
+This code builds upon original work and tools for making the FIT file format more accessible. Special thanks to Max Candocia, whose fit_to_csv code is instrumental to this project. Source: https://maxcandocia.com/article/2017/Sep/22/converting-garmin-fit-to-csv/
 
-## Upcoming work:
-..*Add option to plot heatmap in style of: http://qingkaikong.blogspot.com/2016/06/using-folium-3-heatmap.html
-..*Broadening the scope of `selenium_downloader.py`
+Additional thanks to the McGill Physics Hackathon 2018, during which this project was developed while assisting participants with visualizing physics concepts.

fit_to_csv.py

@@ -1,12 +1,14 @@
+import argparse
 import csv
+import glob
 import os
+from datetime import timezone
+from typing import Any
+from zoneinfo import ZoneInfo
 
-# to install fitparse, run
-# sudo pip3 install -e git+https://github.com/dtcooper/python-fitparse#egg=python-fitparse
 import fitparse
-import pytz
 
-allowed_fields = [
+FIELDS_ALLOWED = [
     "timestamp",
     "position_lat",
     "position_long",
@@ -15,61 +17,120 @@
     "altitude",
     "enhanced_speed",
     "speed",
+    "avg_heart_rate",
     "heart_rate",
     "cadence",
     "fractional_cadence",
 ]
-required_fields = ["timestamp", "position_lat", "position_long", "altitude"]
+FIELDS_REQUIRED = ["timestamp", "position_lat", "position_long"]
 
-UTC = pytz.UTC
-CST = pytz.timezone("US/Central")
+UTC = timezone.utc
+TZ = ZoneInfo("US/Pacific")
 
 
-def main():
-    files = os.listdir()
-    fit_files = [file for file in files if file[-4:].lower() == ".fit"]
-    for file in fit_files:
-        new_filename = file[:-4] + ".csv"
-        if os.path.exists(new_filename):
-            # print('%s already exists. skipping.' % new_filename)
-            continue
-        fitfile = fitparse.FitFile(
-            file, data_processor=fitparse.StandardUnitsDataProcessor()
-        )
+def write_to_csv(data: list[dict[str, Any]], output_path: str) -> None:
+    """Write extracted data fields from the .fit messages to file
 
-        print("converting %s" % file)
-        write_fitfile_to_csv(fitfile, new_filename)
-    print("finished conversions")
+    Args:
+        data (list[dict[str, Any]]): Data from messages
+        output_path (str): Output path
+    """
+    # write to csv
+    with open(output_path, "w") as f:
+        writer = csv.writer(f)
+        writer.writerow(FIELDS_ALLOWED)
+        for entry in data:
+            writer.writerow([str(entry.get(k, "")) for k in FIELDS_ALLOWED])
+    print("wrote %s" % output_path)
 
 
-def write_fitfile_to_csv(fitfile, output_file="test_output.csv"):
-    messages = fitfile.messages
+def collect_data(filepath: str, tz: ZoneInfo = TZ) -> list[dict[str, Any]]:
+    """Collects data from the .fit file at filepath
+
+    Args:
+        filepath (str): Path to .fit file
+        tz (ZoneInfo, optional): Timezone identifier. Defaults to TZ.
+
+    Returns:
+        list[dict[str, Any]]: List of dicts containing relevant data from each message in the .fit
+    """
+    # Parse the .fit file
+    fitfile = fitparse.FitFile(
+        filepath, data_processor=fitparse.StandardUnitsDataProcessor()
+    )
+
     data = []
+    messages = fitfile.messages
+
     for m in messages:
         skip = False
         if not hasattr(m, "fields"):
             continue
         fields = m.fields
-        # check for important data types
+
+        # check for desired data and collect it
         mdata = {}
         for field in fields:
-            if field.name in allowed_fields:
+            if field.name in FIELDS_ALLOWED:
                 if field.name == "timestamp":
-                    mdata[field.name] = UTC.localize(field.value).astimezone(CST)
+                    timestamp_value = field.value
+                    if timestamp_value.tzinfo is None:
+                        timestamp_value = timestamp_value.replace(tzinfo=UTC)
+                    mdata[field.name] = timestamp_value.astimezone(tz)
                 else:
                     mdata[field.name] = field.value
-        for rf in required_fields:
-            if rf not in mdata:
+
+        for required_field in FIELDS_REQUIRED:
+            if required_field not in mdata:
                 skip = True
+
         if not skip:
             data.append(mdata)
-    # write to csv
-    with open(output_file, "w") as f:
-        writer = csv.writer(f)
-        writer.writerow(allowed_fields)
-        for entry in data:
-            writer.writerow([str(entry.get(k, "")) for k in allowed_fields])
-    print("wrote %s" % output_file)
+
+    return data
+
+
+def parse_args() -> argparse.Namespace:
+    args = argparse.ArgumentParser(description="Convert .fit to .csv")
+
+    args.add_argument(
+        "--dir",
+        help="Path to directory containing .fit files",
+        type=str,
+        default=os.getcwd(),
+    )
+    args.add_argument(
+        "--timezone",
+        help="Timezone for timestamps, e.g. 'US/Pacific'",
+        default="US/Pacific",
+    )
+    args.add_argument(
+        "--overwrite",
+        help="Overwrite any .csv files already converted from .fit",
+        action="store_true",
+    )
+
+    return args.parse_args()
+
+
+def main():
+    args = parse_args()
+
+    # Identify .fit files
+    fit_files = glob.glob(args.dir + "/*.fit")
+
+    for file in fit_files:
+        # Use the same filename, just change extension to .csv
+        base_filename = file.removesuffix(".fit")
+        new_filename = base_filename + ".csv"
+        if not args.overwrite and os.path.exists(new_filename):
+            continue
+
+        print("converting %s" % file)
+        data = collect_data(file, tz=ZoneInfo(args.timezone))
+        write_to_csv(data, new_filename)
+
+    print("finished conversions")
 
 
 if __name__ == "__main__":