How to track the non reproducible bug in version 2.1

So, version 2.1 was crashing for quite a few users on startup. Obviously the app wasn’t crashing for me in any of my tests.

First line of defence when crashes happens is to ask the user what they were doing and try to replicate the same action to see if I can recreate the crash. But here they weren’t doing anything it wasn’t starting.

Second is to look for patterns in the crash. The first few users reporting the issue were all using an iPad. There are a few differences between the iPhone and the iPad, mainly that the iPad displays an activity detail at the same time as the activity list. This can result in some difference in logic and makes the startup on iPad marginally more prone to issues. This can lead to hard to reproduce bugs too as it is a function of the type of activity the user would load on startup (the last one downloaded)

Another help is the crash report from apple. In this case, they were very few and reporting crash in location that are not triggered at start up or inside apple libraries. So not helpful to pinpoint the problem.

Some users affected by the issue kindly sent me their activity data so I could try to see if the problem was linked to their specific activities data. It wasn’t, I could start the app with their data fine.

This leaves one other possibility, a thread concurrency issue. This can be really random, and a function of both the speed of the hardware and the exact activities you have. ConnectStats is quite multi-threaded to try to keep the app responsive as many calculations are performed in parallel. So it is possible that some of this parallel activity result in a collision or some instructions run too early (when required data is not ready).

I try to be very careful on collision conditions and to make the code robust to data not being ready. In version 2.1, in order to optimise the startup, I had done a clean up of all the notifications between events in the app to limit them to the minimum needed. So likely the problem was linked to that: issue affecting random users, not a logic issue (users data didn’t let me reproduce it), but the issue was systematic for some users: so this gives another hint. The one issue that can exhibit these symptoms is if you trigger a User Interface event not from the thread dedicated to the UI (main thread).

I did a review of all the events that happen on start up and found one that, in the clean up of 2.1, wasn’t not forcefully directed to that main thread…

So while I couldn’t reproduce, I feel strongly that this must be the issue:

  • linked to some change I made in version 2.1,
  • random: depends of when precisely event get triggered
  • systematic for a given user: for a set of hardware speed and size of data, always happens or not

I submitted a fix for that event issue to apple, let’s cross fingers until version 2.1.1 is approved by apple and hope the issue is indeed resolved. I requested an expedited review, but it’s not always granted.

Frustrations of a botched release

I try to release ConnectStats regularly. Once in a while, as today, I push an update with an issue. I have been receiving a lot of bug reports today from users who can’t start the app after last night release.

I know it’s quite frustrating for users, and it is for me as well. In this case so far it has been impossible for me to reproduce the issue.

So as I try to sort it out, I decided to write a bit about the testing and debugging of ConnectStats

Before the problem happens

Automatted Testing

First I wrote a bunch of tests that verify the very basic functionality of the app at the lowest level. For example it checks that all the unit conversions continue to be correct, that the statistics computation continue as expected, etc, etc

I have also built a testing sister app to ConnectStats that share most of the code, but run a bunch of test replicating the download process, or statistics on saved samples of activities.

Device Testing

While the automated testing is critical, most of the issues, like the one today I suspect are due to subtle interaction between different aspect of the app, when run with different set of activities or different devices.

Given I develop ConnectStats alone in my free time, this kind of testing is fairly limited for me. I mostly test with my phone and my activities, as well as a secondary device. For example I do not use an iPad, so issues on iPad are more likely to appear as I don’t have as much opportunity to try these.

Once a problem happens

Bug Reports

Typically, the first thing that happens is I’ll start receiving some bug report. The bug report contains workflow information and diagnostic collected as the app executes. It’s only as good as what I anticipated to record. But quite a few bug can be understood this way, especially when people email me and answer a few questions about what they were doing or how to reproduce the problem.

Apple Crash Reports

When as today, a large amount of people email me that the app crash on startup it’s much harder, as the log does not contain any useful information. I can also see some crash report collected by apple, but they take a while to appear. So right now I just need to wait and hope that the information in apple’s report will help me.

Bad Reviews

Of course the next thing that happens, is that ConnectStats will start receiving bad reviews. It’s basically a race, fix it before too many bad reviews accumulate…

 

Year to date or Month to date statistics

Version 2.1 of ConnectStats introduces the ability to see year to date or month to date statistics. The yearly cumulative graph was already giving a good idea of how well you were doing this year versus the last, but now you can see the full stats you reached at the same point of last year.

Below for example you can see that in 2015, so far I ran 758km, while on the full year of 2014 I ran 1356km. If you press on the button All, you get to the YTD screen, that shows that in 2014 at the same point in the year as today (August), I had run 913km.

IMG_1664 IMG_1665

In the monthly view, the graph is slightly different, the full month bar is shown, but the blue shows how much for each month was reached at the same point in the month, here around august 15, or about half way through the month.

 

IMG_1666 IMG_1667

HealthStats, ConnectStats new sister app…

HealthAppLaunchJust released a new app HealthStats. It has the same core as ConnectStats, but it’s geared towards analysing data in the Health app of your iPhone.

It’s aimed at data collected either via the motion coprocessor (M7 or M8) of the latest iPhone, or collected via the Apple Watch.

If you have an Apple Watch and collects workouts, they should also appear in HealthStats.

My wife just got an Apple Watch two months ago, so I could only do limited testing, but on my initial tries the data collected by the watch is pretty limited. But with HealthStats, you will have ability to dig into that data much more than you can with the native Health App from iOS.

As with ConnectStats, it’s all about data, so the look may not be fantastic… But I intend to add as many ways as I can think of to look at the details of the data.

 

Best Rolling Graphs over Time

ConnectStats can now maintain best rolling graph over time. I find it quite interesting to see how one month compare to the other. You access the feature by going to the statistics screen and it’s the graph at the bottom of the summary page.

You can also find information about best rolling plots for an activity here and how to interpret them here

Initial Computation

Computing these graphs can be quite expensive. Every time you do a download, ConnectStats will process a few activities. It requires the full activity details, so only the activities for which you will have looked at the detail screen will be used.

You can force to recalculate older activities by going to the settings, profiles, and select Compute Best for more old Activities. This processes quite a few existing activities. Each time to press it, it will look for activities not already processed.

Interpreting the graphs

Heart Rate

This is the graph showing the best Heart Rate I achieved for any given period of time.

iOS_Simulator_Screen_Shot_8_Aug_2015_09_34_03The arrow 1 below shows the graph for July. You can see here that it was a month for which the best of the year was achieved except for the area pointed out by the arrow 2. The arrow 3 points to the yellow graph showing that the last year (2014) was not as good as this year, I definitely pushed myself more this year, especially in july, when I completed a half marathon…

If you tap once on the graph it will rotate through the last few months, to show you how you performed then.

 

 

Pace or speed

This is the graph with the best speed or pace I achieve for a given distance. Not the heart rate is for time, not distance as I think it makes more sense.

iOS_Simulator_Screen_Shot_8_Aug_2015_09_34_15If you pan to the left on the graph it will switch to the best pace or speed. Arrow 1 here shows you that the best speed for the year on most distances was achieved in july. Note that for pace a graph lower is better, but for speed higher is better. Again, if you tap on the graph, it will rotate through the last few months.

 

 

 

ConnectStats version 2.0

ConnectStats 2.0 brings quite a few exciting new features. The major change is that now ConnectStats will start maintaining some measures over time, while until now it really only acted as a cache for the data collected on either Strava or Garmin.

This opens the door to a lot of new features. So I upgraded the version and I celebrated by redesigning the icon…

BestHRThe first data I was really interested in looking at over time, was how the best rolling heart rate graph compared from month to month and year to year. Similar to the best pace. These graphs are not necessarily very common to see,  but I find they are a fascinating way to see how I am performing on a run or over the month. I will dedicate a post soon to these. Some snapshots here

BestPace

CalendarAllThe other feature is an improved calendar view. It now has more control to only show some activities, and a new visual circle to indicate how much each day fared compared to the maximum for the month. The best day is highlighted with by a darker color in the circle.

 

CalendarRun

App Video Preview for ConnectStats

I have build an app preview to showcase the screens in ConnectStats. Hopefully it gives a good overview of the variety of screens and stats available. This was captured with version 2.0 of ConnectStats so older versions won’t have necessary all the screens. I had to compress as many views as possible in the maximum 30 seconds allowed…

 

Next version of ConnectStats

I have pushed ConnectStats 1.23 to the App Store.

Connectivity upgrades

ConnectStats 1.23 uses withings latest authorisation service. Hopefully it will be more robust going forward. The login relies on a webpage login and a page where you need to explicitly grant access to ConnectStats.

The workflow around Strava log in has also been improved. There is now a button in the Services page to force and test a login to Strava. Also a silly bug that was sometimes complaining about Garmin when trying to access Strava has been fixed.

ConnectStats 1.23 also fixes an issue with connectivity to SportStracks that had stopped working in the previous versions.

Multi-Sport Activities

Garmin introduced multi-sport activities in the FR920 and Fenix 3 at least. Starting from version 1.23, ConnectStats will detect these activities and download the individual sub-activities. It will display as a stub the main multi-sport activity, but note that only the sub activities will be included in the statistics.

Better weather information

ConnectStats 1.23 has an upgraded weather information displayed. It displays the location where the weather was observed and the distance from the location of the run. On the map it also displays a little wind compass indicated the direction the wind was blowing and the rough strength. The length of the arrow is proportional to the wind strength.

Note that the weather information is only available when using Garmin as a service source.

The Apps and Services I use

ConnectStats was really built out of a need I had and over the years, while the landscape of services and apps has changed quite a bit, I continue to use it and develop it. So here is my view on the services and apps I use beside ConnectStats and why I feel ConnectStats is still useful.

Of course a big disclaimer is that my views are biased given I wrote ConnectStats. But at the same time, my goal is not to replace or take out any competing app or service, but more to complement with what I feel is missing, so hopefully, it remains an interesting perspective.

There are mainly 3 apps or services I use: Garmin Connect Mobile on the phone, Strava on the phone and on the web and ConnectStats. I focus here on the iOS app, not the web sites.

All my activity data collection is done with Garmin Devices. I use a Garmin Edge 510 for my cycling, and currently a Fenix 3 for running, and other occasional activities like hiking, skiing, etc. I had a Garmin Forerunner 620 previously and before that a Garmin Forerunner 610.

I will go through the following:

  • Syncing Activities
  • Casual Review of Activities
  • In-depth analysis of an activity
  • Statistics over time
  • Social Features
  • A few other apps and services to note
  • Pricing

Syncing Activities

From the device

Once an activity is recorded, I transfer it to the cloud either using the Garmin iOS app, the device wifi ability (fenix or forerunner 620) or the computer and Garmin Express.

I really wish the Garmin iOS App and bluetooth connection was more robust, it would be my preferred transfer method, but unfortunately it’s quite flaky. Great when it work, but regularly it just can’t connect, turning on/off bluetooth is usually not helping. On the day it works it’s great, on the day it doesn’t quite frustrating, and if I am at home I have to switch to garmin express on the mac and connect the device via cable.

There are no alternative to the Garmin tool for syncing, as they don’t publish any public API for that, so we have to stick to their tools.

Between Services

The situation has much improved now. If you set up connection between Garmin, Strava, TrainingPeak, SportTracks, all the activity get sync seamlessly. I built in the past a feature to upload from ConnectStats to Strava, but this is now really not necessary and I stop using the feature or developing it further, I recommend just using the direct link. I had a few users argue that ConnectStats feature was still useful because it allowed you to upload the activity after you did some edit like activity type or activity name, but I feel it’s a niche benefit.

ConnectStats caches locally all the data so once downloaded you can review at will all the activities and statistics without an internet connection. From my point of view it also makes the app quicker to use and navigate as everything is calculated locally. Of course a side effect is that it’s harder to do large data analysis like Strava does with segments or comparing runs automatically.

Casual Review of your activities

Garmin Connect Mobile

Most of the apps will let you see the basic data of an activity. For my taste, the Garmin iOS app probably fares the worst both in terms of look and data. You really only have basic data here as well as simple laps statistics. Also the look is extremely basic and not like a modern iOS app. The graphs are quite slow to come up as it seems to do a internet call. This is the case for all data, so if you are without internet connection, you won’t be able to review any activities, even one that you looked at in the past in the app.

IMG 1451
IMG 1452

Strava

Strava is the app that has the best look by far. I prefer it to how ConnectStats looks, Strava is much slicker. I only wish I had the time and the artistic skills to make ConnectStats look as good. It has most of the basic information handy and I particularly like the comparison to previous similar run. I like the Heart Rate Zone Analysis and pace analysis, though you need to subscribe to their premium service.

IMG 1453
IMG 1454

ConnectStats

ConnectStats also has a fairly basic look, I have little time and skills to make it look pretty, but I focused on having the most possible useful information accessible on one screen. One of the benefit of the Garmin Connect data versus Strava is that it contains a lot more information, like all the running dynamics, weather, maximum, calories, etc, etc. I wanted to have all that information displayed and an overall map and graph visible at once in one screen that you can scroll. One side effect is that to focus on any information makes it harder to design the screen to look good. The strava screens look much better but their are more specialised for pace, heart rate etc, therefore less generic information.

IMG 1457

IMG 1458

Activity Plots

This is something that always bothered me with most apps and services. The plots of pace especially, but sometimes heart rate as well seem to be scaled and smoothed so poorly that they didn’t allow you to see really what happened. This was one of the key initial motivation for writing ConnectStats (along with more information on the map). I spend a lot of time working on the scaling, filtering and smoothing of the graphs in connectstats so that it gives you a good picture of what happened at a glance. Recently I also added overlay of the laps average which I feel is quite helpful.

Here are the pace plots for the same activity in Strava, Garmin Connect and ConnectStats. I doubt anyone would be able to draw the conclusion I sped up a bit in the middle of this run, only to clearly slowdown toward the end, from either Strava or Garmin Connect’s graph… Note that the lap histogram on strava shows you that story as well, so one point for strava, but the plot is quite useless.

IMG 1461
IMG 1460
IMG 1459

In-depth analysis of an activity

This was really one of the driver behind building ConnectStats so this remains the app I favour by far for this. Of course, likely biased by the fact I put there all the analysis I wanted, that others didn’t provide.

Plots

Garmin Connect Mobile will simply show you the basics plots and they are not very pretty and slow as they get pulled from the internet. Strava, if you are a premium customer, will show you in addition time in zone and pace zone analysis in pretty looking fashion.

ConnectStats shows a lot of extra plots. Time in zone is not as nice looking as the equivalent in strava. But there all the relevant fields can be seen on a plot, including run dynamics. And two types of plots just don’t exist in the two other apps and quite useful to me.

Scatter plot to see how two variables relate together, how steep is the regression between heart rate and pace, or cadence and power. You can also compare the graph of two activities.

Best rolling plots are also unique to ConnectStats and somehow addictive for me to look at.

Maps

The maps in Garmin Connect Mobile are quite basic. I do not like the very thick lines to show the route personally. Strava’s map look better, but also simple in feature.

ConnectStats shows a lot more flexibility in what it shows on the map. It use a color coding to show values of different variables like heart rate, pace, etc at each point on the map.

ConnectStats can also display laps as recorded or calculated ones like fastest kilometre (or mile), breakdown in kilometres, etc.

Finally ConnectStats lets you use either apple maps or google maps included terrain’s view, which can be handy when off road.

Auto Laps and Segments

Auto laps is a feature that also make me use ConnectStats over the other two. This features lets you calculate arbitrary laps after the fact. You can calculate the split time this way or the split distance. But also the lap corresponding to each heart rate zone, laps breaking the activity in fixed distances or best rolling laps (where did you run your faster 50m, 100m, 200m, etc, etc). For each of these calculated laps, you can then see it on the maps or see the relevant statistics as numbers or plots (like a recorded lap)

For segments Strava is the best app. ConnectStats doesn’t support them at all and Garmin Connect Mobile has them, but they are nowhere as functional as Strava’s.

Garmin Connect has a separate page showing you were you rank and very basic stats like time and speed for your run.

Strava’s segment view has both ability to compare yourself to other, but also analyse your performance in details, overtime and for a given effort. It shows a plot overtime of the speed you reached on a given segment along with the list of days you ran it. You can also analyse the effort on that segment and compare it to your PR using nice looking plots.

   
 

Statistics over time

Garmin Connect Mobile here has no such feature. Or if it has, they are well hidden and I could not find as much as a current week or month accumulated distance.

Strava and ConnectStats both go much further but in a different direction. ConnectStats focuses on statistics and more advanced analysis of the data, while Strava focuses on more basic data but presenting it better, and matching equivalent data over time.

Strava will present you the week and daily distance in great looking graphs and elegantly shows your progress toward a weekly goal. You can also see a few basics statistics over the year, like Avg Runs/Week, total year to date distance, etc. For a given activity, I really like the matched run feature which at a glance lets you see the progress when you run an activity over and over.

ConnectStats will show you many more graph, but the data is presented to you as is. It’s not put into context like strava and compared to a goal or against matched run or segments. That said the graphs and data you can extra from connectstats is quite wide. Because it’s less about context, ConnectStats lets you explore any data in the same fashion. All recorded data can be analysed. How did your Max HR evolve over time? your Avg Cadence? Any of the run dynamics? How does speed get affected by avg temperature? Or your weight?

ConnectStats also let you plot many type of graphs. You can see scatter plots of any measure vs another, weekly, monthly summary of basics stats but also quartiles, medians, etc.

There are a few unique plots which I constantly refer to as well. I feel the yearly comparison of the total distance ran or biked quite motivating to go beyond the previous year. The long term vs short term analysis is also a good indicator of your current form vs effort.

ConnectStats also has a calendar view that lets you have a good overview with color coding or basic statistics of how you did in a given month.

Social Features

ConnectStats has no social features, and this was the key reason I initially used Strava that has in my opinion the best social features.

The feed is neatly shows your and your friend’s activities. It’s extremely easy to add a quick thumb up or comment. And it also let you look in detail at your friends activity and of course compare yourself to them on given segment. Also automatically detects when you ran with a friend.

Garmin Connect Mobile has a news feed also, but it does not look as nice and unlike Strava where things are intuitive and simple (one feed and a filter), Garmin Connect Mobile has a news feed, and an activities within a fitness menu, a social menu. All very convoluted for a less simple and pleasant experience as Strava, not to mention the overall look of the app.

Conclusion

Strava and ConnectStats are my two favourite apps to analyse my data. Strava is much better looking and has great social and segment matching features, but ConnectStats continues to enable me to look with much more depths into the numbers. So for now I’ll continue to use and develop ConnectStats…

Working with iOS Simulator files

Simulator Data Finder greatly improves my workflow in working and debugging apps in Xcode 6. You can download it here. My apps relies on many downloaded json and xml files, as well as keeping all its data as sqlite files. I use Simulator Data Finder to easily access these files, copy them from one simulator to another or bring into the simulator the files from a device.

Accessing the files

The first use is to access the files, either by the finder or terminal. The terminal is actually quite handy to work with sqlite files and sqlite3 directly, use grep or any kind of other shell tools for old school developers… Simulator Data finder has a clipboard button so when terminal’s tool are not enough, it’s easy to paste the directory and access it from emacs for example.

OneSimulator

Copying file between iOS Simulator versions

Another typical use is when you have a lot of files and you want to keep the same setup into a new iOS Simulator version. For example you have everything setup on the iPhone 6 simulator, but want to see how it looks on iPhone 5. Simulator Data Finder makes that a breeze. You can easily choose the two simulators from the table, open the two Documents folder with the finder and copy the files over.

This makes it very easy when a new version of iOS or Xcode comes out to move all the files you need. The older version will be easily accessible and clearly labeled.

If you work across two device simulators, the modified date also lets you see at a glance which of the two simulator has the latest modified files.

BetweenSimulator

Copying file from device to the simulator

The other task that Simulator Data Finder makes easy is to move around of files between the device container and the simulators.

When a developer device is connected, download the application container to your download folder using the standard name. Simulator Data Finder will then present all the container it finds in the Download folder that match the app you have currently selected. If you pop up the finder for the container and for the simulator it becomes very easy to copy files. The containers will also be conveniently sorted with the most recent on the top of the table.

DeviceAppData