The crashes Crashlytics didn't show: it only sees the builds you added it to
Three days after adding Firebase Crashlytics to 21 apps, we found a user-facing crash in Play Console first. Crashlytics only reports from builds that include the SDK, and crashes from older versions still on users' phones pile up only in Play vitals. Results from checking all 21 apps, and the order we check in now.
The crash emails were quiet
Starting with builds made on September 18, we added Firebase Crashlytics to the 21 apps we run. Crashes send an email, and the five that came in on day one were fixed that day. After that the emails mostly stopped, and we took it to mean there weren't many crashes.
On September 21, a crash in our markdown viewer app turned up in Play Console. Four users had the app close with an IOException while creating a file, on version 1.0.22 and earlier. Crashlytics had nothing.
Why it wasn't visible
Crashlytics only reports crashes from builds that include its SDK. Obvious, but in practice it matters a lot.
- Shipping a new version doesn't update everyone. Phones with auto-update off, or phones that haven't been opened in a while, keep running a version from weeks ago.
- Those old versions don't have Crashlytics, so their crashes never reach Firebase.
- Google Play collects crashes from the device regardless of any SDK. They pile up per version under Play Console → Android vitals → Crashes and ANRs.
So right after adding Crashlytics, quiet email doesn't mean "no crashes". It means "not many people are on the new version yet".
All 21 apps: 8 had crashes
On September 22 we opened vitals for all 21 apps in Play Console, one by one. Period: the last 28 days, with the default "user-perceived" filter removed. With that filter on, you only see crashes classified as ones the user directly experienced.
- No results: 13 apps
- Crashes: 8 apps
The biggest one was shared by three of our TV remote apps.
java.lang.SecurityException: Need android.permission.BLUETOOTH_CONNECT permission
at ...ble.HidPeripheral.start
Starting Bluetooth remote mode without the Bluetooth connect permission granted closed the app. In the main clusters that was 25 users across the three apps (10, 9 and 6), on every version from 1.2.2 up to 1.2.10, which was live at the time. Only one of those, on 1.2.10, had arrived as a Crashlytics email.
Other apps looked similar.
- Our tick photo app: about 20 users hit a native crash (
SIGSEGV,TfLiteXNNPackDelegateDelete) when the TensorFlow Lite interpreter was closed. It only appeared on 1.1.12 and 1.1.13. On September 6 we put inference (run) and release (close) behind the same lock so the interpreter is never closed mid-inference, and no later version shows it. The bug was fixed, but it kept happening on phones still running the old versions. - Our QR scanner app: a native crash in document mode (
SIGABRT, onnxruntime) was still showing on 1.1.14. It had been fixed in 1.1.16.
What we fixed
For the remote apps (four apps sharing the same code), version 1.2.12 on September 22 closed three paths.
- At the entry point that starts Bluetooth mode (
startHid), if the permission is missing, don't start; request the permission instead. - Inside
HidPeripheral.startandClassicHid.start, check the permission again. If some path skips the entry point, it returns instead of crashing. - On the pairing guide screen, check
BLUETOOTH_ADVERTISEbefore starting Bluetooth advertising. Vitals had one separate crash from this path.
The order we check in now
- Same-day crashes: Crashlytics. Email and console are close to real time.
- Everything live: Play vitals. Remove the "user-perceived" filter and look at the version column. If a fixed crash only shows on old versions, it fades as people update. If it shows on the current version, it needs a new fix.
- We also read vitals through the API. Play Developer Reporting API
errorReports:searchreturns the full stack trace (reportText). Vitals data arrives about a day late, so today's crashes aren't there yet. - If the weekly report shows a non-zero crash count, open the cluster right there. Per-app crash counts were already in our weekly report before all this. Copying the numbers without opening them was the real cause.
Takeaways
- Crashlytics only sees builds it was added to. Quiet email right after adding it is normal.
- Old versions stay on users' phones for weeks. Their crashes are only in Play vitals.
- In vitals, clear the default filter and check the version column. That's where "already fixed but still happening" splits from "new bug in the current version".
- A number in a report isn't a check. If it isn't zero, open it.
One of the apps in this post is Genie TV Remote (Google Play).