Should I test that?

Should I test on low-end devices?

Verdict

Yes

Test minimally on low-end devices: before each release, run one automated test of the main flow on one real Android phone with 3 GB of RAM or less, and stop the release when the app crashes, stops responding, or starts slower than a set limit.

Why

Test minimally on low-end devices: one automated check of the main flow on one cheap phone before each release. My typical case is a consumer Android app, released monthly, that the team builds on recent phones. Blast radius is users, who get slow starts and frozen screens. Change frequency is regularly, since each release can add a heavier screen or library. Detectability is eventually: a slow start on a phone nobody on the team owns raises no alert, and users uninstall without writing to support. Reversibility is costly, because the fix needs a new store release. Test cost is heavy, because the test needs a real device and startup times vary between runs, so rule R12 gives Test minimally.

When the decision changes
WhenDecisionWhy
The main flows already have an instrumented suite that CI can run on one low-end physical phoneTest: run the whole suite on that phone for every release buildTest cost falls to moderate: one more device reuses the existing tests, and a crash or an ANR fails the run the same way each time
Buyers pay in the app, and a checkout screen that freezes on a 2 GB phone loses the orderTest mandatory: run the checkout test on a low-end phone before every releaseBlast radius rises to money: a sale lost to a frozen screen does not come back
A remote flag can switch the heavy feature off for low-memory phones without a store releaseTest it differently: alert on ANR rate and slow starts per device model, and switch the flag off where they riseReversibility falls to with-effort: the remote flag switches the feature off within minutes
The product is a website that visitors open in phone browsersTest it differently: record load time and input delay from real visitors by device memory, and alert when low-memory visits slow downReversibility falls to trivial: a redeploy fixes the page for every visitor, and a slow page stores nothing
A staff app runs only on the one phone model the company issuesDo not test on other low-end devices: try each build on the issued modelBlast radius falls to internal, Detectability to same-day and Reversibility to with-effort: staff report a frozen screen that day, and device management installs the fix without store review

What breaks if you don't test

On a 2 GB phone, Android kills background apps early. A user fills in a long form, taps Add photo, and the camera pushes your app out of memory, so the form comes back empty. The team's 8 GB phones never show it. Freezes also cost store reach: above an 8% user-perceived ANR rate on one phone model over 28 days, Play may reduce the app's visibility and warn users on the listing.

What you lose if you over-test

Cheap Android phones come from many makers with different chips. Firebase Test Lab charges $5 per physical device hour beyond 30 free minutes a day, so a 15-minute suite on ten phones for 20 pull requests a day costs about $250 a day. Strict time limits fail at random on slow phones, and the team reruns until green.

How to test

  1. Pick one phone: in Android vitals by device model, take the most used model among those with the least RAM.
  2. Keep it on a CI runner, or book it in Firebase Test Lab, whose free plan allows 5 physical device runs a day.
  3. On each release build, run the main-flow instrumented test there; a crash or an ANR fails it.
  4. Measure cold start with am start -S -W from adb, take the median of several runs, and fail above a limit well below the 5 seconds that Android vitals counts as an excessive cold start.
  5. Roll out to a small share of users first and read vitals per device model.

When the answer changes

  • Money moves in the flow that can freeze.
  • The product is a website, so a redeploy fixes every visitor.
  • The app runs only on phones the company issues.

Real incident + Code example

The video preload that froze cheap phones

On a recipe app I worked on, one release preloaded three step videos when the recipe screen opened. On the team's 8 GB phones the screen opened in under a second; on 2 GB phones it froze into the "App isn't responding" dialog. A week later Android vitals flagged the two most common cheap models, and one-star reviews said "freezes on every recipe". The fix, loading a video on tap, needed a new store release. We then plugged a used 2 GB phone into CI and ran this on every release build:

#!/usr/bin/env bash
# Median cold start on the low-end phone; fails the release above the limit.
set -euo pipefail
ACTIVITY=com.example.recipes/.MainActivity
LIMIT_MS=2500   # our median on this phone before the video release, plus a margin
times=()
for i in $(seq 9); do
  # -S force-stops the app first, so every run is a cold start
  ms=$(adb shell am start -S -W -n "$ACTIVITY" | tr -d '\r' | awk '/TotalTime/ {print $2}')
  times+=("$ms")
  sleep 3
done
median=$(printf '%s\n' "${times[@]}" | sort -n | sed -n 5p)
echo "median cold start: ${median} ms, limit ${LIMIT_MS} ms"
if [ "$median" -gt "$LIMIT_MS" ]; then
  echo "cold start too slow on the low-end phone"
  exit 1
fi

FAQ

Which low-end Android phone should I test my app on?

Test on the model with the most of your users among those with the least RAM, which Android vitals shows by device model. With no users yet, buy a current Android Go phone: Android 13 Go edition runs on 2 GB of RAM.

Can the Android emulator simulate a low-end device?

The Android emulator can set a small RAM size, which shows memory crashes, but it runs on your laptop's processor and hides slow starts. Measure startup and freezes on a real low-end phone.

Should I test my website on low-end phones?

Watch a website on low-end phones instead of testing on them: record load time and input delay from real visitors, and alert when low-memory visits slow down. Safari and Firefox do not report navigator.deviceMemory, so the grouping covers Chromium browsers only.