← Simplicity.IO

OnDevice AI Meter: Questions and answers

Answers to questions you may have before installing, while a test runs, and when reading your results. Labels in the app link to the matching section here.

Last updated: 2026-11-13 (app 0.3.18).


Before you install

What does the app do?

It measures how well your phone runs AI on the phone itself, without the internet. It times the same AI models on each of your phone's processors (CPU, GPU and the AI processor, called NPU), checks that each one gives correct answers, and sums it up in one score. The details are in the test methodology.

Which phones are supported?

Phones with Android 11 or newer. Shortly after launch, the Play Store offers the app only on phone models we have confirmed to work well. More models are added as results come in, so if your phone can't install it yet, please check again later.

Why Android 11 or newer?

The tests need system features that arrived in Android 11, such as reading how close the phone is to slowing itself down because of heat. We keep the minimum as low as these features allow.

How much space does it need?

The app includes its test photos and sounds and the models for photo, object, sound and image-style tests. The chat model (about 1 GB) and the speech-to-text model (about 40 MB) are separate, optional downloads you choose on the start screen. Without them the result is "partial": a score from the tests that ran, but no level.

Does it need the internet?

No, the tests run entirely on your phone. The internet is used to download the optional chat and speech models and, outside the tests, for ads (see Does the app show ads?).

What data leaves my phone?

None of your personal data. Test photos and questions are built into the app; nothing from your phone's camera, gallery or files is used. Released versions send anonymous usage statistics (for example, how many tests finished on which phone model) so we can see which phones work well. They never include names, accounts, IDs, photos or messages. A problem report is sent only when you tap Send. Crash reports are sent automatically, with the phone model and app version but nothing personal. The ads come from Google AdMob, which may use your phone's advertising ID depending on the choice you make in the consent form (you can change it under "Privacy settings"). Details: the privacy policy.

Does the app show ads?

Yes, the app is free and shows a few ads: one under your result after a test has finished, and a banner on the test history and downloads pages. Ads never load or show while a test runs, so they can't change a result. Where the law requires it, Google's consent form asks for your choices first; you can change them any time under "Privacy settings" in the menu. Details are in the privacy policy.

Is it hard on my battery or phone?

A full test keeps the processors busy for about 12 to 27 minutes, so the phone gets warm and uses some battery, much like a demanding game. The last part, the efficiency test, runs the same photo job at a steady rate for 2 minutes; it is on by default and can be unticked. It's safe: phones protect themselves by slowing down when they get hot, and the app stops the test by itself if the phone gets too hot (see Why does the test stop when my phone is hot?). The app checks the battery and temperature before starting and warns you if the battery is low or the phone is already hot.


While the test runs

How long does a test take?

About 12 to 27 minutes, depending on the phone. Most of the time goes to the chat model, which loads a 1 GB model and answers questions on each processor. Leaving out the chat model or the efficiency test makes it shorter. Faster phones finish sooner.

Should I use the phone during the test?

Please don't. Leave the app open and the phone on a table. Other apps, notifications, video calls or games take processor time and make the results slower and less steady. The app keeps the screen on by itself.

Charging or on battery?

Both work, and the app records which one you used. Results are compared only with other results in the same state, because some phones run faster or slower while charging. "On battery" is the reference for rankings. Since app 0.3.10 the result says "Measured while charging" next to the score, because the reference phone is measured on battery.

Why does the screen stop moving now and then?

While a measurement runs, the app pauses its animations so that drawing the screen doesn't steal time from the test. Results and pictures appear between measurements.

Why does the test stop when my phone is hot?

To protect your phone. Before a run the app shows a heat warning (until you tick "Don't show this again"). Between measurements it checks the phone, and it stops the run by itself if Android reports a severe heat state or the battery is above 45 °C. The results so far are kept, and such a run counts neither for nor against your phone's support. Phones already protect themselves by slowing down when hot, which also makes results slower; and heat wears batteries faster. If the test stopped, let the phone cool down, take it out of any case and out of the sun, and run it again.

What is the efficiency test?

It is on by default (a checkbox on the start screen; you can untick it). It runs last. The phone recognises photos at a steady 30 per second, like a camera, for 1 minute on the CPU and then 1 minute on its fastest accelerator that answers correctly: the AI processor (NPU) or the graphics processor (GPU). The work is the same in both halves. We measure how much CPU time the app needs per photo in each half, using Android's standard CPU time counter. Work done on an accelerator is not CPU time, which is the point. "Efficiency: the AI processor needs 92% less CPU work than the CPU for the same photos" means the CPU spent 92% less time per photo. If no accelerator answers correctly, the app says that nothing takes work off the CPU. We also check that the speed holds under the steady load ("slowed down" if the time per photo grows by more than 10%) and record the phone's heat state and battery temperature. It is shown beside the score, not in it. It measures CPU load, not AI ability or power use. The test can stop early if the phone gets too hot.

What is the chat test?

A small AI chat model (Qwen3 1.7B) runs on your phone without the internet. We measure how fast it reads a question and writes an answer, in tokens per second (a token is roughly a short word or part of a word). It also answers three simple questions, so we can check that each processor runs it correctly. It runs only if you downloaded the model.


Does the chat quiz really tell phones apart? What if no phone gets every question right?

The quiz doesn't rank phones; speed does. The chat score comes from how fast the phone reads a prompt and writes an answer (tokens per second), and that differs a lot between phones: a recent flagship writes several times faster than an older mid-range phone.

Every phone runs the same model with the same settings, so what the model knows is the same everywhere, and it may miss a question on every phone. The quiz checks something else: whether a faster processor, such as the graphics or AI processor, still calculates correctly. Its speed only counts if it answers at least as many questions right as the same phone's CPU. This check has caught graphics processors that gave wrong answers; their speed was left out of the score.

Your score

What does the score mean?

It sums up how fast your phone runs the AI tests, with correct answers. A reference phone, a recent flagship, scores 1000 in each of the six tests on battery, so 6000 in total; a phone that scores 3000 is about half as fast at these tasks.

What are Recognition AI and Generative AI?

The tests come in two groups. Recognition AI is about understanding what is in the input: photo recognition, object detection and sound recognition. Generative AI is about making new content: image style, speech to text and chat. The efficiency test is an extra and is not scored.

Each group's score is the sum of its three test scores, and the AI Capability Score is the sum of the two groups (since app 0.3.18). Before, the app used averages (geometric means) and the reference phone scored 1000; your earlier results are shown recomputed as sums, so you can still compare them. App versions before 0.1.66 weighed the tests differently, so their scores are not comparable.

Why is a test marked "trial"?

A new test first runs as a trial: it is measured and shown, but not counted in your score or level until it has finished with correct answers on all our reference phones. No test is a trial at the moment. Sound recognition, speech to text and image style were trials earlier and are now scored (image style since app 0.3.9).

What is the level?

Five levels group scores into Basic, Capable, Advanced, Flagship and Cutting-edge. If a test couldn't run on your phone, the level is capped, because the phone can't do everything the level stands for. Each level starts at twice the score of the one below: Basic is under 1200, Capable 1200 to 2399, Advanced 2400 to 4799, Flagship 4800 to 9599 and Cutting-edge 9600 or more. A level needs every scored test, so a partial result gets none.

Why does my result have no level?

You left out the chat model or speech to text (you didn't download it), so the result is "partial". The score is then made from the tests that did run (when both groups have at least one), but a level needs every scored test. Download the missing models on the start screen and run again to get a level.

Why is my score different from last time, or from the same phone elsewhere?

Small differences (a few percent) are normal: temperature, charging, background apps and system updates all play a part. For the steadiest result, test on battery, with the phone cool, nothing else running, and the screen on.

A test shows "—" or "couldn't run". Is my phone broken?

No. Some phones can't run every test: for example, the chat model needs about 1.2 GB of free memory. A test that tried and failed counts as the lowest result in the score and caps the level, and the details screen tells you why. A test you left out (not downloaded, efficiency test unticked) is simply not run: see Why does my result have no level?.


What the labels mean

Correct answers

This processor gave the same answers as the CPU (the reference). Only processors with correct answers count in the score.

Wrong answers

This processor gave different answers from the CPU on more than one in ten test photos. It comes either from the phone's driver (the software the phone maker ships for that processor) or from how the AI engine inside the app, Google's LiteRT, handles that processor; our test alone can't tell which. When phones report it, we check with Google's own tools before deciding. We don't add workarounds for individual phones, because that would make results unfair. The processor's time is shown but not counted, and your score uses the processors that answered correctly.

Used in score

The fastest processor with correct answers for this test. That is the time your score is based on.

Full precision (FP32)

The graphics processor first runs the model at half precision (FP16), which is faster. If its answers don't match the CPU's, it runs the same model once more at full precision (FP32) and counts if those answers match. Every phone gets the same rule. Full precision is usually slower, but the answers are correct.

Steady

The time stayed nearly the same across the 5 rounds of measurement (within 10%).

Unsteady on this phone

The time changed a lot between rounds (more than 10%). It is not a failure: the score uses the middle value, which holds even when single rounds jump. The most common cause is the graphics processor lowering its speed in the tiny gaps between photos and raising it again; heat, charging and background apps can add to it. Try again with the phone cool, on battery and with nothing else running. Since app 0.3.10 a lane that comes out unsteady is timed a second time, and the steadier timing counts. If the timing that decides a test's score is still unsteady, that score is shown with "≈" (approximate).

No faster than the CPU

This processor gave correct answers but wasn't at least 3% faster than the CPU. Often the phone quietly ran the work on the CPU anyway, or this processor isn't built for this kind of model. It is shown so the result isn't mistaken for a working accelerator.

Skipped

We don't run this model on this processor, on purpose. For example, compressed (INT8) models give wrong answers on many phones' graphics processors, so they are run only on the CPU and the AI processor.

Not on this phone

Your phone doesn't offer this processor to apps.

Couldn't run

The processor is there but refused the model or didn't finish in time (for example, preparing the model took longer than Android's 60-second limit). Your score uses the processors that did run.

Tests finished

How many of the test items that were run finished with correct answers: the photo, object, sound and image-style models, the speech and chat models and the efficiency test. Tests you left out (not downloaded, efficiency test unticked) are not counted. A phone that finishes more than 60% counts as supported. A run stopped because the phone got too hot counts neither way.


AI processor (NPU) status

Why AI processors differ between phones

Each chip maker decides how apps can reach its AI processor (NPU), so it can be easy, hard or impossible for an app to use. The app measures only what an ordinary app can reach, the same way on every phone, and never estimates a processor it can't use:

Google is retiring NNAPI, so phones that depend on it may lose AI processor access in a later update. This is also why the gaps in the phone rankings can be large: they reflect both the chip and how open its AI processor is to apps.

Working well

Your phone's AI processor ran the test with correct answers and was faster than the CPU.

Results don't match

The AI processor ran, but its answers differed from the CPU's, so its results aren't counted. It comes from the phone's driver or from how the AI engine in the app handles this processor; a system or app update may fix it.

No Android driver

The chip has an AI processor, but the phone doesn't give apps a standard Android way to use it. Many recent Snapdragon phones work this way; on those, the app uses the chip maker's own libraries where it can.

Locked by phone maker

The AI processor exists, but the phone maker allows only selected apps to use it.

Not supported yet

The app doesn't support this chip's AI processor yet. We add chips as we can test them.

No NPU

The chip has no separate AI processor. AI runs on the CPU and GPU, which is common on entry-level phones.

"via NNAPI"

NNAPI is Android's standard way for apps to reach AI processors. On Google Pixel phones it is how apps reach the Tensor chip's TPU, Google's AI processor. Android decides which unit runs the work, so the app reports it as "via NNAPI".


Problems and reports

What happens when I tap "Report a problem"?

The app sends a short technical report: phone model, chip, graphics driver, Android and app version, and which tests failed. Nothing personal is included, and it is sent only when you tap Send, at most once per phone and app version. Reports help us decide which phone models to support and which problems to fix.

The test stopped or the app closed. What now?

If the app itself stopped the test, the phone was too hot: see Why does the test stop when my phone is hot?. Some phones close apps that run long tasks. Keep the screen on and the app in front, then start again. If it keeps happening, please report it.

My phone can't install the app. Why?

Shortly after launch we offer the app only on phone models we have confirmed to work well, so everyone gets reliable results. More models are added over time.