Building Flutter End-to-End Test for QR Code Generation and Scan Apps

Introduction

In this article, we will build an end-to-end test that will simultaneously test two flutter apps (a QR code generator and a QR code scanner/decoder), running on two different emulators, using flutter_driver.

In other words, we won’t go in-depth with how the apps are made but rather the complexity of testing them with flutter_driver.

The test will consist of:

  • Generating a QR code using the first app and taking a screenshot of that code
  • Pushing the image from the first to the second emulator using ADB tools
  • Scanning the QR code using the second app

Here you can find the code for the QR code generator and the QR code decoder.

But first, I would like to address some issues that I encountered through developing this solution:

  • How to test an already running app?
  • How to simultaneously test two apps?
  • How to mock the image picker?
  • How to transfer a screenshot from an emulator to another?

Note: Since we will be using ADB tools, we are doing our end-to-end test exclusively on android emulators.

Problems you may encounter

Test an already running app with flutter_driver

If you read the flutter driver documentation. You will be first taught to run a test with the following command:

flutter drive --target=test_driver/app.dart

In their example, you will also have a folder named test_driver in the application folders, and in that folder, you will have app.dart (which will run the app and enable the flutter driver extension) and the app_test.dart (which contains the test).

So this command will:

This method of running end to end test tests is good if you have one app and one test, but it has its issues:

  • Every time you want to run the test, you need to rebuild (there’s a — use-application-binary option) and reinstall the app on the device which takes a lot of time and slows development.
  • If you have multiple test files that need to run one after the other, you can’t use the same app for all the test files.
  • Every time you will run a test, the app's state will be reinitialized, so you can’t assume that the app has reached some starting state before the test.
  • The test file has to be named accordingly to the file running the app, which means for each test file, we need a respective main file, and since those files look alike, it will pollute the app’s files with redundancy.

The best solution that came up during the development of my solution was running the app and the test file separately, and connecting the test file to the needed app through the app’s URL.

To run the app, we run the file that contains the enabling of the flutter driver extension and runs the app:

flutter run test_driver/app.dart

And then, we catch the URL relative to the app in the output of the command:

An Observatory debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:56335/VBrQsPOWCg0=/

The above is the line from the output containing the URL.

Next, some modifications need to be made to the test file so that the driver connects to that URL. A way to do that was to send the URL as an argument for the command running the test and then using that argument with the FlutterDriver.connect method. So a file that looked like this:

Will become more like this:

And to launch the test, we should use the following command:

dart test_driver/app_test.dart <app_url>

Ex:

dart test_driver/app_test.dart http://127.0.0.1:56335/VBrQsPOWCg0=/

Simultaneously test two apps:

Since we now know how to select an already running app to run a test on, the task of running two tests simultaneously seems more plausible.

The only thing we have to do is run the apps on different emulators with:

flutter run test_driver/app.dart

To select a specific emulator we can use:

flutter run test_driver/app.dart -d <device_id>

But it didn’t work for me, so either way, when having multiple emulators available, you’ll be asked to pick one, the output looks like this:

Multiple devices found:
Android SDK built for x86 (mobile) • emulator-5554 • android-x86 • Android 10 (API 29) (emulator)
Android SDK built for x86 (mobile) • emulator-5556 • android-x86 • Android 10 (API 29) (emulator)
[1]: Android SDK built for x86 (emulator-5554)
[2]: Android SDK built for x86 (emulator-5556)
Please choose one (To quit, press “q/Q”):

After picking a device, get the apps’ URLs from the outputs of each flutter run (as we previously did), and use it to run the tests of your choice on the apps of your choice.

Chosen app’s flutter run output:

An Observatory debugger and profiler on Android SDK built for x86 is available at: <chosen_app_url>

Runnin chosen test on the chosen app:

dart <chosen_test_path> <chosen_app_url>

If you have a specific scenario of tests in mind, you can group your commands in a script. It can be any kind of script, even a Dart script.

An example is available later in the Wrap it up! part.

Mocking the image picker

In some cases (like the one we are about to dive into), you might need to have a preselected image for your tests or at least skip the part where you need to click in the gallery to find an image during your end-to-end test (since flutter_driver can only interact with your app’s elements and not the gallery/image picker).

To find a solution, I was mainly inspired by the following StackOverflow post but the solution I came up with was much simpler. You can adapt it as much as you need, for my part, I already knew the path (on the emulator) for the image I needed, so I just returned the path to that image.

I obtained the following test_driver/app.dart file:

So when running the test_driver/app.dart (instead of lib/main.dart) any call for the image_picker will instead call the mocked function and, in this case, return the preselected path (no picker will show).

This behavior is only present in the testing apk, which is good in the way that it doesn’t pollute the actual application code.

Screenshot capture and transfer

The screenshot transfer will be done in two big steps:

Requirements

From now on, we assume that you have the following tools are ready to use on your machine:

About the Apps

QR code generator:

QR code generator app screenshots
QR code generator app

This app uses the following dependencies:

dependencies:
...
qr_flutter: ^4.0.0
...
dev_dependencies:
test: any
flutter_test:
sdk: flutter
flutter_driver:
sdk: flutter
...

Note: qr_flutter will be used to generate QR codes.

And here’s the lib/main.dart:

QR code decoder:

QR code decoder app screenshots
QR code decoder app screenshots
QR code decoder app

This app uses the following dependencies:

dependencies:
...
qr_code_tools: 0.0.7
image_picker: 0.8.0
...
dev_dependencies:
test: any
flutter_test:
sdk: flutter
flutter_driver:
sdk: flutter
...

Note: qr_code_tools will be used to scan QR codes from an image (not from the camera) and image_picker will allow the user to pick an image from the gallery.

And here’s the lib/main.dart

Let’s test!

And now, the moment you’ve all been waiting for! It’s time to start testing. If you don’t have a lot of experience with flutter_driver it’s not a problem (actually we will be on the same level). What you need to know is that we can use it to do some actions on widgets (ex: tap) or retrieve some information from it (ex: text).

To find our widgets, we’re going to be assisted by “finders”. In our case, we will see two types of finders: finders by types and finders by keys.

All the files related to testing (it doesn’t matter if it’s in the first or the second app) will be in a folder named test_drive. With the commands used here, the name of the folder is not mandatory. All the test codes can also be found in the GitHub projects shared above.

Note: The apps folders are stored in the same folder in our example.

QR generation test

We will start by creating the test_driver/app.dart file that will run the app while enabling the flutter_driver extension.

Therefore, to run the app for our tests, we use the following command:

flutter run test_driver/app.dart

If you have multiple emulators available, a list of emulators will appear and you’ll be asked to choose one of them. (We will see this case later on)

In the output of the command, we will need to get the app’s port and name. It will be needed to connect the test to the app later on. It looks something like that:

Multiple devices found:
Android SDK built for x86 (mobile) • emulator-5554 • android-x86 • Android 10 (API 29) (emulator)
Android SDK built for x86 (mobile) • emulator-5556 • android-x86 • Android 10 (API 29) (emulator)
[1]: Android SDK built for x86 (emulator-5554)
[2]: Android SDK built for x86 (emulator-5556)
Please choose one (To quit, press “q/Q”): 2
Using hardware rendering with device Android SDK built for x86. If you notice graphics artifacts, consider enabling software rendering with “ — enable-software-rendering”.
Launching test_driver/app.dart on Android SDK built for x86 in debug mode…
Running Gradle task ‘assembleDebug’…
Running Gradle task ‘assembleDebug’… Done 108.1s
√ Built build\app\outputs\flutter-apk\app-debug.apk.
Installing build\app\outputs\flutter-apk\app.apk… 5.1s
D/EGL_emulation( 7423): eglMakeCurrent: 0xf207f9a0: ver 3 1 (tinfo 0xe759c370)
Syncing files to device Android SDK built for x86… 243ms
Flutter run key commands.
r Hot reload.
R Hot restart.
h Repeat this help message.
d Detach (terminate “flutter run” but leave application running).
c Clear the screen
q Quit (terminate the application on the device).
Running with unsound null safety
For more information see https://dart.dev/null-safety/unsound-null-safety
An Observatory debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:56335/VBrQsPOWCg0=/
The Flutter DevTools debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:9103?uri=http%3A%2F%2F127.0.0.1%3A56335%2FVBrQsPOWCg0%3D%2F

In this example, we will keep http://127.0.0.1:56335/VBrQsPOWCg0=/ for the next step.

After launching the app, we have to prepare the first test which will generate a QR code (given a certain string) and take a screenshot that will be stored on the machine running the tests. This test will be stored in test_driver/app_test.dart.

This test will do the following:

The command launching the test should look like:

dart test_driver/app_test.dart <app_url> <value_to_encode>

In our example, we could use:

dart test_driver/app_test.dart http://127.0.0.1:56335/VBrQsPOWCg0=/ Success!

Screenshot transfer

Now that we have our QR code in a screenshot ready to be scanned (on the machine running the tests), we need to transfer this screenshot to the scanning emulator. Therefore, we will need the help of ADB tools.

The transfer part will be executed in two steps. ‘Why?’, you might ask. It’s to spare you from a LOT of permission issues.

So the first step will be transferring the screenshot from the machine to the scanning emulator’s sd card (since the machine has access to this folder) using the following command:

adb -s <decoding_emulator> push ../test.png /sdcard/Pictures/test.png

The “decoding_emulator” is the id of the emulator on which will run the QR decoding app. You can find the list of all the devices using:

adb devices

So for the second part of the transfer, the file will be copied from the sd card to the cache of the app (since it’s accessible by the app). To do that, we will use a small script with the superuser option.

adb -s <decoding_emulator> shell su 0 cp /sdcard/Pictures/test.png /data/user/0/com.example.flutter_qr_decoder/cache/test.png

Decoding the QR code

Now the screenshot is in the emulator with the decoding app running. Let’s prepare the test_driver/app.dart for this app. In addition to enabling flutter_driver, we will mock the call to the image picker to directly return the path of the transferred screenshot.

And in the same way, as we did for the QR code generation, we run the flutter app with the following command (and choose the emulator):

flutter run test_driver/app.dart

And in the output, we catch the app URL.

Multiple devices found:
Android SDK built for x86 (mobile) • emulator-5554 • android-x86 • Android 10 (API 29) (emulator)
Android SDK built for x86 (mobile) • emulator-5556 • android-x86 • Android 10 (API 29) (emulator)
[1]: Android SDK built for x86 (emulator-5554)
[2]: Android SDK built for x86 (emulator-5556)
Please choose one (To quit, press "q/Q"): 1
Using hardware rendering with device Android SDK built for x86. If you notice graphics artifacts, consider enabling software rendering with "--enable-software-rendering".
Launching test_driver/app.dart on Android SDK built for x86 in debug mode...
Running Gradle task 'assembleDebug'...
Running Gradle task 'assembleDebug'... Done 61.7s
√ Built build\app\outputs\flutter-apk\app-debug.apk.
Installing build\app\outputs\flutter-apk\app.apk... 4.0s
D/EGL_emulation(18086): eglMakeCurrent: 0xebc6b740: ver 3 1 (tinfo 0xdd785030)
Syncing files to device Android SDK built for x86... 894ms
Flutter run key commands.
r Hot reload.
R Hot restart.
h Repeat this help message.
d Detach (terminate "flutter run" but leave application running).
c Clear the screen
q Quit (terminate the application on the device).
Running with unsound null safety
For more information see https://dart.dev/null-safety/unsound-null-safety
An Observatory debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:57030/gGnADAHH2H8=/
The Flutter DevTools debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:9102?uri=http%3A%2F%2F127.0.0.1%3A57030%2FgGnADAHH2H8%3D%2F

In this example, we will keep http://127.0.0.1:56335/VBrQsPOWCg0=/ for the next step.

Now it’s the test_driver/app_test.dart‘s time to shine:

This test will do the following:

The command launching the test should look like:

dart test_driver/app_test.dart <app_url> <value_to_expect>

In our example, we could use:

dart test_driver/app_test.dart http://127.0.0.1:56335/VBrQsPOWCg0=/ Success!

Wrap it up!

To make it easier when launching the end-to-end test, I tried to write everything in one main Dart script (This script is located, in my case in the flutter_qr_decoder folder under test_driver). So here what we obtain in the test_driver/main_test.dart:

If we keep the file as it is, we use the following command:

dart test_driver/main_test.dart

If we comment lines 6 to 9 and uncomment the lines 12 to15, we can use the following command.

dart test_driver/main_test.dart <generator_app_url> <decoder_app_url> <decoding_emulator> <value_to_expect>

Ex:

dart test_driver/main_test.dart http://127.0.0.1:56335/VBrQsPOWCg0=/ http://127.0.0.1:56335/VBrQsPOWCg0=/ emulator-5554 Success!