| commit | 9e9d635848132f8ebecc43754d9406922db7d3ab | [log] [tgz] |
|---|---|---|
| author | Zygmunt Krynicki <zygmunt.krynicki@linaro.org> | Tue Aug 21 22:17:17 2012 +0200 |
| committer | Zygmunt Krynicki <zygmunt.krynicki@linaro.org> | Tue Aug 21 22:17:17 2012 +0200 |
| tree | 17a2b9aec19f132ad5618cb63ed5c30fabff3678 | |
| parent | b1a803e5ff92922c95146c4634987318e11828d1 [diff] |
Add reference output for skia_test skia_test is not a gtest-based test and will require a new parser but having the reference output is the first stage to make that possible.
This module contains a collection of tools that run on an Android system and simplify the automation of tests. Android includes a lot of built-in tests but they differ in output formats and a way of executing them.
To use this software you need to incorporate it in your build. The best way to do that is to augment the manifest to pull code from the official github repository: git://github.com/zyga/android-lava-wrapper.git
For instance, to add it to one of the Linaro builds all I had to do was to patch the manifest with the following diff:
diff --git a/landing-snowball.xml b/landing-snowball.xml index f3764b4..7d53aa1 100644 --- a/landing-snowball.xml +++ b/landing-snowball.xml @@ -10,6 +10,8 @@ fetch="git://git.linaro.org/" /> <remote name="android_input_bridge" fetch="git://github.com/rperier/" /> + <remote name="android-lava-wrapper" + fetch="git://github.com/zyga/" /> <default revision="refs/tags/android-4.1.1_r4" remote="aosp" sync-j="4" /> @@ -20,6 +22,7 @@ <!-- External extras --> <project path="packages/apps/0xbench" name="platform/packages/apps/0xbench" revision="linaro-master" /> <project path="external/android_input_bridge" name="android_input_bridge" remote="android_input_bridge" revision="master" /> + <project path="external/android-lava-wrapper" name="android-lava-wrapper" remote="android-lava-wrapper" revision="master" /> <!-- Linaro extras --> <project path="packages/apps/DisableSuspend" name="platform/packages/apps/DisableSuspend" revision="master" />
Having done that run repo sync to fetch the new repository.
It is important to know that android-lava-wrapper runs only in builds made with TARGET_BUILD_VARIANT=tests. If in doubt have a look at my build scripts at http://github.com/zyga/android-build-scripts The configurations ending with -tests build the proper variant.
Once you have a build transfer it onto your device (make flash if you have used my build scripts) and get adb shell or direct serial shell to your device. You will need to run as root so you may need to use su first.
To run all of the tests at once simply run:
$ lava-wrapper --run-all-tests
Observe the output as tests fly by to know where each one gets saved but in general all results will be saved to /sdcard/LAVA.
You can fetch each result with adb pull but ideally you will use lava-android-test to run this and obtain a single bundle directly.
This topic shall be extended as lava-android-test parts get written.
All of the tests that use the Google test framework are supported. Support for additional tests can be added in the future. You can add support for additional tests by simply shipping a wrapper finder that will identify new tests.
This tool, runs on an Android device, once triggered it requires no outside help to produce test results. It follows this simple algorithm to complete its task:
There are a few other things to know about:
The tool creates a directory to store all of the bundles. By default that is /sdcard/LAVA but it can be overridden on the command line. Inside that directory, each time the script runs, a new unique directory is created. Inside that directory, in turn, are all of the bundles created by subsequent wrappers.
The tool passes -r option to each wrapper to produce human readable output.
There is no way for two wrapper finders to offer support for a single test, if that happens the result is not deterministic and depends on the ordering of file system traversal.
If a test or test wrapper hangs the whole tool will hang as well.
lava-wrapper: This tool, the main script that should be launched (on an Android device) to start testing.
wrapper finders: Small programs that helps lava-wrapper to recognize valid tests and suggests which wrappers should be used to process that test. android-lava-wrapper ships with one wrapper finder by default, lava-wrapper-finder-gtest that recognizes certain gtest-based tests. All wrapper finders must be placed in /system/bin and must follow the naming convention lava-wrapper-finder-*
wrappers: A small utility that can convert output of a certain kind of tests into the Linaro Dashboard Bundle format. Each wrapper has a standardized interface and must accept -o and -r options as well as a single argument that points to the test to execute. The wrapper must execute the test in any means necessary and create a bundle with the processed test results. android-lava-wrapper ships with lava-gtest-wrapper that knows how to execute and parse gtest-based tests.
Linaro Dashboard Bundle: A JSON-based data interchange format designed to store test results and rich meta-data. This format is consumed by the LAVA Dashboard. The format is documented at http://linaro-dashboard-bundle.readthedocs.org/
LAVA Dashboard: Part of LAVA that stores displays test results. You can think of it a big web-accessible archive. You can learn more about it at http://lava-dashboard.rtfd.org/