Project quality assurance
New development means high risk of regression.
Therefore this project require some formal validation procedure to cover the most important features.
Let's use this issue to accumulate the experience ( scenarios, procedures, scripts... ) I hope someday we can make it a part of the CI.
Test 1. Run UIBench <Semi-automated> (Prefer using test 2 instead)
Preconditions:
- Test tool: https://github.com/joshuous/uibencher
- Use same equipment (Board and Monitor) and same release (for example AOSP-11r27)
Procedure:
- Run the test on
drm_hwcomposer/main
and store baseline values - Run the test with the new changes
- Collect the data. Number of Janks should be equal or less than in the baseline run.
Test 2. Run UIBench from AOSP tree
Preconditions:
- Use same equipment (Board and Monitor) and same release (for example AOSP-11r27)
- Disable flattening in drm_hwc to get true results
- Patch AOSP test manifest file to enable all tests instead of 1:
--- a/platform_testing/tests/microbenchmarks/uibench/AndroidTest.xml
+++ b/platform_testing/tests/microbenchmarks/uibench/AndroidTest.xml
@@ -39,3 +39,2 @@
<option name="package" value="com.android.uibench.microbenchmark" />
- <option name="class" value="com.android.uibench.microbenchmark.UiBenchDialogListFlingMicrobenchmark#testDialogListFling" />
<option name="device-listeners" value="android.device.collectors.PerfettoListener,android.device.collectors.SfStatsListener" />
Procedure:
- Connect the board
- Execute:
atest UiBenchMicrobenchmark | tee results.txt
- Filter-out useful data:
cat results.txt | grep -i "MISS\|CLIENT\|PASSED"
Test 3. Run Composer HAL VTS tests from AOSP tree
Procedure:
- Connect the board
- Execute:
atest VtsHalGraphicsComposerV2_1TargetTest VtsHalGraphicsComposerV2_2TargetTest VtsHalGraphicsComposerV2_3TargetTest VtsHalGraphicsComposerV2_4TargetTest