tools/testing/kunit/android/README

HOW TO RUN KUNIT TESTS IN ANDROID
=================================

Prerequisites
-------------
  If you want to run a vendor module KUnit tests, please run the tests with a
  "no trim" kernel (e.g. add `--notrim` to bazel build command).

Run tests on a physical or virtual device:
------------------------------------------
  ```
  $ kernel/tests/tools/run_test_only.sh -t kunit -s <serial_number> -td <test_dir>
  ```

  test_dir is the same directory as specified when running:
  ```
  $ tools/bazel run //common:kunit_tests_arm64 -- -v --destdir <test_dir>
  ```

Before the tests, you can use the following command to launch a virtual device:
  ```
  $ kernel/tests/tools/launch_cvd.sh
  ```

After the tests, you can use the following command to remove the virtual device:
  ```
  $ prebuilts/asuite/acloud/linux-x86/acloud delete
  ```

The following are command examples:
 * Build kernel and launch a virtual device from a specific platform build:
   ```
   $ kernel/tests/tools/launch_cvd.sh -pb \
         ab://aosp-main/aosp_cf_x86_64_phone-trunk_staging-userdebug/12505199
   ```

 * Run a specific test:
   ```
   $ kernel/tests/tools/run_test_only.sh \
         -t 'kunit soc-utils-test' -s <serial_number>
   ```

 * Check other available options:
   ```
   $ kernel/tests/tools/launch_cvd.sh -h
   $ kernel/tests/tools/run_test_only.sh -h
   ```

Load and run a test module on Android device manually
-----------------------------------------------------
 * Push the KUnit test framework module kunit.ko over to the device. For
 example:
   ```
   $ adb push kunit.ko /data
   ```

 * Load test module on device:
   ```
   $ cd /data
   $ insmod kunit.ko enable=1
   ```

   If the kunit.ko has been installed already but without enable=1 passed,
   it needs to remove it first via the rmmod command, and install again
   via the insmod command

 * Push the KUnit test module over to the device. For example using adb:
   ```
   $ adb push kunit-example-test.ko /data
   ```

 * (Optional) - Mount debugfs on device:
   ```
   $ mount -t debugfs debugfs /sys/kernel/debug
   ```

 * Load test module on device:
   ```
   $ cd /data
   $ insmod kunit-example-test.ko
   ```

 * View test results
    * If debugfs is mounted:
      ```
      $ cat /sys/kernel/debug/kunit/<test name>/results
      KTAP version 1
      1..1
          KTAP version 1
          # Subtest: example
          1..4
          # example_simple_test: initializing

          ok 1 example_simple_test
      ....
      ```

    * Via dmesg (check before log cycles out):
      ```
      $ dmesg
      ....
      [172434.032618] 1..1
      [172434.032618]    KTAP version 1
      [172434.032618]    # Subtest: example
      [172434.032618]    1..4
      [172434.032618]    # example_simple_test: initializing
      [172434.032618]
      [172434.032618]    ok 1 example_simple_test
      ....
      ```

Run KUnit tests on Android Device via test automation infrastructure tradefed
-----------------------------------------------------------------------------
 * Build ACK KUnit tests and install (e.g. /tmp/kunit_tests):
   ```
   $ tools/bazel run -- //common:kunit_tests_x86_64 -v --destdir /tmp/kunit_tests
   ```
   Or
   ```
   $ tools/bazel run -- //common:kunit_tests_arm64 -v --destdir /tmp/kunit_tests
   ```

 * With device connected and accessible via adb run the tests:
   ```
   $ prebuilts/tradefed/filegroups/tradefed/tradefed.sh run commandAndExit \
         template/local_min --template:map test=suite/test_mapping_suite   \
         --include-filter kunit --tests-dir=/tmp/kunit_tests \
         -s <your_device_serial_number>
   ....
   =======================================================
   =============== Summary ===============
   Total Run time: 23s
   1/1 modules completed
   Total Tests       : 9
   PASSED            : 9
   FAILED            : 0
   ============== End of Results ==============
   ============================================
   ....
   ```

Troubleshooting
---------------

1. Test module fails to load.

   Check dmesg for load errors. If undefined symbol errors are shown, you're
   likely running with a trimmed kernel where the symbols are not available.
   Run with a "no trim" kernel.

   Check the test module dependency with `modinfo <module_name>.ko` on your
   local host machine or on the Android device with
   `adb shell modinfo <module_name>.ko`.
   All dependent modules need to be installed before the test module can be
   installed successfully.

   Check if the module is already installed with `adb shell lsmod`. The
   `adb shell rmmod` can be used to remove the already installed test module,
   and installing the test module again will trigger the test rerun.

   `adb shell lsmod` will also show the module dependency for your test module
   in the `Used by` column. You can not remove a module with `adb shell rmmod`
   if it is being used by another module. Other modules that are using it need
   to be removed first.

2. Test module loaded but no test results.

   Check dmesg for KUnit errors.
   ```
   $ dmesg | grep kunit
   ```

   If "kunit: disabled" is shown then kunit.ko is not installed with
   `enable=1`.

   If kunit.ko or \<module_name\>.ko fails to install, check for whether they
   are already installed with `adb shell lsmod`.

HOW TO WRITE KUNIT TESTS
========================

If you want to
just run the example test and skip through the walkthrough, it is sufficient to
apply [patch1.txt](patch1.txt) and [patch2.txt](patch2.txt) and
then [run the test](#run-tests-on-a-physical-or-virtual-device).

Walkthrough
-----------

This section follows the
[Writing Your First Test](https://docs.kernel.org/dev-tools/kunit/start.html#writing-your-first-test)
example from the KUnit docs.

### Create an example driver to test

1. Create the header file for the driver `common/drivers/misc/example.h`:

   ```c
   int misc_example_add(int left, int right);
   ```

2. Define and export `misc_example_add` that will be tested later
   `common/drivers/misc/example.c`:

   ```c
   #include <linux/module.h>

   #include "example.h"

   int misc_example_add(int left, int right)
   {
           return left + right;
   }
   EXPORT_SYMBOL_GPL(misc_example_add);
   ```

3. Add a kconfig option for the driver to `common/drivers/misc/Kconfig`:

   ```
   config MISC_EXAMPLE
   	bool "My example"
   ```

4. Add the build rule to `common/drivers/misc/Makefile`:

   ```makefile
   obj-$(CONFIG_MISC_EXAMPLE)	+= example.o
   ```

### Write the test case

1. Define the test functions and suite `common/drivers/misc/example_test.c`:

   ```c
   #include <kunit/test.h>
   #include "example.h"

   /* Define the test cases. */

   static void misc_example_add_test_basic(struct kunit *test)
   {
           KUNIT_EXPECT_EQ(test, 1, misc_example_add(1, 0));
           KUNIT_EXPECT_EQ(test, 2, misc_example_add(1, 1));
           KUNIT_EXPECT_EQ(test, 0, misc_example_add(-1, 1));
           KUNIT_EXPECT_EQ(test, INT_MAX, misc_example_add(0, INT_MAX));
           KUNIT_EXPECT_EQ(test, -1, misc_example_add(INT_MAX, INT_MIN));
   }

   static void misc_example_test_failure(struct kunit *test)
   {
           KUNIT_FAIL(test, "This test never passes.");
   }

   static struct kunit_case misc_example_test_cases[] = {
           KUNIT_CASE(misc_example_add_test_basic),
           KUNIT_CASE(misc_example_test_failure),
           {}
   };

   static struct kunit_suite misc_example_test_suite = {
           .name = "misc-example",
           .test_cases = misc_example_test_cases,
   };
   kunit_test_suite(misc_example_test_suite);

   MODULE_DESCRIPTION("KUnit test for misc_example_add");
   MODULE_LICENSE("GPL");
   ```

   Consult the
   [KUnit usage guide](https://docs.kernel.org/dev-tools/kunit/usage.html#) and
   the
   [KUnit API reference](https://docs.kernel.org/dev-tools/kunit/api/index.html)
   for more in-depth guidance on the test API.

2. Create a kconfig option for the test `common/drivers/misc/Kconfig`:

   ```
   config MISC_EXAMPLE_TEST
   	tristate "Test for my example" if !KUNIT_ALL_TESTS
   	depends on MISC_EXAMPLE && KUNIT
   	default KUNIT_ALL_TESTS
   ```

3. Add the build rule to the makefile `common/drivers/misc/Makefile`:

   ```makefile
   obj-$(CONFIG_MISC_EXAMPLE_TEST)	+= example_test.o
   ```

4. Add the kconfig options for kunit.py `common/drivers/misc/.kunitconfig`:

   ```bash
   CONFIG_KUNIT=y
   CONFIG_MISC_EXAMPLE=y
   CONFIG_MISC_EXAMPLE_TEST=y
   ```

### Run the test against User-mode Linux

1. The test can then be run against User-mode Linux using the kunit.py script:

   ```
   $ cd $REPO/common
   $ tools/testing/kunit/kunit.py run --kunitconfig drivers/misc
   [01:21:36] Configuring KUnit Kernel ...
   Regenerating .config ...
   Populating config with:
   $ make ARCH=um O=.kunit olddefconfig
   [01:21:38] Building KUnit Kernel ...
   Populating config with:
   $ make ARCH=um O=.kunit olddefconfig
   Building with:
   $ make ARCH=um O=.kunit --jobs=96
   [01:21:49] Starting KUnit Kernel (1/1)...
   [01:21:49] ============================================================
   Running tests with:
   $ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
   [01:22:01] ================ misc-example (2 subtests) =================
   [01:22:01] [PASSED] misc_example_add_test_basic
   [01:22:01]     # misc_example_test_failure: EXPECTATION FAILED at drivers/misc/example_test.c:17
   [01:22:01] This test never passes.
   [01:22:01] [FAILED] misc_example_test_failure
   [01:22:01]     # module: example_test
   [01:22:01] # misc-example: pass:1 fail:1 skip:0 total:2
   [01:22:01] # Totals: pass:1 fail:1 skip:0 total:2
   [01:22:01] ================== [FAILED] misc-example ===================
   [01:22:01] ============================================================
   [01:22:01] Testing complete. Ran 2 tests: passed: 1, failed: 1
   [01:22:01] Elapsed time: 24.906s total, 2.271s configuring, 10.889s building, 11.694s running
   ```

2. `kunit.py` will create symlinks under the `.kunit` directory that interfere
   with Kleaf/Bazel. This directory is ignored by `.bazelignore`.

### Configure the test for Android

Now that the example test is created, the test needs to be configured for
Android. We will need to add build rules for Kleaf/Bazel and create a Tradefed
test config.

1. Set `CONFIG_MISC_EXAMPLE=y` and `CONFIG_MISC_EXAMPLE_TEST=m` in
   [gki_defconfig](https://source.corp.google.com/h/android/kernel/superproject/+/common-android-mainline:common/arch/x86/configs/gki_defconfig).
   (This can be done using menuconfig; refer to the
   [Kleaf documentation](https://android.googlesource.com/kernel/build/+/refs/heads/main-kernel/kleaf/docs/kernel_config.md)):

   ```
   $ cd $REPO
   $ tools/bazel run //common:kernel_x86_64_config -- menuconfig
   ```

2. Add the test module to `_KUNIT_COMMON_MODULES_LIST` in `common/modules.bzl`:

   ```bazel
   _KUNIT_COMMON_MODULES_LIST = [
       ....
       "drivers/misc/example_test.ko",
       ....
   ]
   ```

3. Add the test under `KUnitModuleTest` in the tradefed configuration
   `common/tools/testing/kunit/android/tradefed_configs/config_x86_64.xml`:

   ```xml
       <test class="com.android.tradefed.testtype.binary.KUnitModuleTest" >
           ....
           <option name='binary' key='drivers/misc/example_test' value='/data/kunit/arm64/example_test.ko' />
           ....
       </test>
   ```