Unit Testing Using CMocka

Getting started

Although not required, when using CMocka, it is recommended to have the test cases as separate executables, so these examples will have their own separate main() function.

In CMocka, each test case is a function. The following is a test case testing an addition:

static void test_addition(void **state)
{
        // given
        int a = 1;
        int b = 2;
        int expected_result = 3;
        int result;

        // when
        result = a + b;

        // then
        assert_int_equal(expected_result, result);
}

To execute this test, we need a test suite. In the following example, we execute a CMocka test suite containing our addition test:

int main(void)
{
        const struct CMUnitTest tests[] = {
                cmocka_unit_test(test_addition),
        };

        return cmocka_run_group_tests(tests, NULL, NULL);
}

Running this example will give us the following result:

[==========] tests: Running 1 test(s).
[ RUN      ] test_addition
[       OK ] test_addition
[==========] tests: 1 test(s) run.
[  PASSED  ] 1 test(s).

This means that everything went well in our test (note that the output format follows the standard set by GoogleTest).

The following is the output of our verification failing when I set expected_result to 4. Note that CMocka tells us what was the error, the file, and the line.

[==========] tests: Running 1 test(s).
[ RUN      ] test_addition
[  ERROR   ] --- 0x4 != 0x3
[   LINE   ] --- /cmake-cmocka-template/test/test_add2.c:19: error: Failure!
[  FAILED  ] test_addition
[==========] tests: 1 test(s) run.
[  PASSED  ] 0 test(s).
[  FAILED  ] tests: 1 test(s), listed below:
[  FAILED  ] test_addition

 1 FAILED TEST(S)

Note about including cmocka.h

CMocka requires some headers to be included before its header is included. This allows the user to customize some behavior, e.g. integer size, depending on the resources available on the target platform. Including the CMocka library should look something like the following:

#include <limits.h>
#include <setjmp.h>
#include <stdarg.h>
#include <stddef.h>
// IMPORTANT: cmocka needs the imports above
#include <cmocka.h>

Assertions

CMocka comes with a big list of assertions. You can find them here. You can also use the function fail_msg to instantly fail with the given msg. This function uses a syntax similar to printf.

Custom test state

Some of you might have noticed that on the previous examples, our test_addition function received a state parameter that we didn't use. This parameter is a mechanism of passing an initial state the test. For example, imagine that you want to parametrize the test arguments of a test case:

typedef struct {
        int *test_args;
        ptrdiff_t current_test;
} addition_params_t;

static void test_addition(void **state)
{
        // given
        addition_params_t *addition_params = (addition_params_t *)(*state);
        // skip to our current test
        int *test_args = addition_params->test_args +
                         (addition_params->current_test * 3);
        int a = test_args[0];
        int b = test_args[1];
        int expected_result = test_args[2];
        int result;
        addition_params->current_test++;

        // when
        result = a + b;

        // then
        assert_int_equal(expected_result, result);
}

int main(void)
{
        int test_args[] = {
                1, 2, 3,
                2, 2, 4,
        };
        addition_params_t addition_params = {
                .test_args = test_args,
                .current_test = 0,
        };

        const struct CMUnitTest tests[] = {
                cmocka_unit_test_prestate(test_addition, &addition_params),
                cmocka_unit_test_prestate(test_addition, &addition_params),
        };

        return cmocka_run_group_tests(tests, NULL, NULL);
}

In this example we call the same test function twice, with different parameters, so effectively get 2 test cases. First we test that 1 + 2 == 3 and then we test that 2 + 2 == 4. Note that instead of adding a test case to the suite with cmocka_unit_test, we add it using cmocka_unit_test_prestate so we can pass it the initial state.

Test setup and teardown

Sometimes we need to do some setup steps that are common between multiple test cases. These can range from initializing some memory region, to setting up a certain state/condition. Either way, CMocka has a mechanism to register setup and teardown callbacks for specific test cases or for the whole suite.

Test case specific setup/teardown

Let's get back to our simpler version of the addition test:

static void test_addition(void **state)
{
        // given
        int a = 1;
        int b = 2;
        int expected_result = 3;
        int result;

        // when
        result = a + b;

        // then
        assert_int_equal(expected_result, result);
}

Consider that we wanted to run something before and after the test. In this case, we'll just say hi and goodbye:

static int test_addition_setup(void **state)
{
        printf("Hello from test_addition_setup\n");
        return 0;
}

static int test_addition_teardown(void **state)
{
        printf("Goodbye from test_addition_teardown\n");
        return 0;
}

int main(void)
{
        const struct CMUnitTest tests[] = {
                cmocka_unit_test_setup_teardown(test_addition,
                                                &test_addition_setup,
                                                &test_addition_teardown),
                cmocka_unit_test_setup_teardown(test_addition,
                                                &test_addition_setup,
                                                &test_addition_teardown),
        };

        return cmocka_run_group_tests(tests, NULL, NULL);
}

Resulting in each test (same test function ran twice) saying hello and goodbye:

[==========] tests: Running 2 test(s).
[ RUN      ] test_addition
Hello from test_addition_setup
Goodbye from test_addition_teardown
[       OK ] test_addition
[ RUN      ] test_addition
Hello from test_addition_setup
Goodbye from test_addition_teardown
[       OK ] test_addition
[==========] tests: 2 test(s) run.
[  PASSED  ] 2 test(s).

Note that contrary to our test case function that return nothing, both setup and teardown return an int. When the return of either setup or teardown is not zero, the test will fail. For example, if we return 1 from test_addition_setup:

[==========] tests: Running 2 test(s).
[ RUN      ] test_addition
Hello from test_addition_setup
Could not run test: Test setup failed
[  ERROR   ] test_addition
[ RUN      ] test_addition
Hello from test_addition_setup
Could not run test: Test setup failed
[  ERROR   ] test_addition
[==========] tests: 2 test(s) run.
[  PASSED  ] 0 test(s).

Test suite setup/teardown

Let's consider the case we just saw. We might want to perform the setup only once before all tests and the teardown only once after all tests. In this case we want to register a test suite setup/teardown instead of a test case one:

static int test_suite_setup(void **state)
{
        printf("Hello from test_suite_setup\n");
        return 0;
}

static int test_suite_teardown(void **state)
{
        printf("Goodbye from test_suite_teardown\n");
        return 0;
}

int main(void)
{
        const struct CMUnitTest tests[] = {
                cmocka_unit_test(test_addition),
                cmocka_unit_test(test_addition),
        };

        return cmocka_run_group_tests(tests, &test_suite_setup, &test_suite_teardown);
}

This time we see the messages only once:

[==========] tests: Running 2 test(s).
Hello from test_suite_setup
[ RUN      ] test_addition
[       OK ] test_addition
[ RUN      ] test_addition
[       OK ] test_addition
Goodbye from test_suite_teardown
[==========] tests: 2 test(s) run.
[  PASSED  ] 2 test(s).

Just like in the test case specific setup/teardown, an output different from zero will result in a test suite execution failure. In the case of a setup failure test tests don't even run:

[==========] tests: Running 2 test(s).
Hello from test_suite_setup
[  FAILED  ] GROUP SETUP
[  ERROR   ] tests
Goodbye from test_suite_teardown
[==========] tests: 0 test(s) run.
[  PASSED  ] 0 test(s).

Combining setup, teardown, and state

Just like the test case function, the test setup and teardown function also receive a state parameter. When you pass state to a test, first it is passed to the setup function, then to the test, and finally to the teardown function.

CMocka allows us to do any combination of setup, teardown, or state passing through its API. Take a look at the following test case registration function:

• cmocka_unit_test
• cmocka_unit_test_setup
• cmocka_unit_test_teardown
• cmocka_unit_test_setup_teardown
• cmocka_unit_test_prestate
• cmocka_unit_test_prestate_setup_teardown

Any of the arguments can be set to NULL, with the exception of the test case function, if not needed.

Mocks

Mocks are a great tool to test a function's dependencies. Let's consider the existence of two functions: add and add3:

int add(int a, int b)
{
        return a + b;
}

int add3(int a, int b, int c)
{
        int a_plus_b = add(a, b);
        return add(a_plus_b, c);
}

add3 adds 3 numbers and calls add twice to do that. When we're unit testing add3 we might want to mock add to isolate add3's behavior. Let's create a mock for add and a test case for add3 using it:

int my_add(int a, int b);

int __wrap_add(int a, int b) {
  printf("HELLO FROM __wrap_add\n");
  check_expected(a);
  check_expected(b);
  return mock_type(int);
}

static void test_add3_mock(void **state) {
  // given
  int a = 1, b = 2, c = 4;
  int expected_result = 7;
  int result;

  expect_value(__wrap_add, a, 1);
  expect_value(__wrap_add, b, 2);
  will_return(__wrap_add, 3);
  expect_value(__wrap_add, a, 3);
  expect_value(__wrap_add, b, 4);
  will_return(__wrap_add, 7);

  // when
  result = add3(a, b, c);

  // then
  assert_int_equal(expected_result, result);
}

int main(void) {
  const struct CMUnitTest tests[] = {
      cmocka_unit_test(test_add3_mock),
  };

  return cmocka_run_group_tests(tests, NULL, NULL);
}

Mocks in CMocka are just functions where you can implement any behavior you want. CMocka gives us tools to check that the input arguments have the values that we expected (see the checking parameters page). This checks double has a way to count the number of times the mock was called. Note that since the add function will be called twice, we specify the arguments and return values twice as well:

[==========] tests: Running 1 test(s).
[ RUN      ] test_add3_mock
HELLO FROM __wrap_add
HELLO FROM __wrap_add
[       OK ] test_add3_mock
[==========] tests: 1 test(s) run.
[  PASSED  ] 1 test(s).

As expected, our mock was called twice. If we change the test case to expect a single call, the test will fail:

static void test_add3_mock(void **state) {
  // given
  int a = 1, b = 2, c = 4;
  int expected_result = 7;
  int result;

  expect_value(__wrap_add, a, 1);
  expect_value(__wrap_add, b, 2);
  will_return(__wrap_add, 3);

  // when
  result = add3(a, b, c);

  // then
  assert_int_equal(expected_result, result);
}

[==========] tests: Running 1 test(s).
[ RUN      ] test_add3_mock
HELLO FROM __wrap_add
HELLO FROM __wrap_add
[  ERROR   ] --- No entries for symbol __wrap_add.
/cmake-cmocka-template/test/test_add2.c:16: error: Could not get value to check parameter a of function __wrap_add
/cmake-cmocka-template/test/test_add2.c:28: note: Previously declared parameter value was declared here

[  FAILED  ] test_add3_mock
[==========] tests: 1 test(s) run.
[  PASSED  ] 0 test(s).
[  FAILED  ] tests: 1 test(s), listed below:
[  FAILED  ] test_add3_mock

 1 FAILED TEST(S)

Linker tricks

In order to replace the calls to add by calls to our mock in add3's code, we use a linker trick: the --wrap flag. This linker flag/options allows us to define a symbol that will replace another at link time. In this case, __wrap_add replaces any undefined reference to add at link time, because we compiled the code with the flags -Wl,--wrap=add. Note that function names starting with __wrap are reserved by the compiler for this reason. Here is an excerpt from ld's manual explaining the functionality:

Use a wrapper function for symbol. Any undefined reference to symbol will be resolved to "wrap_symbol". Any undefined reference to "__real_symbol" will be resolved to symbol. This can be used to provide a wrapper for a system function. The wrapper function should be called "wrap_symbol". If it wishes to call the system function, it should call "__real_symbol".

If for some reason we're using a linker that doesn't support this feature, we need to do some dependency injection, which can involve changes to the source code under test. Design your code with this in mind.

Conclusion

CMocka is pretty cool and powerful. Highly recommend it. If you're starting a new C project or just want to see more examples on how to integrate and use CMocka in your projects, take a look at my cmake-cmocka-template project