C project IDE and skeleton [part 2]: vim, autotools, ctags, cmocka, lcov, dejagnu, apr, doxygen, git, guard

This is second part. The first part is here.

After setting up my development environment I am starting with writing tests. I do prefer TDD way and usually the process RED-GREEN-REFACTORING takes many changes during development as TDD allows to improve and optimize code any time (refactoring). But here I will put just final version of the code.

The final version can be found on my github repo.

Before starting, I run "guard" command in the console to let this amazing utility run all commands for my.



My working environment looks like this:

On top there is vim editor with configuration from , then on left bottom there is guard which is running make and tests. And another shell just for running commands if needed.

Unit tests with cmocka

This is a small project so there is only one unit test file "test/luhn_tests.c". Test cases are for testing when number is valid or invalid Luhn numbers, testing generated numbers and test length (min and max).

Numbers for tests:

const char* valid_nums[] = {
  "4556224242479034",
  "3535297944875111",
  "372542317279595",
  "1234567812345670",
  "49927398716"
};

const char* invalid_nums[] = {
  "49927398717",
  "1234567812345678",
  "453992406665297",
  "5544850398429482",
  "5917375561450081"
};

This project is using Apache Portable Runtime and its convinient memory management API. To avoid repeating memory management functions initialization and termination, we can use cmocka "setup" and "teardown" helper function:

static int setup(void **state) {
  apr_pool_t *mp;

  apr_initialize();
  apr_pool_create(&mp, NULL);

  *state = mp;

  return 0;
}

static int teardown(void **state) {
  apr_pool_destroy(*state);
  apr_terminate();
  return 0;
}

In setup, APR memory pool is initialized adn the destroyed in teardown function. In setup, APR memory pool is stored in "*state" parameter for later use in test function. Example of such a test run:

cmocka_unit_test_setup_teardown (luhn_gen_test, setup, teardown)

The complete implementation of the unit tests is here.

When we save the file, guard will run the tests and fail (RED). It is expected behaviour, as library functions are not yet implemented.

Lirary "luhn" provides only two functions:
int luhn_valid (const char *num)
and
char* luhn_gen (apr_pool_t *mp, int size)

The whole library implementation can be found here.

When the library source code is implemented and saved, guard runs the tests and they should pass (GREEN).

Frontend tests with DejaGNU

The application behavior is defined by the parameters passed to it when we run it from in console. Now is the time to do front-end tests to assure that application behaves properly. Dejagnu with its "expect" backend is a perfect tool to do that.
The luhn application is designed to run in normal and verbose mode. In normal mode, the application does not print any ouput when it verifies a number with Luhn algorithm. The result is stored in shell exit status when application terminates. In verbose mode there will be some text output that reports validation results.
It is normal behavior for most of Linux command line tools.

The DejaGNU scenario must cover both sytuations.

First, here is tests to check exit status of the application in normal mode (file test/frontend/luhn.exp):

set testcases_status {
  {"Check number status"      "-c 49927398715"          9}
  {"Generate number status"   "-g -l 2"                 10}
  {"Generate number status"   "-g -l 2000"              10}
  {"Invalid parameter status" "-q"                      11}
  {"Check number status"      "--check 49927398716"     0}
}

foreach pattern $testcases_status {
  set msg    [lindex $pattern 0]
  set argv   [lindex $pattern 1]
  set retval [lindex $pattern 2]
  eval "spawn $LUHN $argv"

  eval "exp_spawn $LUHN $argv"
  set ret [exp_wait]
  set pid         [lindex $ret 0]
  set spawn_id    [lindex $ret 1]
  set os_error    [lindex $ret 2]
  set exit_status [lindex $ret 3]

  if { $exit_status == $retval } {
    pass $msg
  } else {
    fail $msg
  }
}

I am not sure if it is the best way but it is the working version I could come up with.

Testing application in verbose mode is kind of natural DejaGNU operation and explained in many tutorials. Here it is:

set testcases_verbose {
  {"Check number"    "-v -c 49927398716"        "OK: valid Luhn number"}
  {"Check number"    "--verbose --check 49927398716" "OK: valid Luhn number"}
  {"Check number"    "-v -c 49927398715"        "FAIL: invalid Luhn number"}
  {"Invalid parameter" "-@"                     "invalid option character"}
  {"Generate number" "-g"                       "^[0-9]{16}"}
  {"Generate number" "--generate"               "^[0-9]{16}"}
  {"Generate number" "-g -l 12"                 "^[0-9]{12}"}
  {"Generate number" "--generate --length 12"   "^[0-9]{12}"}
  {"Generate number" "-v -g -l 2"               "FAIL: length must be beween 4 and 1024"}
  {"Usage message"   "-h"                       "Usage: "}
  {"Usage message"   "--help"                   "Usage: "}
  {"Print version"   "-V"                       "luhn [0-9\.]+"}
  {"Print version"   "--version"                "luhn [0-9\.]+"}
  {"Invalid parameter" "--foo"                  "invalid option"}
  {"No params prints usage" ""                  "Usage: "}
}

foreach pattern $testcases_verbose {
  set msg    [lindex $pattern 0]
  set argv   [lindex $pattern 1]
  set output [lindex $pattern 2]
  eval "spawn $LUHN $argv"

  expect {
    -re $output { pass $msg }
    default     { fail $msg }
  }
}

When we save the file test/frontend/luhn.exp, guard will again run all tests and our frontend tests will fail as they are still not implemented.

Implementation will take several utility functions which we will store in src/utils.c file. First of all, the function to parse command line arguments:

/**
 * Parse command line arguments.
 * @param mp    APR memory pool
 * @param opts  Options structure
 * @param argc  Number of argumets
 * @param argv  Array of arguments
 * @return 0 if parameters successfull
 */
int parse_opts (apr_pool_t **mp, luhn_opts *opts, int argc, const char *argv[]);

And several additional functions to implement verbose output, print usage etc. The implementation is pretty trivial and can be found here.

And the last step is to implement the "main" function in "src/main.c" file which will use Luhn library functionallity and utility functions. The source code can be found here. This one is also not complecated.

When the source code is saved, guard should run all the tests, unit and fronend, and they should pass.

Tests coverage

And the last step is to check if the tests cover all functionality and also if we have some source code that is not used in our application.
In the first part of the blog it was explained how to setup test coverage with lcov.
Now we need only to run following command to generate coverage:

./configure --with-coverage
make clean
make cov

Not "make clean" before "make cov". This is necessary to be sure that make rebuild all the binaries with profiling flags required for lcov.

If everything went well, we can open file coverage/index.html.

In my case I have this page:

Pretty good coverage! We can get more details on each file by following the links in "Directory".

Travis CI platform setup

Travis is a continuous integration service. It is very convenient to use with GitHub projects. Travis will watch GitHub repositories, build and run tests when new commits are pushed.

There are lots of configuration but basic can be relatively easy. Configuration file ".travis.yml" has settings for basic packages install. In case of this project we need APR installed. See "before_build" part with apt-get commands.

This project also need cmocka, but in ubuntu trusty distro cmocka is too old. So we will build cmocka from sources. cmocka is using cmake to build so it will be installed with apt-get in "before_build" part.

Special script ".travis-install-cmocka.sh" will build cmacka library.

Finally, travis will build the project and run tests. Results can be displayed on GitHub README file:

That's it.