This a test suite of small "unit tests" that verify individual pieces of ZeekCtl
functionality. They all utilize BTest, a simple framework/driver for
writing unit tests. More information about BTest can be found at
http://www.zeek.org/development/README.btest.html
The test suite's BTest configuration is handled through the
``btest.cfg`` file. Of particular interest is the "TestDirs" settings,
which specifies which directories BTest will recursively search for
test files.
Significant Subdirectories
==========================
* Baseline/
Validated baselines for comparison against the output of each
test on future runs. If the new output differs from the Baseline
output, then the test fails.
* Traces/
Packet captures utilized by the various BTest tests.
* Cfg/
This is a collection of config files utilized by various BTest tests.
The directory hierarchy emulates the hierarchy of the Zeek installation
directory.
Running Tests
=============
In order to run the tests you must clone the entire Zeek repo (not just the
zeekctl repo). Next, either use the ``make all`` or ``make brief``
``Makefile`` targets to run all the tests (these will automatically build
Zeek before running any of the zeekctl tests). If Zeek fails to build, you can
define an environment variable ZEEKCTL_TEST_BUILDARGS which specifies
additional options that will be passed to Zeek's "configure" script.
If Zeek builds successfully, but one or more tests fail, then you can
re-run all failed tests with ``make rerun``.
If you want to run just a subset of the tests, then build Zeek with
``make buildzeek``, followed by ``btest`` directly with desired
options/arguments. Examples:
* btest <no arguments>
If you simply execute btest in this directory with no arguments,
then all directories listed as "TestDirs" in btest.cfg will be
searched recursively for test files.
* btest <btest options> test_directory
You can specify a directory on the command line to run just the
tests contained in that directory. This is useful if you wish to
run all of a given type of test, without running all the tests
there are. For example, "btest command" will run all of the zeekctl
command unit tests.
* btest <btest options> test_directory/test_file
You can specify a single test file to run just that test. This
is useful when testing a single failing test or when developing
a new test.
How The Tests Work
==================
The zeekctl testsuite uses btest, but not all test artifacts reside in each
test's subdirectory below .tmp/. Instead, the tests use the Zeek build created
as described above, copying it into per-test subdirectories to achieve isolation
of config tweaks. Those directories are $ZEEKCTLSRCDIR/build/testing/test.<num>,
where $ZEEKCTLSRCDIR is the parent of the directory this README resides
in. Inspect the generated cluster-layout.zeek and other resulting scripts in
those directory trees.
The tests normally wipe these temporary directories. btest.cfg provides a
"debug" environment ("btest -a debug" ...) that preserves them. The individual
test's .stdout/stderr files contain enough information to tell you which of
those temporary directories they relied on.
In many cases, zeekctl's tests don't actually run Zeek. Instead, they run a
Python stub script, ./Cfg/bin/zeek__test, to allow controlled injection of
failure modes. This is why you don't see rampant listening-port collisions when
running this testsuite.
A subset of tests is serialized at the btest level via @TEST-SERIALIZE -- those
are tests that require actual listening ports.
Adding Tests
============
See either the `BTest documentation
<http://www.zeek.org/development/README.btest.html>`_ or the existing unit
tests for examples of what they actually look like. The essential
components of a new test include:
* A test file in one of the subdirectories listed in the ``TestDirs``
of the ``btest.cfg`` file.
* If the unit test requires a known-good baseline output against which
future tests will be compared (via ``btest-diff``), then that baseline
output will need to live in the ``Baseline`` directory. Manually
adding that is possible, but it's easier to just use the ``-u`` or
``-U`` options of ``btest`` to do it for you (using ``btest -d`` on a
test for which no baseline exists will show you the output so it can
be verified first before adding/updating the baseline output).
If you create a new top-level testing directory for collecting related
tests, then you'll need to add it to the list of ``TestDirs`` in
``btest.cfg``. Do this only if your test really doesn't fit logically in
any of the extant directories.
Note that any new test you add this way will automatically be included
in the testing done in Zeek's central automated build & test
environment.
