| tdc - Adding test cases for tdc | |
| Author: Lucas Bates - lucasb@mojatatu.com | |
| ADDING TEST CASES | |
| ----------------- | |
| User-defined tests should be added by defining a separate JSON file. This | |
| will help prevent conflicts when updating the repository. Refer to | |
| template.json for the required JSON format for test cases. | |
| Include the 'id' field, but do not assign a value. Running tdc with the -i | |
| option will generate a unique ID for that test case. | |
| tdc will recursively search the 'tc' subdirectory for .json files. Any | |
| test case files you create in these directories will automatically be included. | |
| If you wish to store your custom test cases elsewhere, be sure to run tdc | |
| with the -f argument and the path to your file. | |
| Be aware of required escape characters in the JSON data - particularly when | |
| defining the match pattern. Refer to the tctests.json file for examples when | |
| in doubt. | |
| TEST CASE STRUCTURE | |
| ------------------- | |
| Each test case has required data: | |
| id: A unique alphanumeric value to identify a particular test case | |
| name: Descriptive name that explains the command under test | |
| category: A list of single-word descriptions covering what the command | |
| under test is testing. Example: filter, actions, u32, gact, etc. | |
| setup: The list of commands required to ensure the command under test | |
| succeeds. For example: if testing a filter, the command to create | |
| the qdisc would appear here. | |
| cmdUnderTest: The tc command being tested itself. | |
| expExitCode: The code returned by the command under test upon its termination. | |
| tdc will compare this value against the actual returned value. | |
| verifyCmd: The tc command to be run to verify successful execution. | |
| For example: if the command under test creates a gact action, | |
| verifyCmd should be "$TC actions show action gact" | |
| matchPattern: A regular expression to be applied against the output of the | |
| verifyCmd to prove the command under test succeeded. This pattern | |
| should be as specific as possible so that a false positive is not | |
| matched. | |
| matchCount: How many times the regex in matchPattern should match. A value | |
| of 0 is acceptable. | |
| teardown: The list of commands to clean up after the test is completed. | |
| The environment should be returned to the same state as when | |
| this test was started: qdiscs deleted, actions flushed, etc. | |
| SETUP/TEARDOWN ERRORS | |
| --------------------- | |
| If an error is detected during the setup/teardown process, execution of the | |
| tests will immediately stop with an error message and the namespace in which | |
| the tests are run will be destroyed. This is to prevent inaccurate results | |
| in the test cases. | |
| Repeated failures of the setup/teardown may indicate a problem with the test | |
| case, or possibly even a bug in one of the commands that are not being tested. | |
| It's possible to include acceptable exit codes with the setup/teardown command | |
| so that it doesn't halt the script for an error that doesn't matter. Turn the | |
| individual command into a list, with the command being first, followed by all | |
| acceptable exit codes for the command. | |