Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 1 | == Test API |
| 2 | |
Pau Espin Pedrol | ab78df2 | 2020-06-04 19:45:13 +0200 | [diff] [blame] | 3 | All tests run by {app-name} are python script files. On top of usual python |
| 4 | standard features, {app-name} provides a set of public APIs and tools that |
| 5 | these tests can use in order to interact with the core of {app-name}, like |
| 6 | creating object classes, run processes synchronously or asynchronously, wait for |
| 7 | events, retrieve specific configuration, etc. This section aims at documenting |
| 8 | the most relevant tools available for tests. |
| 9 | |
| 10 | First of all, it is important to avoid blocking out of the core's main event loop in |
| 11 | the test, since doing that will prevent {app-name} core functionalities to work |
| 12 | properly, such as control of asynchronous processes. |
| 13 | |
| 14 | To get access to those functionalities, a test must import a test environment |
| 15 | previously prepared by {app-name} before the test was started: |
| 16 | [source,python] |
| 17 | ---- |
| 18 | #!/usr/bin/env python3 |
| 19 | from osmo_gsm_tester.testenv import * |
| 20 | ---- |
| 21 | |
| 22 | After the test environment is imported, aome usual functionalities are available |
| 23 | directly under the global scope. Specially noticeable is the existence of object |
| 24 | _tenv_, which provides access to most of the functionalities. |
| 25 | |
| 26 | The test can simply ask {app-name} to sleep some time, while giving control back |
| 27 | to {app-name} core's mainloop: |
| 28 | [source,python] |
| 29 | ---- |
| 30 | sleep(3) # sleep for 3 seconds |
| 31 | ---- |
| 32 | |
| 33 | One can also wait for events in the background, for instance launch a child |
| 34 | process locally in the same host and wait for its termination: |
| 35 | [source,python] |
| 36 | ---- |
Pau Espin Pedrol | cedee0a | 2020-06-16 11:39:19 +0200 | [diff] [blame] | 37 | proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4') # <1> |
| 38 | tenv.remember_to_stop(proc) # <2> |
| 39 | proc.launch() # <3> |
| 40 | proc.wait() # <4> |
Pau Espin Pedrol | ab78df2 | 2020-06-04 19:45:13 +0200 | [diff] [blame] | 41 | ---- |
| 42 | <1> Create process object. This line doesn't yet runs it. |
| 43 | <2> Make sure the core will kill the process if this test fails |
| 44 | <3> Start process asynchronously |
| 45 | <4> wait until process is done. One could waiting generically here too: _wait(proc.terminated)_ |
| 46 | |
| 47 | If running asynchronously is not needed, one can run synchronously in an easy |
| 48 | way: |
| 49 | [source,python] |
| 50 | ---- |
| 51 | proc = process.Process('process_description_name', working_dir_to_store_logs, 'sleep 4') |
| 52 | proc.launch_sync() |
| 53 | ---- |
| 54 | |
| 55 | One can also log output using either the regular _print_ function from python, |
| 56 | or using {app-name} specific functions available: |
| 57 | [source,python] |
| 58 | ---- |
| 59 | log('this is a regular log message') |
| 60 | dbg('this is a dbg message, only printed on outputs where dbg is enabled') |
| 61 | err('outputs log message for non-expected events') |
| 62 | print('this is the same as log()') |
| 63 | ---- |
| 64 | |
| 65 | The test also gains access to suite and/or test specific configuration through |
| 66 | different APIs: |
| 67 | [source,python] |
| 68 | ---- |
| 69 | test_config = tenv.config_test_specific() |
| 70 | threshold = int(test_config.get('threshold', 2)) |
| 71 | suite_config = tenv.config_suite_specific() |
| 72 | foobar = suite_config['foobar'] |
| 73 | ---- |
| 74 | |
| 75 | A test requiring a really specific config file for an object class it is going |
| 76 | to run can provide its own template files by overlaying an own directory |
| 77 | containing them on top of the usual default directory where object class |
| 78 | templates are (_osmo-gsm-tester.git/src/osmo_gsm_tester/obj/templates/_): |
| 79 | [source,python] |
| 80 | ---- |
| 81 | tenv.set_overlay_template_dir(os.path.join(os.path.dirname(__file__), 'mytemplatedir')) |
| 82 | ---- |
| 83 | |
| 84 | Several tests in a suite can also share code by using some APIs provided by |
| 85 | {app-names}. The shared python code must be placed in files under the 'lib/' |
| 86 | subdirectory in the suite directory where the test belongs to. |
| 87 | [source,python] |
| 88 | ---- |
| 89 | # File containing function foobar() available under ${suite_dir}/lib/testlib.py: |
| 90 | import testlib |
| 91 | tenv.test_import_modules_register_for_cleanup(testlib) |
| 92 | from testlib import foobar |
| 93 | ---- |
| 94 | |
| 95 | For a complete set of features and how to use them, one can have a look at real |
| 96 | examples present in {app-name} git repository under the _sysmocom/_ directory. |
| 97 | Besides those, have a look too a _testenv.py_ file, which implements the 'tenv' |
| 98 | object available to tests. |
| 99 | |
| 100 | === Test verdict |
| 101 | |
| 102 | In general, a test reaching the end of the file and returning control to |
| 103 | {app-name} core will be flagged as a successful test (PASS). |
| 104 | |
| 105 | If an exception is thrown from within the test file and propagated to |
| 106 | {app-name}, the test will be considered as failed and {app-name} will store all |
| 107 | failure related information from the caught exception. |