Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 1 | == Introduction with Examples |
| 2 | |
| 3 | The osmo-gsm-tester is software to run automated tests of real GSM hardware, |
| 4 | foremost to verify that ongoing Osmocom software development continues to work |
| 5 | with various BTS models, while being flexibly configurable and extendable. |
| 6 | |
| 7 | A 'main unit' (general purpose computer) is connected via ethernet and/or USB to |
| 8 | any number of BTS models and to any number of GSM modems via USB. The modems |
| 9 | and BTS instances' RF transceivers are typically wired directly to each other |
| 10 | via RF distribution chambers to bypass the air medium and avoid disturbing real |
| 11 | production cellular networks. Furthermore, the setup may include adjustable RF |
| 12 | attenuators to model various distances between modems and base stations. |
| 13 | |
| 14 | The osmo-gsm-tester software runs on the main unit to orchestrate the various |
| 15 | GSM hardware and run predefined test scripts. It typically receives binary |
| 16 | packages from a jenkins build service. It then automatically configures and |
| 17 | launches an Osmocom core network on the main unit and sets up and runs BTS |
| 18 | models as well as modems to form a complete ad-hoc GSM network. On this setup, |
| 19 | predefined test suites, combined with various scenario definitions, are run to |
| 20 | verify stability of the system. |
| 21 | |
| 22 | The osmo-gsm-tester is implemented in Python (version 3). It uses the ofono |
| 23 | daemon to control the modems connected via USB. BTS software is either run |
| 24 | directly on the main unit (e.g. for osmo-bts-trx, osmo-bts-octphy), run via SSH |
| 25 | (e.g. for a sysmoBTS) or assumed to run on a connected BTS model (e.g. for |
| 26 | ip.access nanoBTS). |
| 27 | |
| 28 | .Typical osmo-gsm-tester setup |
| 29 | [graphviz] |
| 30 | ---- |
| 31 | digraph G { |
| 32 | rankdir=LR; |
| 33 | jenkins |
| 34 | subgraph cluster_gsm_hardware { |
| 35 | label = "GSM Hardware"; |
| 36 | style=dotted |
| 37 | |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 38 | modem0 [shape=box label="Modems..."] |
| 39 | modem1 [shape=box label="Modems..."] |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 40 | osmo_bts_sysmo [label="sysmocom sysmoBTS\nrunning osmo-bts-sysmo" shape=box] |
| 41 | B200 [label="Ettus B200" shape=box] |
Pau Espin Pedrol | 48d2564 | 2017-11-02 14:03:58 +0100 | [diff] [blame] | 42 | sysmoCell5K [label="sysmocom sysmoCell5000" shape=box] |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 43 | octphy [label="Octasic octphy BTS" shape=box] |
| 44 | nanoBTS [label="ip.access nanoBTS" shape=box] |
| 45 | rf_distribution [label="RF distribution"] |
| 46 | |
Pau Espin Pedrol | 48d2564 | 2017-11-02 14:03:58 +0100 | [diff] [blame] | 47 | {modem0 modem1 osmo_bts_sysmo B200 octphy nanoBTS sysmoCell5K}->rf_distribution [dir=both arrowhead="curve" arrowtail="curve"] |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 48 | } |
| 49 | subgraph cluster_main_unit { |
| 50 | label = "Main Unit" |
| 51 | osmo_gsm_tester [label="Osmo-GSM-Tester\ntest suites\n& scenarios"] |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 52 | subgraph { |
| 53 | rank=same |
| 54 | ofono [label="ofono daemon"] |
Pau Espin Pedrol | 48d2564 | 2017-11-02 14:03:58 +0100 | [diff] [blame] | 55 | osmo_trx [label="osmo-trx"] |
| 56 | osmo_bts_trx [label="osmo-bts-trx"] |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 57 | osmo_bts_octphy [label="osmo-bts-octphy"] |
Pau Espin Pedrol | 48d2564 | 2017-11-02 14:03:58 +0100 | [diff] [blame] | 58 | OsmoNITB [label="BSC + Core Network\n(Osmo{NITB,MSC,BSC,HLR,...})"] |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 59 | } |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 60 | } |
| 61 | |
| 62 | |
| 63 | jenkins->osmo_gsm_tester [label="trial\n(binaries)"] |
| 64 | osmo_gsm_tester->jenkins [label="results"] |
Pau Espin Pedrol | 48d2564 | 2017-11-02 14:03:58 +0100 | [diff] [blame] | 65 | ofono->{modem0 modem1} [label="USB"] |
| 66 | osmo_gsm_tester->{OsmoNITB osmo_bts_trx osmo_bts_octphy} |
| 67 | osmo_gsm_tester->osmo_bts_sysmo [taillabel="SSH"] |
| 68 | osmo_gsm_tester->ofono [taillabel="DBus"] |
| 69 | osmo_trx->B200 [label="USB"] |
| 70 | osmo_bts_trx->{osmo_trx sysmoCell5K} [dir=both label="UDP"] |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 71 | osmo_bts_octphy->octphy [label="raw eth"] |
Pau Espin Pedrol | 48d2564 | 2017-11-02 14:03:58 +0100 | [diff] [blame] | 72 | {osmo_bts_sysmo nanoBTS}->OsmoNITB [label="IP"] |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 73 | {B200 octphy}->OsmoNITB [label="eth" style=invis] |
| 74 | {osmo_bts_trx osmo_bts_octphy}->OsmoNITB |
| 75 | } |
| 76 | ---- |
| 77 | |
| 78 | .Example of how to select resources and configurations: scenarios may pick specific resources (here BTS and ARFCN), remaining requirements are picked as available (here two modems and a NITB interface) |
| 79 | [graphviz] |
| 80 | ---- |
| 81 | digraph G { |
| 82 | rankdir=TB; |
| 83 | |
| 84 | suite_scenarios [label="Suite+Scenarios selection\nsms:sysmo+band1800"] |
| 85 | |
| 86 | subgraph { |
| 87 | rank=same; |
| 88 | suite |
| 89 | scenarios |
| 90 | } |
| 91 | |
| 92 | subgraph cluster_scenarios { |
| 93 | label = "Scenarios"; |
| 94 | u_sysmoBTS [label="Scenario: sysmo\nbts: type: osmo-bts-sysmo"] |
| 95 | u_trx [label="Scenario: trx\nbts: type: osmo-bts-trx"] |
| 96 | u_arfcn [label="Scenario: band1800\narfcn: band: GSM-1800"] |
| 97 | } |
| 98 | |
| 99 | subgraph cluster_suite { |
| 100 | label = "Suite: sms"; |
Neels Hofmeyr | 5d0330f | 2017-05-18 18:38:50 +0200 | [diff] [blame] | 101 | requires [label="Requirements (suite.conf):\nmodem: times: 2\nbts\nip_address\narfcn"] |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 102 | subgraph cluster_tests { |
| 103 | label = "Test Scripts (py)"; |
| 104 | mo_mt_sms |
| 105 | etc |
| 106 | } |
| 107 | } |
| 108 | |
| 109 | subgraph cluster_resources { |
| 110 | label = "Resources"; |
| 111 | rankdir=TB; |
| 112 | nitb_addr1 [label="NITB interface addr\n10.42.42.1"] |
| 113 | nitb_addr2 [label="NITB interface addr\n10.42.42.2"] |
| 114 | Modem0 |
| 115 | Modem1 |
| 116 | Modem2 |
| 117 | sysmoBTS [label="osmo-bts-sysmo"] |
| 118 | osmo_bts_trx [label="osmo-bts-trx"] |
| 119 | arfcn1 [label="arfcn: 512\nband: GSM-1800"] |
| 120 | arfcn2 [label="arfcn: 540\nband: GSM-1900"] |
| 121 | |
| 122 | arfcn1->arfcn2 [style=invis] |
| 123 | nitb_addr1->nitb_addr2 [style=invis] |
| 124 | Modem0 -> Modem1 -> Modem2 [style=invis] |
| 125 | sysmoBTS -> osmo_bts_trx [style=invis] |
| 126 | } |
| 127 | |
| 128 | suite_scenarios -> {suite scenarios} |
| 129 | scenarios -> { u_arfcn u_sysmoBTS } |
| 130 | |
| 131 | suite -> requires |
| 132 | requires -> Modem0 |
| 133 | requires -> Modem1 |
| 134 | requires -> sysmoBTS |
| 135 | requires -> arfcn1 |
| 136 | requires -> nitb_addr1 |
| 137 | |
| 138 | { u_sysmoBTS u_arfcn } -> requires [label="influences\nresource\nselection"] |
| 139 | } |
| 140 | ---- |
| 141 | |
| 142 | .Example of a "trial" containing binaries built by a jenkins |
| 143 | [graphviz] |
| 144 | ---- |
| 145 | digraph G { |
| 146 | subgraph cluster_trial { |
| 147 | label = "Trial (binaries)" |
| 148 | sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"] |
Pau Espin Pedrol | cb47407 | 2017-11-02 14:13:54 +0100 | [diff] [blame] | 149 | trx [label="osmo-bts.build-5.tgz\n(osmo-bts-octphy + osmo-bts-trx\n+ deps\ncompiled for main unit)"] |
Neels Hofmeyr | 73207e1 | 2017-05-05 02:50:45 +0200 | [diff] [blame] | 150 | nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"] |
| 151 | checksums [label="checksums.md5"] |
| 152 | |
| 153 | checksums -> {sysmo trx nitb} |
| 154 | } |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 155 | } |
| 156 | ---- |
| 157 | |
| 158 | === Typical Test Script |
| 159 | |
| 160 | A typical single test script (part of a suite) may look like this: |
| 161 | |
| 162 | ---- |
| 163 | #!/usr/bin/env python3 |
Pau Espin Pedrol | 4c47659 | 2017-11-17 17:16:54 +0100 | [diff] [blame] | 164 | from osmo_gsm_tester.testenv import * |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 165 | |
Pau Espin Pedrol | c8cd1fa | 2017-11-02 14:15:46 +0100 | [diff] [blame] | 166 | hlr = suite.hlr() |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 167 | bts = suite.bts() |
Pau Espin Pedrol | c8cd1fa | 2017-11-02 14:15:46 +0100 | [diff] [blame] | 168 | mgcpgw = suite.mgcpgw(bts_ip=bts.remote_addr()) |
| 169 | msc = suite.msc(hlr, mgcpgw) |
| 170 | bsc = suite.bsc(msc) |
| 171 | stp = suite.stp() |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 172 | ms_mo = suite.modem() |
| 173 | ms_mt = suite.modem() |
| 174 | |
Pau Espin Pedrol | c8cd1fa | 2017-11-02 14:15:46 +0100 | [diff] [blame] | 175 | hlr.start() |
| 176 | stp.start() |
| 177 | msc.start() |
| 178 | mgcpgw.start() |
| 179 | |
| 180 | bsc.bts_add(bts) |
| 181 | bsc.start() |
| 182 | |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 183 | bts.start() |
| 184 | |
Pau Espin Pedrol | c8cd1fa | 2017-11-02 14:15:46 +0100 | [diff] [blame] | 185 | hlr.subscriber_add(ms_mo) |
| 186 | hlr.subscriber_add(ms_mt) |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 187 | |
Pau Espin Pedrol | c8cd1fa | 2017-11-02 14:15:46 +0100 | [diff] [blame] | 188 | ms_mo.connect(msc.mcc_mnc()) |
| 189 | ms_mt.connect(msc.mcc_mnc()) |
| 190 | |
| 191 | ms_mo.log_info() |
| 192 | ms_mt.log_info() |
Pau Espin Pedrol | 422f8ba | 2017-06-14 15:57:12 +0200 | [diff] [blame] | 193 | |
| 194 | print('waiting for modems to attach...') |
Pau Espin Pedrol | c8cd1fa | 2017-11-02 14:15:46 +0100 | [diff] [blame] | 195 | wait(ms_mo.is_connected, msc.mcc_mnc()) |
| 196 | wait(ms_mt.is_connected, msc.mcc_mnc()) |
| 197 | wait(msc.subscriber_attached, ms_mo, ms_mt) |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 198 | |
Pau Espin Pedrol | 422f8ba | 2017-06-14 15:57:12 +0200 | [diff] [blame] | 199 | sms = ms_mo.sms_send(ms_mt) |
| 200 | wait(ms_mt.sms_was_received, sms) |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 201 | ---- |
| 202 | |
| 203 | === Resource Resolution |
| 204 | |
Pau Espin Pedrol | dc53c9c | 2017-11-02 14:19:32 +0100 | [diff] [blame] | 205 | - A global configuration 'resources.conf' defines which hardware is connected to the |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 206 | osmo-gsm-tester main unit. |
| 207 | - Each suite contains a number of test scripts. The amount of resources a test |
| 208 | may use is defined by the test suite's 'suite.conf'. |
| 209 | - Which specific modems, BTS models, NITB IP addresses etc. are made available |
Pau Espin Pedrol | dc53c9c | 2017-11-02 14:19:32 +0100 | [diff] [blame] | 210 | to a test run is typically determined by 'suite.conf' and a combination of scenario |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 211 | configurations -- or picked automatically if not. |
| 212 | |
| 213 | [[resources_conf_example]] |
| 214 | === Typical 'resources.conf' |
| 215 | |
| 216 | A global configuration of hardware may look like below; for details, see |
| 217 | <<resources_conf>>. |
| 218 | |
| 219 | ---- |
Neels Hofmeyr | 5d0330f | 2017-05-18 18:38:50 +0200 | [diff] [blame] | 220 | ip_address: |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 221 | - addr: 10.42.42.2 |
| 222 | - addr: 10.42.42.3 |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 223 | - addr: 10.42.42.4 |
| 224 | - addr: 10.42.42.5 |
| 225 | - addr: 10.42.42.6 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 226 | |
| 227 | bts: |
| 228 | - label: sysmoBTS 1002 |
| 229 | type: osmo-bts-sysmo |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 230 | ipa_unit_id: 1 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 231 | addr: 10.42.42.114 |
| 232 | band: GSM-1800 |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 233 | ciphers: |
| 234 | - a5_0 |
| 235 | - a5_1 |
| 236 | - a5_3 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 237 | |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 238 | - label: Ettus B200 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 239 | type: osmo-bts-trx |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 240 | ipa_unit_id: 6 |
| 241 | addr: 10.42.42.50 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 242 | band: GSM-1800 |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 243 | launch_trx: true |
| 244 | ciphers: |
| 245 | - a5_0 |
| 246 | - a5_1 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 247 | |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 248 | - label: sysmoCell 5000 |
| 249 | type: osmo-bts-trx |
| 250 | ipa_unit_id: 7 |
| 251 | addr: 10.42.42.51 |
| 252 | band: GSM-1800 |
| 253 | trx_remote_ip: 10.42.42.112 |
| 254 | ciphers: |
| 255 | - a5_0 |
| 256 | - a5_1 |
| 257 | |
| 258 | - label: OCTBTS 3500 |
| 259 | type: osmo-bts-octphy |
| 260 | ipa_unit_id: 8 |
| 261 | addr: 10.42.42.52 |
| 262 | band: GSM-1800 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 263 | trx_list: |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 264 | - hw_addr: 00:0c:90:2e:80:1e |
| 265 | net_device: eth1 |
| 266 | - hw_addr: 00:0c:90:2e:87:52 |
| 267 | net_device: eth1 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 268 | |
| 269 | arfcn: |
| 270 | - arfcn: 512 |
| 271 | band: GSM-1800 |
| 272 | - arfcn: 514 |
| 273 | band: GSM-1800 |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 274 | - arfcn: 516 |
| 275 | band: GSM-1800 |
| 276 | - arfcn: 546 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 277 | band: GSM-1900 |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 278 | - arfcn: 548 |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 279 | band: GSM-1900 |
| 280 | |
| 281 | modem: |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 282 | - label: sierra_1 |
| 283 | path: '/sierra_1' |
| 284 | imsi: '901700000009031' |
| 285 | ki: '80A37E6FDEA931EAC92FFA5F671EFEAD' |
| 286 | auth_algo: 'xor' |
| 287 | ciphers: |
| 288 | - a5_0 |
| 289 | - a5_1 |
| 290 | features: |
| 291 | - 'sms' |
| 292 | - 'voice' |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 293 | |
Pau Espin Pedrol | c9cbbc6 | 2017-11-02 14:23:35 +0100 | [diff] [blame] | 294 | - label: gobi_0 |
| 295 | path: '/gobi_0' |
| 296 | imsi: '901700000009030' |
| 297 | ki: 'BB70807226393CDBAC8DD3439FF54252' |
| 298 | auth_algo: 'xor' |
| 299 | ciphers: |
| 300 | - a5_0 |
| 301 | - a5_1 |
| 302 | features: |
| 303 | - 'sms' |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 304 | ---- |
| 305 | |
| 306 | === Typical 'suites/*/suite.conf' |
| 307 | |
| 308 | The configuration that reserves a number of resources for a test suite may look |
| 309 | like this: |
| 310 | |
| 311 | ---- |
| 312 | resources: |
Neels Hofmeyr | 5d0330f | 2017-05-18 18:38:50 +0200 | [diff] [blame] | 313 | ip_address: |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 314 | - times: 1 |
| 315 | bts: |
| 316 | - times: 1 |
| 317 | modem: |
| 318 | - times: 2 |
Pau Espin Pedrol | 3ed0391 | 2017-11-02 14:25:39 +0100 | [diff] [blame] | 319 | features: |
| 320 | - sms |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 321 | ---- |
| 322 | |
| 323 | It may also request e.g. specific BTS models, but this is typically left to |
| 324 | scenario configurations. |
| 325 | |
| 326 | === Typical 'scenarios/*.conf' |
| 327 | |
| 328 | For a suite as above run as-is, any available resources are picked. This may be |
| 329 | combined with any number of scenario definitions to constrain which specific |
| 330 | resources should be used, e.g.: |
| 331 | |
| 332 | ---- |
| 333 | resources: |
| 334 | bts: |
| 335 | - type: osmo-bts-sysmo |
| 336 | ---- |
| 337 | |
Neels Hofmeyr | 5d0330f | 2017-05-18 18:38:50 +0200 | [diff] [blame] | 338 | Which 'ip_address' or 'modem' is used in particular doesn't really matter, so |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 339 | it can be left up to the osmo-gsm-tester to pick these automatically. |
| 340 | |
| 341 | Any number of such scenario configurations can be combined in the form |
| 342 | '<suite_name>:<scenario>+<scenario>+...', e.g. 'my_suite:sysmo+tch_f+amr'. |
| 343 | |
| 344 | === Typical Invocations |
| 345 | |
| 346 | Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for |
| 347 | the Osmocom core network as well as for the Osmocom based BTS models. To create |
| 348 | such a set of binaries, see <<trials>>. |
| 349 | |
| 350 | Examples for launching test trials: |
| 351 | |
| 352 | - Run the default suites (see <<default_suites>>) on a given set of binaries: |
| 353 | |
| 354 | ---- |
| 355 | osmo-gsm-tester.py path/to/my-trial |
| 356 | ---- |
| 357 | |
| 358 | - Run an explicit choice of 'suite:scenario' combinations: |
| 359 | |
| 360 | ---- |
| 361 | osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts |
| 362 | ---- |
| 363 | |
| 364 | - Run one 'suite:scenario' combination, setting log level to 'debug' and |
| 365 | enabling logging of full python tracebacks, and also only run just the |
| 366 | 'mo_mt_sms.py' test from the suite, e.g. to investigate a test failure: |
| 367 | |
| 368 | ---- |
| 369 | osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -l dbg -T -t mo_mt |
| 370 | ---- |
| 371 | |
| 372 | A test script may also be run step-by-step in a python debugger, see |
| 373 | <<debugging>>. |
| 374 | |
| 375 | === Resource Reservation for Concurrent Trials |
| 376 | |
| 377 | While a test suite runs, the used resources are noted in a global state |
| 378 | directory in a reserved-resources file. This way, any number of trials may be |
| 379 | run consecutively without resource conflicts. Any test trial will only use |
| 380 | resources that are currently not reserved by any other test suite. The |
| 381 | reservation state is human readable. |
| 382 | |
| 383 | The global state directory is protected by a file lock to allow access by |
| 384 | separate processes. |
| 385 | |
| 386 | Also, the binaries from a trial are never installed system-wide, but are run |
| 387 | with a specific 'LD_LIBRARY_PATH' pointing at the trial's 'inst', so that |
Pau Espin Pedrol | 627aea5 | 2017-11-02 15:03:58 +0100 | [diff] [blame] | 388 | several trials can run consecutively without conflicting binary versions. For |
| 389 | some specific binaries which require extra permissions (such as osmo-bts-octphy |
| 390 | requiring 'CAP_NET_RAW'), 'patchelf' program is used to modify the binary |
| 391 | 'RPATH' field instead because the OS dynamic linker skips 'LD_LIBRARY_PATH' for |
| 392 | binaries with special permissions. |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 393 | |
| 394 | Once a test suite run is complete, all its reserved resources are torn down (if |
| 395 | the test scripts have not done so already), and the reservations are released |
| 396 | automatically. |
| 397 | |
| 398 | If required resources are unavailable, the test trial fails. For consecutive |
| 399 | test trials, a test run needs to either wait for resources to become available, |
| 400 | or test suites need to be scheduled to make sense. (*<- TODO*) |