Pau Espin Pedrol | 06376b7 | 2020-05-05 18:39:44 +0200 | [diff] [blame] | 1 | == Installation |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 2 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 3 | === Trial Builder |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 4 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 5 | The Trial Builder is the jenkins build slave (host) building all sysroot binary |
| 6 | packages used later by {app-name} to run the tests. It's purpose is to build the |
| 7 | sysroots and provide them to {app-anme}, for instance, as jenkins job artifacts |
| 8 | which the {app-name} runner job can fetch. |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 9 | |
| 10 | [[jenkins_deps]] |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 11 | ==== Osmocom Build Dependencies |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 12 | |
| 13 | Each of the jenkins builds requires individual dependencies. This is generally |
| 14 | the same as for building the software outside of osmo-gsm-tester and will not |
| 15 | be detailed here. For the Osmocom projects, refer to |
| 16 | http://osmocom.org/projects/cellular-infrastructure/wiki/Build_from_Source . Be |
| 17 | aware of specific requirements for BTS hardware: for example, the |
| 18 | osmo-bts-sysmo build needs the sysmoBTS SDK installed on the build slave, which |
| 19 | should match the installed sysmoBTS firmware. |
| 20 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 21 | ==== Add Build Jobs |
| 22 | |
| 23 | There are various jenkins-build-* scripts in osmo-gsm-tester/contrib/, which |
| 24 | can be called as jenkins build jobs to build and bundle binaries as artifacts, |
| 25 | to be run on the osmo-gsm-tester main unit and/or BTS hardware. |
| 26 | |
| 27 | Be aware of the dependencies, as hinted at in <<jenkins_deps>>. |
| 28 | |
| 29 | While the various binaries could technically be built on the osmo-gsm-tester |
| 30 | main unit, it is recommended to use a separate build slave, to take load off |
| 31 | of the main unit. |
| 32 | |
| 33 | Please note nowadays we set up all the osmocom jenkins jobs (including |
| 34 | {app-name} ones) using 'jenkins-job-builder'. You can find all the |
| 35 | configuration's in Osmocom's 'osmo-ci.git' files 'jobs/osmo-gsm-tester-*.yml. |
| 36 | Explanation below on how to set up jobs manually is left as a reference for |
| 37 | other projects. |
| 38 | |
| 39 | On your jenkins master, set up build jobs to call these scripts -- typically |
| 40 | one build job per script. Look in contrib/ and create one build job for each of |
| 41 | the BTS types you would like to test, as well as one for the 'build-osmo-nitb'. |
| 42 | |
| 43 | These are generic steps to configure a jenkins build |
| 44 | job for each of these build scripts, by example of the |
| 45 | jenkins-build-osmo-nitb.sh script; all that differs to the other scripts is the |
| 46 | "osmo-nitb" part: |
| 47 | |
| 48 | * 'Project name': "osmo-gsm-tester_build-osmo-nitb" + |
| 49 | (Replace 'osmo-nitb' according to which build script this is for) |
| 50 | * 'Discard old builds' + |
| 51 | Configure this to taste, for example: |
| 52 | ** 'Max # of build to keep': "20" |
| 53 | * 'Restrict where this project can be run': Choose a build slave label that |
| 54 | matches the main unit's architecture and distribution, typically a Debian |
| 55 | system, e.g.: "linux_amd64_debian8" |
| 56 | * 'Source Code Management': |
| 57 | ** 'Git' |
| 58 | *** 'Repository URL': "git://git.osmocom.org/osmo-gsm-tester" |
| 59 | *** 'Branch Specifier': "*/master" |
| 60 | *** 'Additional Behaviors' |
| 61 | **** 'Check out to a sub-directory': "osmo-gsm-tester" |
| 62 | * 'Build Triggers' + |
| 63 | The decision on when to build is complex. Here are some examples: |
| 64 | ** Once per day: + |
| 65 | 'Build periodically': "H H * * *" |
| 66 | ** For the Osmocom project, the purpose is to verify our software changes. |
| 67 | Hence we would like to test every time our code has changed: |
| 68 | *** We could add various git repositories to watch, and enable 'Poll SCM'. |
| 69 | *** On jenkins.osmocom.org, we have various jobs that build the master branches |
| 70 | of their respective git repositories when a new change was merged. Here, we |
| 71 | can thus trigger e.g. an osmo-nitb build for osmo-gsm-tester everytime the |
| 72 | master build has run: + |
| 73 | 'Build after other projects are built': "OpenBSC" |
| 74 | *** Note that most of the Osmocom projects also need to be re-tested when their |
| 75 | dependencies like libosmo* have changed. Triggering on all those changes |
| 76 | typically causes more jenkins runs than necessary: for example, it rebuilds |
| 77 | once per each dependency that has rebuilt due to one libosmocore change. |
| 78 | There is so far no trivial way known to avoid this. It is indeed safest to |
| 79 | rebuild more often. |
| 80 | * 'Build' |
| 81 | ** 'Execute Shell' |
| 82 | + |
| 83 | ---- |
| 84 | #!/bin/sh |
| 85 | set -e -x |
| 86 | ./osmo-gsm-tester/contrib/jenkins-build-osmo-nitb.sh |
| 87 | ---- |
| 88 | + |
| 89 | (Replace 'osmo-nitb' according to which build script this is for) |
| 90 | |
| 91 | * 'Post-build Actions' |
| 92 | ** 'Archive the artifacts': "*.tgz, *.md5" + |
| 93 | (This step is important to be able to use the built binaries in the run job |
| 94 | below.) |
| 95 | |
| 96 | |
| 97 | TIP: When you've created one build job, it is convenient to create further |
| 98 | build jobs by copying the first one and, e.g., simply replacing all "osmo-nitb" |
| 99 | with "osmo-bts-trx". |
| 100 | |
| 101 | [[install_main_unit]] |
| 102 | === Main Unit |
| 103 | |
| 104 | The main unit is a general purpose computer that orchestrates the tests. It |
| 105 | runs the core network components, controls the modems and so on. This can be |
| 106 | anything from a dedicated production rack unit to your laptop at home. |
| 107 | |
| 108 | This manual will assume that tests are run from a jenkins build slave, by a user |
| 109 | named 'jenkins' that belongs to group 'osmo-gsm-tester'. The user configuration |
| 110 | for manual test runs and/or a different user name is identical, simply replace |
| 111 | the user name or group. |
| 112 | |
| 113 | Please, note installation steps and dependencies needed will depend on lots of |
| 114 | factors, like your distribution, your specific setup, which hardware you plan to |
| 115 | support, etc. |
| 116 | |
| 117 | This section aims at being one place to document the rationale behind certain |
| 118 | configurations being done in one way or another. For an up to date step by step |
| 119 | detailed way to install and maintain the Osmocom {app-name} setup, one will want |
| 120 | to look at the <<ansible,ansible scripts section>>. |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 121 | |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 122 | [[configure_jenkins_slave]] |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 123 | ==== Create 'jenkins' User |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 124 | |
| 125 | On the main unit, create a jenkins user: |
| 126 | |
| 127 | ---- |
| 128 | useradd -m jenkins |
| 129 | ---- |
| 130 | |
Neels Hofmeyr | 3f596ed | 2017-07-14 00:04:40 +0200 | [diff] [blame] | 131 | ==== Install Java on Main Unit |
| 132 | |
| 133 | To be able to launch the Jenkins build slave, a Java RE must be available on |
| 134 | the main unit. For example: |
| 135 | |
| 136 | ---- |
| 137 | apt-get install default-jdk |
| 138 | ---- |
| 139 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 140 | ==== Allow SSH Access from Jenkins Master |
| 141 | |
| 142 | Create an SSH keypair to be used for login on the osmo-gsm-tester. This may be |
| 143 | entered on the jenkins web UI; alternatively, use the jenkins server's shell: |
| 144 | |
| 145 | Login on the main jenkins server shell and create an SSH keypair, for example: |
| 146 | |
| 147 | ---- |
| 148 | # su jenkins |
Neels Hofmeyr | 3f596ed | 2017-07-14 00:04:40 +0200 | [diff] [blame] | 149 | $ mkdir -p /usr/local/jenkins/keys |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 150 | $ ssh-keygen |
| 151 | Generating public/private rsa key pair. |
| 152 | Enter file in which to save the key (/home/jenkins/.ssh/id_rsa): /usr/local/jenkins/keys/osmo-gsm-tester-rnd |
| 153 | Enter passphrase (empty for no passphrase): <enter a passphrase> |
| 154 | Enter same passphrase again: <enter a passphrase> |
| 155 | Your identification has been saved in /usr/local/jenkins/keys/osmo-gsm-tester-rnd |
| 156 | Your public key has been saved in /usr/local/jenkins/keys/osmo-gsm-tester-rnd.pub. |
| 157 | The key fingerprint is: |
| 158 | ... |
| 159 | ---- |
| 160 | |
| 161 | Copy the public key to the main unit, e.g. copy-paste: |
| 162 | |
| 163 | ---- |
| 164 | cat /usr/local/jenkins/keys/osmo-gsm-tester-rnd.pub |
| 165 | # copy this public key |
| 166 | ---- |
| 167 | |
| 168 | On the main unit: |
| 169 | |
| 170 | ---- |
| 171 | mkdir ~jenkins/.ssh |
| 172 | cat > ~jenkins/.ssh/authorized_keys |
| 173 | # paste above public key and hit Ctrl-D |
| 174 | chown -R jenkins: ~jenkins/.ssh |
| 175 | ---- |
| 176 | |
| 177 | Make sure that the user running the jenkins master accepts the main unit's host |
| 178 | identification. There must be an actual RSA host key available in the |
| 179 | known_hosts file for the jenkins master to be able to log in. Simply calling |
| 180 | ssh and accepting the host key as usual is not enough. Jenkins may continue to |
| 181 | say "Host key verification failed". |
| 182 | |
| 183 | To place an RSA host key in the jenkins' known_hosts file, you may do: |
| 184 | |
| 185 | On the Jenkins master: |
| 186 | |
| 187 | ---- |
| 188 | main_unit_ip=10.9.8.7 |
| 189 | ssh-keyscan -H $main_unit_ip >> ~jenkins/.ssh/known_hosts |
| 190 | chown jenkins: ~jenkins/.ssh/known_hosts |
| 191 | ---- |
| 192 | |
| 193 | Verify that the jenkins user on the Jenkins master has SSH access to the main |
| 194 | unit: |
| 195 | |
| 196 | ---- |
| 197 | su jenkins |
| 198 | main_unit_ip=10.9.8.7 |
Neels Hofmeyr | 3f596ed | 2017-07-14 00:04:40 +0200 | [diff] [blame] | 199 | ssh -i /usr/local/jenkins/keys/osmo-gsm-tester-rnd jenkins@$main_unit_ip |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 200 | exit |
| 201 | ---- |
| 202 | |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 203 | [[install_add_jenkins_slave]] |
| 204 | ==== Add Jenkins Slave |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 205 | |
| 206 | In the jenkins web UI, add a new build slave for the osmo-gsm-tester: |
| 207 | |
| 208 | * 'Manage Jenkins' |
| 209 | ** 'Manage Nodes' |
| 210 | *** 'New Node' |
| 211 | **** Enter a node name, e.g. "osmo-gsm-tester-1" + |
| 212 | (the "-1" is just some identification in case you'd like to add another |
| 213 | setup later). |
| 214 | **** 'Permanent Agent' |
| 215 | |
| 216 | Configure the node as: |
| 217 | |
| 218 | * '# of executors': 1 |
| 219 | * 'Remote root directory': "/home/jenkins" |
| 220 | * 'Labels': "osmo-gsm-tester" + |
| 221 | (This is a general label common to all osmo-gsm-tester build slaves you may set up in the future.) |
| 222 | * 'Usage': 'Only build jobs with label expressions matching this node' |
| 223 | * 'Launch method': 'Launch slave agents via SSH' |
| 224 | ** 'Host': your main unit's IP address |
| 225 | ** 'Credentials': choose 'Add' / 'Jenkins' |
| 226 | *** 'Domain': 'Global credentials (unrestricted)' |
| 227 | *** 'Kind': 'SSH Username with private key' |
| 228 | *** 'Scope': 'Global' |
| 229 | *** 'Username': "jenkins" + |
| 230 | (as created on the main unit above) |
| 231 | *** 'Private Key': 'From a file on Jenkins master' |
| 232 | **** 'File': "/usr/local/jenkins/keys/osmo-gsm-tester-rnd" |
| 233 | *** 'Passphrase': enter same passphrase as above |
| 234 | *** 'ID': "osmo-gsm-tester-1" |
| 235 | *** 'Name': "jenkins for SSH to osmo-gsm-tester-1" |
| 236 | |
| 237 | The build slave should be able to start now. |
| 238 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 239 | ==== Add Run Job |
| 240 | |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 241 | This is the jenkins job that runs the tests on the GSM hardware: |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 242 | |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 243 | * It sources the artifacts from jenkins' build jobs. |
| 244 | * It runs on the osmo-gsm-tester main unit. |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 245 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 246 | Sample script to run {app-name} as a jenkins job can be found in |
| 247 | 'osmo-gsm-tester.git' file 'contrib/jenkins-run.sh'. |
| 248 | |
| 249 | Please note nowadays we set up all the osmocom jenkins jobs (including |
| 250 | {app-name} ones) using 'jenkins-job-builder'. You can find all the |
| 251 | configuration's in Osmocom's 'osmo-ci.git' files 'jobs/osmo-gsm-tester-*.yml. |
| 252 | Explanation below on how to set up jobs manually is left as a reference for |
| 253 | other projects. |
| 254 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 255 | Here is the configuration for the run job: |
| 256 | |
| 257 | * 'Project name': "osmo-gsm-tester_run" |
| 258 | * 'Discard old builds' + |
| 259 | Configure this to taste, for example: |
| 260 | ** 'Max # of build to keep': "20" |
| 261 | * 'Restrict where this project can be run': "osmo-gsm-tester" + |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 262 | (to match the 'Label' configured in <<install_add_jenkins_slave>>). |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 263 | * 'Source Code Management': |
| 264 | ** 'Git' |
| 265 | *** 'Repository URL': "git://git.osmocom.org/osmo-gsm-tester" |
| 266 | *** 'Branch Specifier': "*/master" |
| 267 | *** 'Additional Behaviors' |
| 268 | **** 'Check out to a sub-directory': "osmo-gsm-tester" |
| 269 | **** 'Clean before checkout' |
| 270 | * 'Build Triggers' + |
| 271 | The decision on when to build is complex. For this run job, it is suggested |
| 272 | to rebuild: |
| 273 | ** after each of above build jobs that produced new artifacts: + |
| 274 | 'Build after other projects are built': "osmo-gsm-tester_build-osmo-nitb, |
| 275 | osmo-gsm-tester_build-osmo-bts-sysmo, osmo-gsm-tester_build-osmo-bts-trx" + |
| 276 | (Add each build job name you configured above) |
| 277 | ** as well as once per day: + |
| 278 | 'Build periodically': "H H * * *" |
| 279 | ** and, in addition, whenever the osmo-gsm-tester scripts have been modified: + |
| 280 | 'Poll SCM': "H/5 * * * *" + |
| 281 | (i.e. look every five minutes whether the upstream git has changed) |
| 282 | * 'Build' |
| 283 | ** Copy artifacts from each build job you have set up: |
| 284 | *** 'Copy artifacts from another project' |
| 285 | **** 'Project name': "osmo-gsm-tester_build-osmo-nitb" |
| 286 | **** 'Which build': 'Latest successful build' |
| 287 | **** enable 'Stable build only' |
| 288 | **** 'Artifacts to copy': "*.tgz, *.md5" |
| 289 | *** Add a separate similar 'Copy artifacts...' section for each build job you |
| 290 | have set up. |
| 291 | ** 'Execute Shell' |
| 292 | + |
| 293 | ---- |
| 294 | #!/bin/sh |
| 295 | set -e -x |
| 296 | |
| 297 | # debug: provoke a failure |
| 298 | #export OSMO_GSM_TESTER_OPTS="-s debug -t fail" |
| 299 | |
| 300 | PATH="$PWD/osmo-gsm-tester/src:$PATH" \ |
| 301 | ./osmo-gsm-tester/contrib/jenkins-run.sh |
| 302 | ---- |
| 303 | + |
| 304 | Details: |
| 305 | |
| 306 | *** The 'jenkins-run.sh' script assumes to find the 'osmo-gsm-tester.py' in the |
| 307 | '$PATH'. To use the most recent osmo-gsm-tester code here, we direct |
| 308 | '$PATH' to the actual workspace checkout. This could also run from a sytem |
| 309 | wide install, in which case you could omit the explicit PATH to |
| 310 | "$PWD/osmo-gsm-tester/src". |
| 311 | *** This assumes that there are configuration files for osmo-gsm-tester placed |
| 312 | on the system (see <<config_paths>>). |
| 313 | *** If you'd like to check the behavior of test failures, you can uncomment the |
| 314 | line below "# debug" to produce a build failure on every run. Note that |
| 315 | this test typically produces a quite empty run result, since it launches no |
| 316 | NITB nor BTS. |
| 317 | * 'Post-build Actions' |
| 318 | ** 'Archive the artifacts' |
Neels Hofmeyr | 8d8a136 | 2017-06-18 02:31:38 +0200 | [diff] [blame] | 319 | *** 'Files to archive': "*-run.tgz, *-bin.tgz" + |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 320 | This stores the complete test report with config files, logs, stdout/stderr |
Neels Hofmeyr | 8d8a136 | 2017-06-18 02:31:38 +0200 | [diff] [blame] | 321 | output, pcaps as well as the binaries used for the test run in artifacts. |
| 322 | This allows analysis of older builds, instead of only the most recent build |
| 323 | (which cleans up the jenkins workspace every time). The 'trial-N-run.tgz' |
| 324 | and 'trial-N-bin.tgz' archives are produced by the 'jenkins-run.sh' script, |
| 325 | both for successful and failing runs. |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 326 | |
Pau Espin Pedrol | 0baa5c7 | 2020-05-05 19:08:13 +0200 | [diff] [blame] | 327 | ==== Install osmo-gsm-tester dependencies |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 328 | |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 329 | This assumes you have already created the jenkins user (see <<configure_jenkins_slave>>). |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 330 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 331 | Dependencies needed will depend on lots of factors, like your distribution, your |
| 332 | specific setup, which hardware you plan to support, etc. |
| 333 | |
Pau Espin Pedrol | 0baa5c7 | 2020-05-05 19:08:13 +0200 | [diff] [blame] | 334 | On a Debian/Ubuntu based system, these commands install the mandatory packages |
| 335 | needed to run the osmo-gsm-tester.py code, i.e. install these on your main unit: |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 336 | |
| 337 | ---- |
| 338 | apt-get install \ |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 339 | python3 \ |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 340 | python3-yaml \ |
| 341 | python3-mako \ |
| 342 | python3-gi \ |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 343 | locales |
Pau Espin Pedrol | 0baa5c7 | 2020-05-05 19:08:13 +0200 | [diff] [blame] | 344 | ---- |
| 345 | |
| 346 | If one plans to use the 2G ESME (_esme.py_), following extra dependencies shall |
| 347 | be installed: |
| 348 | ---- |
| 349 | apt-get install python3-setuptools python3-pip |
| 350 | pip3 install "git+https://github.com/podshumok/python-smpplib.git@master#egg=smpplib" |
| 351 | ---- |
| 352 | |
| 353 | If one plans to use the 2G OsmoHLR (_hlr_osmo.py_), following extra dependencies shall |
| 354 | be installed: |
| 355 | ---- |
| 356 | apt-get install sqlite3 |
| 357 | ---- |
| 358 | |
| 359 | If one plans to use SISPM power supply hardware (_powersupply_sispm.py_), |
| 360 | following extra dependencies shall be installed: |
| 361 | ---- |
| 362 | apt-get install python3-setuptools python3-pip |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 363 | pip3 install \ |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 364 | pyusb \ |
| 365 | pysispm |
| 366 | ---- |
| 367 | |
Pau Espin Pedrol | 0baa5c7 | 2020-05-05 19:08:13 +0200 | [diff] [blame] | 368 | If one plans to use software-based RF emulation on Amarisoft ENB implemented |
| 369 | through its CTRL interface (_rfemu_amarisoftctrl.py_), following extra |
| 370 | dependencies shall be installed: |
| 371 | ---- |
| 372 | apt-get install python3-websocket |
| 373 | ---- |
| 374 | |
| 375 | If one plans to use srsLTE UE metrics subsystems (_ms_srs.py_), following extra |
| 376 | dependencies shall be installed: |
| 377 | ---- |
| 378 | apt-get install python3-numpy |
| 379 | ---- |
| 380 | |
| 381 | If one plans to use ofono modems (_ms_ofono.py_), following extra dependencies |
| 382 | shall be installed: |
| 383 | ---- |
| 384 | apt-get install \ |
| 385 | dbus \ |
| 386 | python3 \ |
| 387 | ofono \ |
| 388 | python3-pip \ |
| 389 | udhcpc |
| 390 | pip3 install \ |
| 391 | pydbus |
| 392 | ---- |
| 393 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 394 | IMPORTANT: ofono may need to be installed from source to contain the most |
| 395 | recent fixes needed to operate your modems. This depends on the modem hardware |
| 396 | used and the tests run. Please see <<hardware_modems>>. |
| 397 | |
Pau Espin Pedrol | 0baa5c7 | 2020-05-05 19:08:13 +0200 | [diff] [blame] | 398 | Finally, these programs are usually required by osmo-gsm-tester on the Slave Unit to run and manage processes: |
| 399 | |
| 400 | ---- |
| 401 | apt-get install \ |
| 402 | tcpdump \ |
| 403 | patchelf \ |
| 404 | sudo \ |
| 405 | libcap2-bin \ |
| 406 | iperf3 |
| 407 | ---- |
| 408 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 409 | ==== User Permissions |
| 410 | |
| 411 | On the main unit, create a group for all users that should be allowed to use |
| 412 | the osmo-gsm-tester, and add users (here 'jenkins') to this group. |
| 413 | |
| 414 | ---- |
| 415 | groupadd osmo-gsm-tester |
| 416 | gpasswd -a jenkins osmo-gsm-tester |
| 417 | ---- |
| 418 | |
| 419 | NOTE: you may also need to add users to the 'usrp' group, see |
| 420 | <<user_config_uhd>>. |
| 421 | |
| 422 | A user added to a group needs to re-login for the group permissions to take |
| 423 | effect. |
| 424 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 425 | ===== Paths |
| 426 | |
| 427 | Assuming that you are using the example config, prepare a system wide state |
| 428 | location in '/var/tmp': |
| 429 | |
| 430 | ---- |
| 431 | mkdir -p /var/tmp/osmo-gsm-tester/state |
| 432 | chown -R :osmo-gsm-tester /var/tmp/osmo-gsm-tester |
| 433 | chmod -R g+rwxs /var/tmp/osmo-gsm-tester |
| 434 | setfacl -d -m group:osmo-gsm-tester:rwx /var/tmp/osmo-gsm-tester/state |
| 435 | ---- |
| 436 | |
| 437 | IMPORTANT: the state directory needs to be shared between all users potentially |
| 438 | running the osmo-gsm-tester to resolve resource allocations. Above 'setfacl' |
| 439 | command sets the access control to keep all created files group writable. |
| 440 | |
| 441 | With the jenkins build as described here, the trials will live in the build |
| 442 | slave's workspace. Other modes of operation (a daemon scheduling concurrent |
| 443 | runs, *TODO*) may use a system wide directory to manage trials to run: |
| 444 | |
| 445 | ---- |
| 446 | mkdir -p /var/tmp/osmo-gsm-tester/trials |
| 447 | chown -R :osmo-gsm-tester /var/tmp/osmo-gsm-tester |
| 448 | chmod -R g+rwxs /var/tmp/osmo-gsm-tester |
| 449 | ---- |
| 450 | |
| 451 | ===== Allow DBus Access to ofono |
| 452 | |
| 453 | Put a DBus configuration file in place that allows the 'osmo-gsm-tester' group |
| 454 | to access the org.ofono DBus path: |
| 455 | |
| 456 | ---- |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 457 | # cat > /etc/dbus-1/system.d/osmo-gsm-tester.conf <<END |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 458 | <!-- Additional rules for the osmo-gsm-tester to access org.ofono from user |
| 459 | land --> |
| 460 | |
| 461 | <!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN" |
| 462 | "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"> |
| 463 | <busconfig> |
| 464 | |
| 465 | <policy group="osmo-gsm-tester"> |
| 466 | <allow send_destination="org.ofono"/> |
| 467 | </policy> |
| 468 | |
| 469 | </busconfig> |
| 470 | END |
| 471 | ---- |
| 472 | |
| 473 | (No restart of dbus nor ofono necessary.) |
| 474 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 475 | [[install_slave_unit]] |
| 476 | === Slave Unit(s) |
| 477 | |
| 478 | The slave units are the hosts used by {app-name} to run proceses on. It may be |
| 479 | the <<install_main_unit,Main Unit>> itself and processes will be run locally, or |
| 480 | it may be a remote host were processes are run usually through SSH. |
| 481 | |
| 482 | This guide assumes slaves unit(s) use same configuration as the Main Unit, that |
| 483 | is, it runs under 'jenkins' user which is a member of the 'osmo-gsm-tester' user |
| 484 | group. In order to do so, follow the instruction under the |
| 485 | <<install_main_unit,Main Unit>> section above. Keep in mind the 'jenkins' user |
| 486 | on the Main Unit will need to be able to log in through SSH as the slave unit |
| 487 | 'jenkins' user to run the processes. No direct access from Jenkins Master node |
| 488 | is required here. |
| 489 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 490 | [[install_capture_packets]] |
Pau Espin Pedrol | 71a54fb | 2020-03-14 19:20:03 +0100 | [diff] [blame] | 491 | ==== Capture Packets |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 492 | |
| 493 | In order to allow collecting pcap traces of the network communication for later |
| 494 | reference, allow the osmo-gsm-tester group to capture packets using the 'tcpdump' |
| 495 | program: |
| 496 | |
| 497 | ---- |
| 498 | chgrp osmo-gsm-tester /usr/sbin/tcpdump |
| 499 | chmod 750 /usr/sbin/tcpdump |
| 500 | setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump |
| 501 | ---- |
| 502 | |
| 503 | Put 'tcpdump' in the '$PATH' -- assuming that 'tcpdump' is available for root: |
| 504 | |
| 505 | ---- |
| 506 | ln -s `which tcpdump` /usr/local/bin/tcpdump |
| 507 | ---- |
| 508 | |
| 509 | TIP: Why a symlink in '/usr/local/bin'? On Debian, 'tcpdump' lives in |
| 510 | '/usr/sbin', which is not part of the '$PATH' for non-root users. To avoid |
| 511 | hardcoding non-portable paths in the osmo-gsm-tester source, 'tcpdump' must be |
| 512 | available in the '$PATH'. There are various trivial ways to modify '$PATH' for |
| 513 | login shells, but the jenkins build slave typically runs in a *non-login* |
| 514 | shell; modifying non-login shell enviroments is not trivially possible without |
| 515 | also interfering with files installed from debian packages. Probably the |
| 516 | easiest way to allow all users and all shells to find the 'tcpdump' binary is |
| 517 | to actually place a symbolic link in a directory that is already part of the |
| 518 | non-login shell's '$PATH'. Above example places such in '/usr/local/bin'. |
| 519 | |
| 520 | Verify that a non-login shell can find 'tcpdump': |
| 521 | |
| 522 | ---- |
| 523 | su jenkins -c 'which tcpdump' |
| 524 | # should print: "/usr/local/bin/tcpdump" |
| 525 | ---- |
| 526 | |
| 527 | WARNING: When logged in via SSH on your main unit, running 'tcpdump' to capture |
| 528 | packets may result in a feedback loop: SSH activity to send tcpdump's output to |
| 529 | your terminal is in turn is picked up in the tcpdump trace, and so forth. When |
| 530 | testing 'tcpdump' access, make sure to have proper filter expressions in place. |
| 531 | |
Pau Espin Pedrol | 71a54fb | 2020-03-14 19:20:03 +0100 | [diff] [blame] | 532 | ==== Allow Core Files |
Pau Espin Pedrol | 018e104 | 2017-08-28 11:58:25 +0200 | [diff] [blame] | 533 | |
| 534 | In case a binary run for the test crashes, a core file of the crash should be |
| 535 | written. This requires a limit rule. Create a file with the required rule: |
| 536 | |
| 537 | ---- |
| 538 | sudo -s |
| 539 | echo "@osmo-gsm-tester - core unlimited" > /etc/security/limits.d/osmo-gsm-tester_allow-core.conf |
| 540 | ---- |
| 541 | |
| 542 | Re-login the user to make these changes take effect. |
| 543 | |
| 544 | Set the *kernel.core_pattern* sysctl to *core* (usually the default). For each |
| 545 | binary run by osmo-gsm-tester, a core file will then appear in the same dir that |
| 546 | contains stdout and stderr for that process (because this dir is set as CWD). |
| 547 | |
| 548 | ---- |
| 549 | sysctl -w kernel.core_pattern=core |
| 550 | ---- |
| 551 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 552 | TIP: Files required to be installed under '/etc/security/limits.d/' can be found |
| 553 | under 'osmo-gsm-tester.git/utils/limits.d/', so one can simply cp them from |
| 554 | there. |
| 555 | |
Pau Espin Pedrol | 71a54fb | 2020-03-14 19:20:03 +0100 | [diff] [blame] | 556 | ==== Allow Realtime Priority |
Pau Espin Pedrol | 018e104 | 2017-08-28 11:58:25 +0200 | [diff] [blame] | 557 | |
| 558 | Certain binaries should be run with real-time priority, like 'osmo-bts-trx'. |
| 559 | Add this permission on the main unit: |
| 560 | |
| 561 | ---- |
| 562 | sudo -s |
| 563 | echo "@osmo-gsm-tester - rtprio 99" > /etc/security/limits.d/osmo-gsm-tester_allow-rtprio.conf |
| 564 | ---- |
| 565 | |
| 566 | Re-login the user to make these changes take effect. |
| 567 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 568 | TIP: Files required to be installed under '/etc/security/limits.d/' can be found |
| 569 | under 'osmo-gsm-tester.git/utils/limits.d/', so one can simply cp them from |
| 570 | there. |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 571 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 572 | ===== Allow capabilities: 'CAP_NET_RAW', 'CAP_NET_ADMIN', 'CAP_SYS_ADMIN' |
Pau Espin Pedrol | 8ad5e6a | 2017-11-02 16:08:05 +0100 | [diff] [blame] | 573 | |
| 574 | Certain binaries require 'CAP_NET_RAW' to be set, like 'osmo-bts-octphy' as it |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 575 | uses a 'AF_PACKET' socket. Similarly, others (like osmo-ggsn) require |
| 576 | 'CAP_NET_ADMIN' to be able to create tun devices, and so on. |
Pau Espin Pedrol | 8ad5e6a | 2017-11-02 16:08:05 +0100 | [diff] [blame] | 577 | |
| 578 | To be able to set the following capability without being root, osmo-gsm-tester |
| 579 | uses sudo to gain permissions to set the capability. |
| 580 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 581 | This is the script that osmo-gsm-tester expects on the host running the process: |
Pau Espin Pedrol | 8ad5e6a | 2017-11-02 16:08:05 +0100 | [diff] [blame] | 582 | |
| 583 | ---- |
| 584 | echo /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh <<EOF |
| 585 | #!/bin/bash |
| 586 | /sbin/setcap cap_net_raw+ep $1 |
| 587 | EOF |
| 588 | chmod +x /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh |
| 589 | ---- |
| 590 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 591 | Now, again on the same host, we need to provide sudo access to this script for |
Pau Espin Pedrol | 8ad5e6a | 2017-11-02 16:08:05 +0100 | [diff] [blame] | 592 | osmo-gsm-tester: |
| 593 | |
| 594 | ---- |
| 595 | echo "%osmo-gsm-tester ALL=(root) NOPASSWD: /usr/local/bin/osmo-gsm-tester_setcap_net_raw.sh" > /etc/sudoers.d/osmo-gsm-tester_setcap_net_raw |
| 596 | chmod 0440 /etc/sudoers.d/osmo-gsm-tester_setcap_net_raw |
| 597 | ---- |
| 598 | |
| 599 | The script file name 'osmo-gsm-tester_setcap_net_raw.sh' is important, as |
| 600 | osmo-gsm-tester expects to find a script with this name in '$PATH' at run time. |
| 601 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 602 | TIP: Files required to be installed under '/etc/sudoers.d/' can be found |
| 603 | under 'osmo-gsm-tester.git/utils/sudoers.d/', so one can simply cp them from |
| 604 | there. |
| 605 | |
| 606 | TIP: Files required to be installed under '/usr/local/bin/' can be found |
| 607 | under 'osmo-gsm-tester.git/utils/bin/', so one can simply cp them from |
| 608 | there. |
| 609 | |
| 610 | [[user_config_uhd]] |
Pau Espin Pedrol | 71a54fb | 2020-03-14 19:20:03 +0100 | [diff] [blame] | 611 | ==== UHD |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 612 | |
| 613 | Grant permission to use the UHD driver to run USRP devices for osmo-bts-trx, by |
| 614 | adding the jenkins user to the 'usrp' group: |
| 615 | |
| 616 | ---- |
| 617 | gpasswd -a jenkins usrp |
| 618 | ---- |
| 619 | |
| 620 | To run osmo-bts-trx with a USRP attached, you may need to install a UHD driver. |
| 621 | Please refer to http://osmocom.org/projects/osmotrx/wiki/OsmoTRX#UHD for |
| 622 | details; the following is an example for the B200 family USRP devices: |
| 623 | |
| 624 | ---- |
| 625 | apt-get install libuhd-dev uhd-host |
| 626 | /usr/lib/uhd/utils/uhd_images_downloader.py |
| 627 | ---- |
Pau Espin Pedrol | 8ad5e6a | 2017-11-02 16:08:05 +0100 | [diff] [blame] | 628 | |
Neels Hofmeyr | 7468953 | 2017-09-05 19:43:02 +0200 | [diff] [blame] | 629 | ==== Log Rotation |
| 630 | |
| 631 | To avoid clogging up /var/log, it makes sense to choose a sane maximum log size: |
| 632 | |
| 633 | ---- |
| 634 | echo maxsize 10M > /etc/logrotate.d/maxsize |
| 635 | ---- |
| 636 | |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 637 | ==== Install Scripts |
| 638 | |
| 639 | IMPORTANT: When using the jenkins build slave as configured above, *there is no |
| 640 | need to install the osmo-gsm-tester sources on the main unit*. The jenkins job |
| 641 | will do so implicitly by checking out the latest osmo-gsm-tester sources in the |
| 642 | workspace for every run. If you're using only the jenkins build slave, you may |
| 643 | skip this section. |
| 644 | |
| 645 | If you prefer to use a fixed installation of the osmo-gsm-tester sources |
| 646 | instead of the jenkins workspace, you can: |
| 647 | |
| 648 | . From the run job configured above, remove the line that says |
| 649 | + |
| 650 | ---- |
| 651 | PATH="$PWD/osmo-gsm-tester/src:$PATH" \ |
| 652 | ---- |
| 653 | + |
| 654 | so that this uses a system wide installation instead. |
| 655 | |
| 656 | . Install the sources e.g. in '/usr/local/src' as indicated below. |
| 657 | |
| 658 | On the main unit, to install the latest in '/usr/local/src': |
| 659 | |
| 660 | ---- |
| 661 | apt-get install git |
| 662 | mkdir -p /usr/local/src |
| 663 | cd /usr/local/src |
| 664 | git clone git://git.osmocom.org/osmo-gsm-tester |
| 665 | ---- |
| 666 | |
| 667 | To allow all users to run 'osmo-gsm-tester.py', from login as well as non-login |
| 668 | shells, the easiest solution is to place a symlink in '/usr/local/bin': |
| 669 | |
| 670 | ---- |
| 671 | ln -s /usr/local/src/osmo-gsm-tester/src/osmo-gsm-tester.py /usr/local/bin/ |
| 672 | ---- |
| 673 | |
| 674 | (See also the tip in <<install_capture_packets>> for a more detailed |
| 675 | explanation.) |
| 676 | |
| 677 | The example configuration provided in the source is suitable for running as-is, |
| 678 | *if* your hardware setup matches (you could technically use that directly by a |
| 679 | symlink e.g. from '/usr/local/etc/osmo-gsm-tester' to the 'example' dir). If in |
| 680 | doubt, rather copy the example, point 'paths.conf' at the 'suites' dir, and |
| 681 | adjust your own configuration as needed. For example: |
| 682 | |
| 683 | ---- |
| 684 | cd /etc |
| 685 | cp -R /usr/local/src/osmo-gsm-tester/example osmo-gsm-tester |
Pau Espin Pedrol | 69488f6 | 2020-04-07 16:40:41 +0200 | [diff] [blame] | 686 | sed -i 's#\./suites#/usr/local/src/osmo-gsm-tester/suites#' osmo-gsm-tester/paths.conf |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 687 | ---- |
| 688 | |
| 689 | NOTE: The configuration will be looked up in various places, see |
| 690 | <<config_paths>>. |