Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 1 | [[trials]] |
| 2 | == Trial: Binaries to be Tested |
| 3 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 4 | A trial is a set of pre-built sysroot archives to be tested. They are typically built |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 5 | by jenkins using the build scripts found in osmo-gsm-tester's source in the |
Pau Espin Pedrol | f5c83da | 2017-11-02 15:48:10 +0100 | [diff] [blame] | 6 | 'contrib/' dir, see <<install_add_jenkins_slave>>. |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 7 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 8 | A trial comes in the form of a directory containing a number of '<inst-name>.*tgz' tar |
| 9 | archives (containing different sysroots) as well as a 'checksums.md5' file to |
| 10 | verify the tar archives' integrity. |
| 11 | |
| 12 | .Example of a "trial" containing binaries built by a jenkins job |
| 13 | [graphviz] |
| 14 | ---- |
| 15 | digraph G { |
| 16 | subgraph cluster_trial { |
| 17 | label = "Trial (binaries)" |
| 18 | sysmo [label="osmo-bts-sysmo.build-23.tgz\n(osmo-bts-sysmo\n+ deps\ncompiled for sysmoBTS)"] |
| 19 | trx [label="osmo-bts.build-5.tgz\n(osmo-bts-octphy + osmo-bts-trx\n+ deps\ncompiled for main unit)"] |
| 20 | nitb [label="osmo-nitb.build-42.tgz\n(osmo-nitb\n+ deps\ncompiled for main unit)"] |
| 21 | checksums [label="checksums.md5"] |
| 22 | |
| 23 | checksums -> {sysmo trx nitb} |
| 24 | } |
| 25 | } |
| 26 | ---- |
Neels Hofmeyr | c0827c4 | 2017-04-23 15:04:19 +0200 | [diff] [blame] | 27 | |
| 28 | When the osmo-gsm-tester is invoked to run on such a trial directory, it will |
| 29 | create a sub directory named 'inst' and unpack the tar archives into it. |
| 30 | |
| 31 | For each test run on this trial, a new subdirectory in the trial dir is |
| 32 | created, named in the form of 'run.<timestamp>'. A symbolic link 'last-run' |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 33 | will point at the most recently created run dir. This run dir will accumulate: |
| 34 | |
| 35 | * the rendered configuration files used to run the binaries |
| 36 | * stdout and stderr outputs of the binaries |
Pau Espin Pedrol | 0b512ba | 2017-11-02 17:16:21 +0100 | [diff] [blame] | 37 | * pcap files for processes doing relevant network communication |
Neels Hofmeyr | b52df17 | 2017-05-14 20:09:35 +0200 | [diff] [blame] | 38 | * a test log |
Pau Espin Pedrol | 0b512ba | 2017-11-02 17:16:21 +0100 | [diff] [blame] | 39 | * jenkins parsable XML (Junit) reports |
| 40 | |
| 41 | The script in 'contrib/jenkins-run.sh' takes care of related tasks such as |
| 42 | |
| 43 | * creating the dir structure, |
| 44 | * generating md5 sums for the various tar.gz containing software builds to be tested, |
| 45 | * cleaning up after the build, |
| 46 | * saving extra logs such as journalctl output from ofonod, |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 47 | * generating a final .tar.gz file with all the logs and reports to store as jenkins archives. |
| 48 | |
| 49 | {app-name} tests create objects to manage the allocated resources during test |
| 50 | lifetime. These objects, in turn, usually run and manage processes started from |
Pau Espin Pedrol | 6e0b6fb | 2020-05-25 19:49:29 +0200 | [diff] [blame] | 51 | the trial's sysroot binaries. {app-name} provide APIs for those object classes |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 52 | to discover, unpack and run those binaries. An object class simply needs to |
| 53 | request the name of the sysroot it wants to use (for instance 'osmo-bsc'), and |
| 54 | {app-name} will take care of preparing everything and providing the sysroot path |
| 55 | to it. It's a duty of the resource class to copy over the sysroot to the |
| 56 | destination if the intention is to run the binary remotely on another host. |
| 57 | |
Pau Espin Pedrol | 6e0b6fb | 2020-05-25 19:49:29 +0200 | [diff] [blame] | 58 | Moreover, {app-name} supports working with several different versions of a given |
| 59 | sysroot by means of storing them in different subdirectories, which are later |
| 60 | referenced by an object's class 'run_label' attribute named after that |
| 61 | subdirectory. This way, for instance, a sysroot can be provided containing |
| 62 | binaries linked against libraries present on a CentOS distribution, and other |
| 63 | sysroots with the same name can also be provided which are linked against |
| 64 | different versions of CentOS, or a different distro like Debian, or even a |
| 65 | different arch like ARM. |
| 66 | |
Pau Espin Pedrol | 7e0b2dd | 2020-03-10 11:46:39 +0100 | [diff] [blame] | 67 | When seeking a sysroot of a given name '<inst-name>' in the 'inst/' directory, |
| 68 | {app-name} will look for 'tgz' files starting with the pattern '<inst-name>.' |
| 69 | (up to the first dot). That means, suffixes are available for {app-name} user to |
| 70 | identify the content, for instance having an incrementing version counter or a |
| 71 | commit hash. Hence, these example files are considered valid and will be |
| 72 | selected by {app-name} for 'osmo-bsc': 'osmo-bsc.tgz', 'osmo-bsc.build-23.tgz', |
| 73 | 'osmo-bsc.5f3e0dd2.tgz', 'osmo-bsc.armv7.build-2.tgz'. If either none or more |
| 74 | than one valid file is found matching the pattern, an exception will be thrown. |
Pau Espin Pedrol | 6e0b6fb | 2020-05-25 19:49:29 +0200 | [diff] [blame] | 75 | If a 'run_label=foobar' is provided, {app-name} will look up for the 'tgz' in |
| 76 | exactly the same way, but this time in 'inst/foobar/' directory instead of |
| 77 | 'inst/'. |