== Configuration

[[config]]
=== Configuration files and directories

Find in below sub-sections all user-defined files and directories used by
{app-name} to run tests on a given setup.

[[config_main]]
==== 'main.conf'

The main configuration file is basically a placeholder for {app-name} to find
paths to all other files and directories used to operate and run tests.

{app-name} looks for the main configuration file in various standard paths in
this order:

- './main.conf' (Current Working Directory)
- '$HOME/.config/osmo-gsm-tester/main.conf'
- '/usr/local/etc/osmo-gsm-tester/main.conf'
- '/etc/osmo-gsm-tester/main.conf'

The config file location can also be set through '-c' command line argument, which
then overrides the above locations.

{app-name} expects to find the following configuration settings in 'main.conf':

- 'state_dir': Path to <<state_dir,state_dir>> directory
- 'trial_dir': Path to <<trials,trial>> directory to test against (overridden by cmdline argument)
- 'suites_dir': List of paths to <<suites_dir,suites_dir>> directories.
- 'scenarios_dir': List of paths to <<scenarios_dir,scenarios_dir>> directories (optional)
- 'default_suites_conf_path': Path to <<default_suites_conf,default-suites.conf>> file (optional)
- 'defaults_conf_path': Path to <<defaults_conf,defaults.conf>> file (optional)
- 'resource_conf_path': Path to <<resource_conf,resources.conf>> file (optional)

Configuration settings holding a list of paths, such as 'suites_dir' or
'scenarios_dir', are used to look up for paths in regular list of order, meaning
first paths in list take preference over last ones. As a result, if a suite
named 'A' is found in several paths, the one on the first path in the list will
be used.

These are described in detail in the following sections. If no value is provided
for a given setting, sane default paths are used: For 'state_dir',
'/var/tmp/osmo-gsm-tester/state/' is used. All other files and directories are
expected, by default, to be in the same directory as <<config_main,main.conf>>

IMPORTANT: Relative paths provided in 'main.conf' are parsed as being relative
to the directory of that 'main.conf' file itself, and not relative to the CWD
of the {app-name} process parsing it.

.Sample main.conf file:
----
state_dir: '/var/tmp/osmo-gsm-tester/state'
suites_dir: [ '/usr/local/src/osmo-gsm-tester/suites' ]
scenarios_dir: [ './scenarios' ]
trial_dir: './trial'
default_suites_conf_path: './default-suites.conf'
defaults_conf_path: './defaults.conf'
resource_conf_path: './resources.conf'
----

[[state_dir]]
==== 'state_dir'

It contains global or system-wide state for osmo-gsm-tester. In a typical state
dir you can find the following files:

'last_used_*.state'::
	Contains stateful content spanning accross {app-name} instances and
	runs. For instance, 'last used msisdn number.state' is automatically
	(and atomically) increased every time osmo-gsm-tester needs to assign a
	new subscriber in a test, ensuring tests get unique msisdn numbers.
'reserved_resources.state'::
	File containing a set of reserved resources by any number of
	osmo-gsm-tester instances (aka pool of allocated resources). Each
	osmo-gsm-tester instance is responsible to clear its resources from the
	list once it is done using them and are no longer reserved.
'lock'::
	Lock file used to implement a mutual exclusion zone around any state
	files in the 'state_dir', to prevent race conditions between different
	{app-name} instances running in parallel.

This way, several concurrent users of osmo-gsm-tester (ie. several
osmo-gsm-tester processes running in parallel) can run without interfering with
each other (e.g. using same ARFCN, same IP or same ofono modem path).

If you would like to set up several separate configurations (not typical), note
that the 'state_dir' is used to reserve resources, which only works when all
configurations that share resources also use the same 'state_dir'. It's also
important to notice that since resources are stored in YAML dictionary form, if
same physical device is described differently in several
<<resource_conf,resources.conf>> files (used by different {app-name} instances),
resource allocation may not work as expected.

[[suites_dir]]
==== 'suites_dir'

Suites contain a set of tests which are designed to be run together to test a
set of features given a specific set of resources. As a result, resources are
allocated per suite and not per test.

Tests for a given suite are located in the form of '.py' python scripts in the
same directory where the <<suite_conf,suite.conf>> lays.

Tests in the same testsuite willing to use some shared code can do so by putting
it eg. in '$suites_dir/$suitename/lib/testlib.py':
----
#!/usr/bin/env python3
from osmo_gsm_tester.testenv import *

def my_shared_code(foo):
    return foo.bar()
----

and then in the test itself use it this way:
----
#!/usr/bin/env python3
from osmo_gsm_tester.testenv import *

import testlib
suite.test_import_modules_register_for_cleanup(testlib)
from testlib import my_shared_code

bar = my_shared_code(foo)
----

.Sample 'suites_dir' directory tree:
----
suites_dir/
|-- suiteA
|   |-- suite.conf
|   '-- testA.py
|-- suiteB
|   |-- testB.py
|   |-- testC.py
|   |-- lib
|   |   '-- testlib.py
|   '-- suite.conf
----

[[suite_conf]]
===== 'suite.conf'

This file content is parsed using the <<schema_want,Want>> schema.

On the <<schema_want,resources>> section, it provides {app-name} with the base restrictions
(later to be further filtered by <<scenario_conf,scenario>> files) to apply when
allocating resources.

It can also override attributes for the allocated resources through the
<<schema_want,modifiers>> section (to be further modified by
<<scenario_conf,scenario>> files later on). Similarly it can do the same for
general configuration options (no per-resource) through the
<<schema_want,config>> section.

The _schema_ section allows defining a suite's own schema used to validate
parameters passed to it later on through <<scenario_conf,scenario>> files (See
<<scenario_suite_params>>), and which can be retrieved by tests using the
_tenv.config_suite_specific()_ and _tenv.config_test_specific()_ APIs. The first
one will provide the whole dictionary under schema, while the later will return
the dictionary immediatelly inside the former and matching the test name being
run. For instance, if _tenv.config_test_specific()_ is called from test
_a_suite_test_foo.py_, the method will return the contents under dictionary with
key _a_suite_test_foo_.

.Sample 'suite.conf' file:
----
resources:
  ip_address:
  - times: 9 # msc, bsc, hlr, stp, mgw*2, sgsn, ggsn, iperf3srv
  bts:
  - times: 1
  modem:
  - times: 2
    features:
    - gprs
    - voice
  - times: 2
    features:
    - gprs

config:
  bsc:
    net:
      codec_list:
      - fr1

schema:
  some_suite_parameter: 'uint'
  a_suite_test_foo:
    one_test_parameter_for_test_foo: 'str'
    another_test_parameter_for_test_foo: ['bool_str']

defaults:
  timeout: 50s
----

[[scenarios_dir]]
==== 'scenarios_dir'

This dir contains scenario configuration files.

.Sample 'scenarios_dir' directory tree:
----
scenarios_dir/
|-- scenarioA.conf
'-- scenarioB.conf
----

[[scenario_conf]]
===== 'scenario conf file'
Scenarios define further constraints to serve the resource requests of a
<<suite_conf,suite.conf>>, ie. to select specific resources from the general
resource pool specified in <<resource_conf,resources.conf>>.

If only one resource is specified in the scenario, then the resource allocator
assumes the restriction is to be applied to the first resource and that remaining
resources have no restrictions to be taken into consideration.

To apply restrictions only on the second resource, the first element can be left
emtpy, like:

----
resources:
  bts:
  - {}
  - type: osmo-bts-sysmo
----

On the 'osmo_gsm_tester.py' command line and the
<<default_suites_conf,default_suites.conf>>, any number of such scenario
configurations can be combined in the form:

----
<suite_name>:<scenario>[+<scenario>[+...]]
----

e.g.

----
my_suite:sysmo+tch_f+amr
----

*_Parametrized scenario conf files_*:

Furthermore, scenario '.conf' files can be parametrized. The concept is similar to that
of systemd's Template Unit Files. That is, an scenario file can be written so
that some values inside it can be passed at the time of referencing the
scenario name. The idea behind its existence is to re-use the same
scenario file for a set of attributes which are changed and that can have a lot
of different values. For instance, if a scenario is aimed at setting or
filtering some specific attribute holding an integer value, without parametrized
scenarios then a separate file would be needed for each value the user wanted to use.

A parametrized scenario file, similar to systemd Template Unit Files,
contain the character '@' in their file name, ie follow the syntax below:
----
scenario-name@param1,param2,param3,[...],paramN.conf
----

Then, its content can be written this way:
----
$ cat $scenario_dir/my-parametrized-scenario@.conf
resources:
  enb:
  - type: srsenb
    rf_dev_type: ${param1}
modifiers:
  enb:
    - num_prb: ${param2}
----

Finally, it can be referenced during {app-name} execution this way, for instance
when running a suite named '4g':
----
- 4g:my-parametrized-scenario@uhd,6
----
This way {app-name} when parsing the scenarios and combining them with the suite will::
. Find out it is parametrized (name contains '@').
. Split the name
  ('my-parametrized-scenario') from the parameter list (param1='uhd', param2='6')
. Attempt to match a '.conf' file fully matching name and parameters (hence
  specific content can be set for specific values while still using parameters
  for general values), and otherwise match only by name.
. Generate the final
  scenario content from the template available in the matched '.conf' file.

[[scenario_suite_params]]
*_Scenario to set suite/test parameters_*:

First, the suite needs to define its schema in its <<suite_conf,suite.conf>>
file. Check <<suite_conf>> on how to do so.

For instance, for a suite named 'mysuite' containing a test 'a_suite_test_foo.py', and containing this schema in its <<suite_conf,suite.conf>> file:
----
schema:
  some_suite_parameter: 'uint'
  a_suite_test_foo:
    one_test_parameter_for_test_foo: 'str'
    another_test_parameter_for_test_foo: ['bool_str']
----

One could define a parametrized scenario 'myparamscenario@.conf' like this:
----
config:
  suite:
    mysuite:
      some_suite_parameter: ${param1}
      a_suite_test_foo:
        one_test_parameter_for_test_foo: ${param2}
        another_test_parameter_for_test_foo: ['true', 'false', 'false', 'true']
----

And use it in {app-name} this way:
----
mysuite:myparamscenario@4,hello.conf
----

[[resources_conf]]
==== 'resources.conf'

//TODO: update this section
The 'resources.conf' file defines which hardware is connected to the main unit,
as well as which limited configuration items (like IP addresses or ARFCNs)
should be used.

A 'resources.conf' is validated by the <<schema_resources,resources schema>>.
That means it is structured as a list of items for each resource type, where
each item has one or more attributes -- for an example, see
<<resources_conf_example>>.

Side note: at first sight it might make sense to the reader to rather structure
e.g. the 'ip_address' or 'arfcn' configuration as +
'"arfcn: GSM-1800: [512, 514, ...]"', +
but the more verbose format is chosen to stay consistent with the general
structure of resource configurations, which the resource allocation algorithm
uses to resolve required resources according to their traits. These
configurations look cumbersome because they exhibit only one trait / a trait
that is repeated numerous times. No special notation for these cases is
available (yet).

[[default_suites_conf]]
==== 'default-suites.conf'

The 'default-suites.conf' file contains a YAML list of 'suite:scenario+scenario+...'
combination strings as defined by the 'osmo-gsm-tester.py -s' commandline
option. If invoking the 'osmo-gsm-tester.py' without any suite definitions, the
'-s' arguments are taken from this file instead. Each of these suite + scenario
combinations is run in sequence.

A suite name must match the name of a directory in the
<<suites_dir,suites_dir/>> as defined by <<main_conf,main.conf>>.

A scenario name must match the name of a configuration file in the
<<scenarios_dir,scnearios_dir/>> as defined by <<main_conf,main.conf>>
(optionally without the '.conf' suffix).

.Sample 'default-suites.conf' file:
----
- sms:sysmo
- voice:sysmo+tch_f
- voice:sysmo+tch_h
- voice:sysmo+dyn_ts
- sms:trx
- voice:trx+tch_f
- voice:trx+tch_h
- voice:trx+dyn_ts
----

==== 'defaults.conf'

In {app-name} object instances requested by the test and created by the suite
relate to a specific allocated resource. That's not always the case, and even if
it the case the information stored in <<resources_conf,resources.conf>> for that
resource may not contain tons of attributes which the object class needs to
manage the resource.

For this exact reason, the 'defaults.conf' file exist. It contains a set of
default attributes and values (in YAML format) that object classes can use to
fill in the missing gaps, or to provide values which can easily be changed or
overwritten by <<suite_conf,suite.conf>> or <<scenario_conf,scenario.conf>>
files through modifiers.

Each binary run by osmo-gsm-tester, e.g. 'osmo-nitb' or 'osmo-bts-sysmo',
typically has a configuration file template that is populated with values for a
trial run. Hence, a <<suite_conf,suite.conf>>, <<scenario_conf,scenario.conf>>
or a <<resources_conf,resources.conf>> providing a similar setting always has
precedence over the values given in a 'defaults.conf'


.Sample 'defaults.conf' file:
----
nitb:
  net:
    mcc: 901
    mnc: 70
    short_name: osmo-gsm-tester-nitb
    long_name: osmo-gsm-tester-nitb
    auth_policy: closed
    encryption: a5_0

bsc:
  net:
    mcc: 901
    mnc: 70
    short_name: osmo-gsm-tester-msc
    long_name: osmo-gsm-tester-msc
    auth_policy: closed
    encryption: a5_0
    authentication: optional

msc:
  net:
    mcc: 901
    mnc: 70
    short_name: osmo-gsm-tester-msc
    long_name: osmo-gsm-tester-msc
    auth_policy: closed
    encryption: a5_0
    authentication: optional

bsc_bts:
  location_area_code: 23
  base_station_id_code: 63
  stream_id: 255
  osmobsc_bts_type: sysmobts
  trx_list:
  - nominal_power: 23
    max_power_red: 0
    arfcn: 868
    timeslot_list:
    - phys_chan_config: CCCH+SDCCH4
    - phys_chan_config: SDCCH8
    - phys_chan_config: TCH/F_TCH/H_PDCH
    - phys_chan_config: TCH/F_TCH/H_PDCH
    - phys_chan_config: TCH/F_TCH/H_PDCH
    - phys_chan_config: TCH/F_TCH/H_PDCH
    - phys_chan_config: TCH/F_TCH/H_PDCH
    - phys_chan_config: TCH/F_TCH/H_PDCH
----

=== Schemas

All configuration attributes in {app-name} are stored and provided as YAML
files, which are handled internally mostly as sets of dictionaries, lists and
scalars. Each of these configurations have a known format (set of keys and
values), which is called 'schema'. Each provided configuration is validated
against its 'schema' at parse time. Hence, 'schemas' can be seen as a namespace
containing a structured tree of configuration attributes. Each attribute has a
schema type assigned which constrains the type of value it can hold.

There are several well-known schemas used across {app-name}, and they are
described in following sub-sections.

[[schema_main_cfg]]
==== Schema 'main config'

This schema defines all the attributes available in {app-name} the main
configuration file <<main_conf,main.conf>>, and it is used to validate it.

[[schema_resources]]
==== Schema 'resources'

This schema defines all the attributes which can be assigned to
a _resource_, and it is used to validate the <<resources_conf,resources.conf>>
file. Hence, the <<resources_conf,resources.conf>> contains a list of elements
for each resource type. This schema is also used and extended by the
<<schema_want,'want' schema>>.

It is important to understand that the content in this schema refers to a list of
resources for each resource class. Since a list is ordered by definition, it
clearly identifies specific resources by order. This is important when applying
filters or modifiers, since they are applied per-resource in the list. One can
for instance apply attribute A to first resource of class C, while not applying
it or applying another attribute B to second resources of the same class. As a
result, complex forms can be used to filter and modify a list of resources
required by a testsuite.

On the other hand, it's also important to note that lists for simple or scalar
types are currently being treated as unordered sets, which mean combination of
filters or modifiers apply differently. In the future, it may be possible to
have both behaviors for scalar/simple types by using also the YAML 'set' type in
{app-name}.

//TODO: update this list and use a table for each resource type in its own object section
////
These kinds of resources and their attributes are known:

'ip_address'::
	List of IP addresses to run osmo-nitb instances on. The main unit
	typically has a limited number of such IP addresses configured, which
	the connected BTS models can see on their network.
  'addr':::
	IPv4 address of the local interface.

'bts'::
	List of available BTS hardware.
  'label':::
	human readable label for your own reference
  'type':::
	which way to launch this BTS, one of
	- 'osmo-bts-sysmo'
	- 'osmo-bts-trx'
	- 'osmo-bts-octphy'
	- 'ipa-nanobts'
  'ipa_unit_id':::
	ip.access unit id to be used by the BTS, written into BTS and BSC config.
  'addr':::
	Remote IP address of the BTS for BTS like sysmoBTS, and local IP address
	to bind to for locally run BTS such as osmo-bts-trx.
  'band':::
	GSM band that this BTS shoud use (*TODO*: allow multiple bands). One of:
	- 'GSM-1800'
	- 'GSM-1900'
	- (*TODO*: more bands)
  'trx_list':::
	Specific TRX configurations for this BTS. There should be as many of
	these as the BTS has TRXes. (*TODO*: a way to define >1 TRX without
	special configuration for them.)
    'hw_addr'::::
	Hardware (MAC) address of the TRX in the form of '11:22:33:44:55:66',
	only used for osmo-bts-octphy.  (*TODO*: and nanobts??)
    'net_device'::::
	Local network device to reach the TRX's 'hw_addr' at, only used for
	osmo-bts-octphy. Example: 'eth0'.
    'nominal_power'::::
	Nominal power to be used by the TRX.
    'max_power_red'::::
	Max power reduction to apply to the nominal power of the TRX.
'arfcn'::
	List of ARFCNs to use for running BTSes, which defines the actual RF
	frequency bands used.
  'arfcn':::
	ARFCN number, see e.g.
	https://en.wikipedia.org/wiki/Absolute_radio-frequency_channel_number
	(note that the resource type 'arfcn' contains an item trait also named
	'arfcn').
  'band':::
	GSM band name to use this ARFCN for, same as for 'bts:band' above.

'modem'::
	List of modems reachable via ofono and information on the inserted SIM
	card. (Note: the MSISDN is allocated dynamically in test scripts).
  'label':::
	Human readable label for your own reference, which also appears in logs.
  'path':::
	Ofono's path for this modem, like '/modemkind_99'.
  'imsi':::
	IMSI of the inserted SIM card, like '"123456789012345"'.
  'ki':::
	16 byte authentication/encryption KI of the inserted SIM card, in
	hexadecimal notation (32 characters) like +
	'"00112233445566778899aabbccddeeff"'.
  'auth_algo':::
	Authentication algorithm to be used with the SIM card. One of:
	- 'none'
	- 'xor'
	- 'comp128v1'
  'ciphers':::
	List of ciphers that this modem supports, used to match
	requirements in suites or scenarios. Any combination of:
	- 'a5_0'
	- 'a5_1'
	- 'a5_2'
	- 'a5_3'
	- 'a5_4'
	- 'a5_5'
	- 'a5_6'
	- 'a5_7'
  'features':::
	List of features that this modem supports, used to match requirements in
	suites or scenarios. Any combination of:
	- 'sms'
	- 'gprs'
	- 'voice'
	- 'ussd'
////

[[schema_want]]
==== Schema 'want'

This schema is basically the same as the <<schema_resources,resources>> one, but
with an extra 'times' attribute for each resource item. All 'times' attributes
are expanded before matching. For example, if a 'suite.conf' requests two BTS,
one may enforce that both BTS should be of type 'osmo-bts-sysmo' in these ways:

----
resources:
  bts:
  - type: osmo-bts-sysmo
  - type: osmo-bts-sysmo
----

or alternatively,

----
resources:
  bts:
  - times: 2
    type: osmo-bts-sysmo
----

[[schema_config]]
==== Schema 'config'

This schema defines all the attributes which can be used by object classes or
tests during test execution. The main difference between this schema and the
<<schema_resources,resources>> schema is that the former contains configuration
to be applied globally for all objects being used, while the later applies
attributes to a specific object in the list of allocated resources. This schema
hence allows setting attributes for objects which are not allocated as resources
and hence not directly accessible through scenarios, like a BSC or an iperf3
client.

This schema is built dynamically at runtime from content registered by:
- object classes registering their own attributes
- test suite registering their own attributes through <<suite_conf,suite.conf>>
  and tests being able to later retrieve them through 'testenv' API.

[[schema_all]]
==== Schema 'all'

This schema is basically an aggregated namespace for <<schema_want,want>> schema
and <<schema_config,config>> schema, and is the one used by
<<suite_conf,suite.conf>> and <<scenario_conf,scenario.conf>> files. It contains
these main element sections:::

[[schema_all_sec_resources]]
- Section 'resources': Contains a set of elements validated with <<schema_want,want>>
  schema. In  <<suite_conf,suite.conf>> it is used to construct the list of
  requested resources. In  <<scenario_conf,scenario.conf>>, it is used to inject
  attributes to the initial <<suite_conf,suite.conf>> _resources_ section and
  hence further restrain it.
[[schema_all_sec_modifiers]]
- Section 'modifiers': Both in <<suite_conf,suite.conf>> and
  <<scenario_conf,scenario.conf>>, values presented in here are injected into
  the content of the <<schema_all_sec_resources,resources section>> after
  _resource_ allocation, hereby overwriting attributes passed to the object
  class instance managing the specific _resource_ (matches by resource type and
  list position). Since it is combined with the content of
  <<schema_all_sec_resources,resources section>>, it is clear that the
  <<schema_want,want schema>> is used to validate this content.
[[schema_all_sec_config]]
- Section 'config': Contains configuration attributes for {app-name} object
  classes which are not _resources_, and hence cannot be configured with
  <<schema_all_sec_modifiers,modifiers>>. They can overwrite values provided in the
  <<defaults_conf,defaults.conf>> file. Content in this section follows the
  <<schema_config,config>> schema.

//TODO: defaults.timeout should be change in code to be config.test_timeout or similar
//TODO: 'config' should be split into its own schema and validate defaults.conf

=== Example Setup

{app-name} comes with an example official setup which is the one used to run
Osmocom's setup. There are actually two different available setups: a
production one and an RnD one, used to develop {app-name} itself. These two set
ups share mostly all configuration, main difference being the
<<resources_conf,resources.conf>> file being used.

All {app-name} related configuration for that environment is publicly available
in 'osmo-gsm-tester.git' itself:

- <<main_conf,main.conf>>: Available Available under 'sysmocom/', with its paths
  already configured to take required bits from inside the git repository directory.
- <<suite_dir,suites_dir>>: Available under 'sysmocom/suites/'
- <<scenarios_dir,scenarios_dir>>: Available under 'sysmocom/scenarios/'
- <<resource_conf,resources.conf>>: Available under 'sysmocom/' as
  'resources.conf.prod' for Production setup and as 'resources.conf.rnd' for the
  RnD setup. One must use a symbolic link to have it available as
  'resources.conf'.

There are also small sample setups under the 'doc/examples/' directory to
showcase how to set up different types of networks.

==== Typical Invocations

Each invocation of osmo-gsm-tester deploys a set of pre-compiled binaries for
the Osmocom core network as well as for the Osmocom based BTS models. To create
such a set of binaries, see <<trials>>.

Examples for launching test trials:

- Run the default suites (see <<default_suites_conf,default_suites.conf>>) on a
  given set of binaries from 'path/to/my-trial' with <<main_conf,main.conf>>
  available under a standard path:

----
osmo-gsm-tester.py path/to/my-trial
----

- Same as above, but run an explicit choice of 'suite:scenario' combinations:

----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo -s sms:trx -s sms:nanobts
----

- Same as above, but run one 'suite:scenario1+scenario2' combination, setting
  log level to 'debug' and enabling logging of full python tracebacks, and also
  only run just the 'mo_mt_sms.py' test from the suite, e.g. to investigate a
  test failure:

----
osmo-gsm-tester.py path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt
----

- Same as above, but tell {app-name} to read the 'main.conf' in specific
  directory 'path/to/my/main.conf':

----
osmo-gsm-tester.py -c path/to/my/main.conf path/to/my-trial -s sms:sysmo+foobar -l dbg -T -t mo_mt
----

A test script may also be run step-by-step in a python debugger, see
<<debugging>>.