| [[ipc_if]] |
| == osmo-trx-ipc IPC Interface |
| |
| This interface is the one used by _osmo_trx_ipc_ backend to communicate to a |
| third party process in charge of driving the lowest layer device-specific bits |
| (from now on the Driver). |
| |
| It consists of a set of Unix Domain (UD) sockets for the control plane, plus a |
| shared memory region for the data plane. |
| |
| Related code can be found in the |
| https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc[Transceiver52M/device/ipc/] |
| directory in _osmo-trx.git_. |
| |
| If you are a potential driver implementator, the |
| various primitives and data structures are publicly available in header file |
| https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]. |
| |
| === Control plane |
| |
| Control plane protocol is transmitted over Unix Domain (UD) sockets using |
| message based primitives. Each primitive has a type identified by an integer, |
| and each type of primitive has a number of extra attributes attached to it. The |
| IPC interface consists of 2 types of UD sockets: |
| |
| * _Master_ UD socket: One per osmo-trx-ipc process. |
| |
| * _Channel_ UD socket: One for each channel managed by osmo-trx-ipc process. |
| |
| The _Driver_ is in all cases expected to take the server role when creating UD |
| sockets, while _osmo-trx-ipc_ takes the client role and connects to sockets |
| provided by the driver. |
| |
| === Master UD socket |
| |
| During startup, _osmo-trx-ipc_ will try connecting to the _Driver_ Master UD |
| socket located in the path provided by its own (VTY) configuration. As a result, |
| it means the _Driver_ process must be running and listening on the Master UD |
| socket before _osmo-trx-ipc_ is started, otherwise _osmo-trx-ipc_ will fail and |
| exit. |
| |
| Once connected, _osmo-trx-ipc_ will submit a `GREETING_REQ` message primitive |
| announcing the maximum supported protocol version (first version ever is `1`, |
| increasing over time). |
| |
| The _Driver_ shall then answer in `GREETING_CNF` message primitive with its own |
| maximum supported version (`<=` version received), providing 0 if none is |
| supported. |
| |
| If _osmo-trx-ipc_ receives back the requested version, then both sides agreed |
| on the protocol version to use. |
| If _osmo-trx-ipc_ receives back a lower version, it shall decide to continue |
| with version negotiation using a lower version, until a supported version or 0 |
| is received. If finally 0 is received, _osmo-trx-ipc_ will disconnect and exit |
| with failure. |
| |
| Once the version is negotiated (`v1` as of current date), _osmo-trx-ipc_ will |
| ask for device information and available characeristics to the _Driver_ using |
| the `INFO_REQ` message primitive. |
| |
| The _Driver_ shall then answer with a `INFO_CNF` message |
| containing information, such as: |
| |
| * String containing device description |
| |
| * Available reference clocks, |
| |
| * {rx,tx} I/Q scaling factors |
| |
| * Maximum number of channels supported |
| |
| * for each channel: |
| |
| ** List of available {rx,tx} paths/antennas. |
| |
| ** {min,max}{rx,tx} gains |
| |
| ** Nominal transmit power |
| |
| All the information received from the _Driver_ during `INFO_CNF` will be used by |
| _osmo-trx-ipc_ to decide whether it can fullfil the requested configuration from |
| the user, and proceed to open the device, or exit with a failure (for instance |
| number of channels, referece clock or tx/rx antenna selected by the user cannot |
| be fullfilled). |
| |
| _osmo-trx-ipc_ will then proceed to open the device and do an initial |
| configuration using an `OPEN_REQ` message, where it will provide the _Driver_ |
| with the desired selected configuration (such as number of channels, rx/tx |
| paths, clock reference, bandwidth filters, etc.). |
| |
| The _Driver_ shall then configure the device and send back a `OPEN_CNF` with: |
| |
| * `return_code` integer attribute set to `0` on success or `!0` on error. |
| |
| * Name of the Posix Shared Memory region where data plane is going to be |
| transmitted. |
| |
| * One path for each channel, containing the just-created UD socket to manage |
| that channel (for instance by taking Master UD socket path and appending |
| `_$chan_idx`). |
| |
| * Path Delay: this is the loopback path delay in samples (= used as a timestamp |
| offset internally by _osmo-trx-ipc_), this value contains the analog delay as |
| well as the delay introduced by the digital filters in the fpga in the sdr |
| devices, and is therefore device type and bandwidth/sample rate dependant. This |
| can not be omitted, wrong values will lead to a _osmo-trx-ipc_ that just doesn't |
| detect any bursts. |
| |
| Finally, _osmo-trx-ipc_ will connect to each channel's UD socket (see next |
| section). |
| |
| Upon _osmo-trx-ipc_ closing the UD master socket connection, the _Driver_ shall |
| go into _closed_ state: stop all processing and instruct the device to power |
| off. |
| |
| TIP: See |
| https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h] |
| for the detailed definition of all the related message primitives and data |
| types for this socket. |
| |
| === Channel UD Socket |
| |
| This socket can be used by _osmo-trx-ipc_ to start/stop data plane processing or |
| change channel's parameters such as Rx/Tx Frequency, Rx/Tx gains, etc. |
| |
| A channel can be either in _started_ or _stopped_ state. When a channel is |
| created (during `OPEN_REQ` in the Master UD Socket), it's by default in |
| _stopped_ state. `START_REQ` and `STOP_REQ` messages control this state, and |
| eventual failures can be reported through `START_CNF` and `STOP_CNF` by the |
| _Driver_. |
| |
| The message `START_REQ` instructs the _Driver_ to start processing data in the |
| data plane. Similary, `STOP_REQ` instructs the _Driver_ to stop processing data |
| in the data plane. |
| |
| Some parameters are usually changed only when the channel is in stopped mode, |
| for instance Rx/Tx Frequency. |
| |
| TIP: See |
| https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h] |
| for the detailed definition of all the related message primitives and data |
| types for this socket. |
| |
| === Data Plane |
| |
| Data plane protocol is implemented by means of a ring buffer structure on top of |
| Posix Shared Memory (see `man 7 shm_overview`) between _osmo-trx-ipc_ process |
| and the _Driver_. |
| |
| The Posix Shared Memory region is created and its memory structure prepared by |
| the _Driver_ and its name shared with _osmo-trx-ipc_ during _OPEN_CNF_ message |
| in the Master UD Socket from the Control Plane. Resource allocation for the |
| shared memory area and cleanup is up to the ipc server, as is mutex |
| initialization for the buffers. |
| |
| ==== Posix Shared Memory structure |
| |
| [[fig-shm-structure]] |
| .General overview of Posix Shared Memory structure |
| [graphviz] |
| ---- |
| digraph hierarchy { |
| node[shape=record,style=filled,fillcolor=gray95] |
| edge[dir=back, arrowtail=empty] |
| |
| SHM[label = "{Posix Shared Memory region|+ num_chans\l+ Channels[]\l}"] |
| CHAN0[label = "{Channel 0|...}"] |
| CHAN1[label = "{Channel 1|...}"] |
| CHANN[label = "{Channel ...|}"] |
| STREAM0_UL[label = "{UL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"] |
| STREAM0_DL[label = "{DL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"] |
| STREAM1_UL[label = "{UL Stream|...}"] |
| STREAM1_DL[label = "{DL Stream|...}"] |
| STREAMN_UL[label = "{UL Stream|...}"] |
| STREAMN_DL[label = "{DL Stream|...}"] |
| BUF_0DL0[label = "{DL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"] |
| BUF_0DLN[label = "{DL Sample Buffer ....|...}"] |
| BUF_0UL0[label = "{UL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"] |
| BUF_0ULN[label = "{UL Sample Buffer ...|...}"] |
| |
| SHM->CHAN0 |
| SHM->CHAN1 |
| SHM->CHANN |
| |
| CHAN0->STREAM0_DL |
| CHAN0->STREAM0_UL |
| STREAM0_DL->BUF_0DL0 |
| STREAM0_DL->BUF_0DLN |
| STREAM0_UL->BUF_0UL0 |
| STREAM0_UL->BUF_0ULN |
| |
| CHAN1->STREAM1_UL |
| CHAN1->STREAM1_DL |
| |
| CHANN->STREAMN_UL |
| CHANN->STREAMN_DL |
| } |
| ---- |
| |
| The Posix Shared Memory region contains an array of _Channels_. |
| |
| Each _Channel_ contains 2 Streams: |
| |
| * Downlink _Stream_ |
| |
| * Uplink _Stream_ |
| |
| Each _Stream_ handles a ring buffer, which is implemented as: |
| |
| * An array of pointers to _Sample Buffer_ structures. |
| |
| * Variables containing the number of buffers in the array, as well as the |
| maximum size in samples for each Sample Buffer. |
| |
| * Variables containing `next_read` and `next_write` _Sample Buffer_ (its index |
| in the array of pointers). |
| |
| * Unnamed Posix semaphores to do the required locking while using the ring |
| buffer. |
| |
| Each _Sample Buffer_ contains: |
| |
| * A `timestamp` variable, containing the position in the stream of the first |
| sample in the buffer |
| |
| * A `data_len` variable, containing the amount of samples available to process |
| in the buffer |
| |
| * An array of samples of size specified by the stream struct it is part of. |
| |
| ==== Posix Shared Memory format |
| |
| The Posix Shared memory region shall be formatted applying the following |
| considerations: |
| |
| * All pointers in the memory region are encoded as offsets from the start |
| address of the region itself, to allow different processes with different |
| address spaces to decode them. |
| |
| * All structs must be force-aligned to 8 bytes |
| |
| * Number of buffers must be power of 2 (2,4,8,16,...) - 4 appears to be plenty |
| |
| * IQ samples format: One (complex) sample consists of 16bit i + 16bit q, so the |
| buffer size is number of IQ pairs. |
| |
| * A reasonable per-buffer size (in samples) is 2500, since this happens to be |
| the ususal TX (downlink) buffer size used by _osmo-trx-ipc_ with the b210 (rx |
| over-the-wire packet size for the b210 is 2040 samples, so the larger value of |
| both is convenient). |
| |
| TIP: See |
| https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h] |
| for the detailed definition of all the objects being part of the Posix Shared |
| memory region structure |
| |
| ==== Posix Shared Memory procedures |
| |
| The queue in the shared memory area is not supposed to be used for actual |
| buffering of data, only for exchange, so the general expectation is that it is |
| mostly empty. The only exception to that might be minor processing delays, and |
| during startup. |
| |
| Care must be taken to ensure that only timed waits for the mutex protecting it |
| and the condition variables are used, in order to ensure that no deadlock occurs |
| should the other side die/quit. |
| |
| Thread cancellation should be disabled during reads/writes from/to the queue. In |
| general a timeout can be considered a non recoverable error during regular |
| processing after startup, at least with the current timeout value of one second. |
| |
| Should over- or underflows occur a corresponding message should be sent towards |
| _osmo-trx-ipc_. |
| |
| Upon **read** of `N` samples, the reader does something like: |
| |
| . Acquire the semaphore in the channel's stream object. |
| |
| . Read `stream->next_read`, if `next_read==next_write`, become blocked in |
| another sempahore (unlocking the previous one) until writer signals us, then |
| `buff = stream->buffers[next_read]` |
| |
| . Read `buff->data_len` samples, reset the buffer data (`data_len=0`), |
| increment `next_read` and if read samples is `<N`, continue with next buffer |
| until `next_read==next_write`, then block again or if timeout elapsed, then we |
| reach conditon buffer underflow and `return len < N`. |
| |
| . Release the semaphore |
| |
| Upon **write** of `N` samples, the writer does something like: |
| |
| . Acquire the semapore in the channel's stream object. |
| |
| . Write samples to `buff = stream->buffers[next_write]`. If `data_len!=0`, |
| signal `buffer_overflow` (increase field in stream object) and probably |
| increase next_read`. |
| |
| . Increase `next_write`. |
| |
| . If `next_write` was `== next_read`, signal the reader through the other |
| semaphore that it can continue reading. |