Reliable RTP

Abstract

Reliable RTP is a set of new features for the RTP (Real-time Transport Protocol) to improve its ability to present a good quality stream to the RTP client even in the event of network loss and network congestion. It also adds congestion control to RTP so streams behave in a TCP-friendly fashion, without disturbing the real-time nature of the protocol. The algorithms used for retransmitting and congestion control are similar to those used in TCP so as to best interoperate with TCP traffic on the Internet. Additionally, those algorithms are time tested to utilize available bandwidth in a near optimal fashion. 

The new features are as follows:
• ACK packets sent from the client to the server.
• Windowing and congestion control so the server does not exceed the currently available bandwidth.
• Retransmits sent from the server to client in the event of packet loss.
• Faster than real time streaming, also known as "over-buffering".

The following sections detail the exact operation of each of these features, and the reasons why this implementation leads to optimal streaming quality. Each of these features also involve negotiation of parameters, limits, etc. This negotiation primarily takes place out of band in RTSP, and is described in the final section.

Section 1: Client to server ACK packets

The Reliable RTP server expects to receive an ACK for each RTP packet it sends. If it does not receive an ACK for a packet, it will retransmit it subject to constraints discussed in section 2. It is not necessary to send one ACK packet for each data packet received from the server–rather, ACKs for several packets may be coalesced and sent to the server in a single packet.

The Reliable RTP ACK packet format is a type of RTCP APP packet (See RFC 1889 for details on the RTCP protocol) After the standard RTCP APP packet headers, the ACK payload consists of a RTP sequence number followed by a variable length bit mask. The sequence number identifies the first RTP packet that the client is acknowledging. Each additional RTP packet being acknowledged is represented by a bit set in the bitmask. The bit mask is an offset from the specified sequence number, where the high order bit of the first byte in the mask is one greater than the sequence number, second bit is two greater, and so on. Bit masks must be sent in multiples of 4-octets. Setting a bit to 0 in the mask simply means that the client does not wish to acknowledge this sequence number right now, and does not imply a negative acknowledgment in any way.

The Reliable RTP ACK packet format is shown below.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P| subtype |   PT=APP=204  |             length            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           SSRC/CSRC                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          name (ASCII)  = 'qtak'               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           SSRC/CSRC                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Reserved             |          Seq num              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Mask...                                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Section 2: Windowing, congestion control, and retransmits

The Reliable RTP server attempts to fully utilize available bandwidth at all times without overstepping the constraints of the network. It does this using an algorithm similar to the TCP slow start and congestion control algorithms. Though a Reliable RTP client will interoperate with a Reliable RTP server that does not implement windowing, it is important for the server to follow these guidelines closely so as not send Retransmits spuriously or send data to quickly for the network to handle. Deviating from this algorithm will result in sub-optimal use of network resources by the Reliable RTP server.

Section A: Round-trip time estimation

The Reliable RTP server uses Karn's algorithm to estimate the current packet round-trip time. The server marks the time at which it sends each RTP packet. When it receives the ACK for that packet, it marks the time and passes that sample round-trip time to Karn's algorithm to compute the estimate.

If the computed round-trip estimate is below MIN_ROUNDTRIP_THRESHOLD, the server sets the estimate equal to that value. If it is above MAX_ROUNDTRIP_THRESHOLD, the server sets it equal to that value. The server initializes the estimate to MIN_ROUNDTRIP_THRESHOLD.

If the server has not received an ACK for a data packet within current estimated round-trip time, it retransmits the packet. The first time a packet gets resent, the round-trip time estimate is increased to 3/2 of the current round-trip time estimate. Bumping up the round-trip time in this manner prevents spurious retransmits in the event of network congestion or reaching the bandwidth ceiling.

When the server receives an ACK for a packet that has already been retransmitted, it does not use that round-trip time sample as a part of its estimate.

Section B: Windowing

The server's ability to send RTP packets is constrained by the size of the congestion window. The congestion window is the maximum number of bytes the server can send without getting an ACK. When the server sends a RTP packet, its size gets added to the count of bytes in the congestion window. When an ACK packet is received for that RTP packet, the count of bytes in the congestion window is reduced by the RTP packet size. When the congestion window fills up, the server must wait for an ACK packet before sending any more data. Restricting the ability of the server to send RTP data with the congestion window prevents it from overstepping the constraints of a congested network and causing packet loss.

Determining the optimal congestion window size is extremely important: if it is too small, network bandwidth will not be utilized optimally as the server will have to wait often for ACK packets to be received from the client. If too big, the server will be able to send out packets too fast and some will get dropped.

The algorithm used is very similar to the one used by TCP. This method has decades of mileage on the public Internet to prove it works. Additionally, using a method similar to TCP makes Reliable RTP "TCP friendly".

Section B2: The algorithm for sizing the congestion window

The congestion window can never be larger than the maximum window size (CLIENT_WINDOW) advertised by the Reliable RTP client.
The server initializes the slow start threshold to 1/2 CLIENT_WINDOW.
The server initializes the congestion window to 4x MAXIMUM_SEGMENT_SIZE.

When the congestion window is below the slow start threshold, it is increased by 2x the size of the packet being ACKed for every ACK received.
When above the slow start threshold, the congestion window increases by MAXIMUM_SEGMENT_SIZE for every full window of ACKs received.

When the server sends a retransmit, the slow start threshold is reset to be half the current congestion window size, and the congestion window is set to be half of either the slow start threshold or the congestion window, whichever is smaller.

Section B3: The effect of this algorithm

Because the congestion window starts small and grows with each ACK received, the server will not exceed the capacity of the network before it has enough data to determine what that capacity is. It increases the amount of data it can send until there is packet loss, and it needs to retransmit something. At that point, it has exceeded the capacity of the network, so clearly the congestion window needs to be scaled back. Eventually, the window reaches a steady state where it only fluctuates greatly if new congestion is introduced or old congestion is relieved.

Section C: Packet expiration times.

Reliable RTP data is real-time. Therefore, there is a finite time after which a packet is too old to be used by the client. When a packet is sent, it is tagged with an expiration time. The expiration time is determined by the content and how far ahead of time it is being sent. If the Reliable RTP server has not received an ACK for a packet by the expiration time, the packet will be marked as no longer available for retransmit.

When a packet expires, the congestion window is resized as if it had just gotten an ACK for the packet.

Section 3: Over-buffering

The windowing scheme described in section 2 gives the Reliable RTP server a good estimate of the available network bandwidth at any given time. Therefore, it is possible for the server to safely send stream data faster than real-time without unknowingly overstepping network capacity. This "over-buffering" reduces the chance that a temporary reduction of available bandwidth will impact stream quality, by giving the server more time to retransmit lost packets. Additionally, in the event of a significant change in available bandwidth, over-buffering gives the server more time to detect this and smoothly switch to a lower bit-rate stream.

However, two other factors besides available bandwidth must constrain the amount of over-buffering that takes place. The first is the rate at which the server over-buffers. If it sends data too fast, even if the network can handle it, over-buffering may disturb more important network traffic going to the client or from the server. For instance, a client may be receiving two RTP streams, one high-bit-rate and one low-bit-rate. Without constraining the over-buffer rate, the two streams will share network bandwidth equally. If the difference in normal stream bit-rate is great enough, the high bit-rate stream may not be able to play at all because the low bit-rate stream is using bandwidth for over-buffering.

The second factor is client memory. Any data received early by the client must remain in the client's buffer until it can be played. If server over-buffering is not constrained and there is enough network bandwidth, the client may actually run out of memory because it has so much stream data in its buffer. In addition, the available memory on the client might change during streaming. Therefore, the server must be periodically notified of how much can be put in the client's buffer.

Additionally, the server will currently constrain over-buffer speed to 2-times the bit-rate of the media stream.  2-times is an arbitrary number that may require some tuning or should be derived through calculation.

Also, the current implementation will only over-buffer a maximum of 25 seconds worth of media.  This is also an arbitrary number that might benefit from tuning or calculation.

In Reliable RTP over-buffering, the server receives an RTCP APP packet containing the size, in bytes, of the client's "over-buffer window". A packet is part of the over-buffer window if it is sent ahead of its normal transmit time. Packets fall out of the over-buffer window when the normal transmit time passes. A Reliable RTP server must not send any stream data if the over-buffer window is full. Instead, it must wait for some packets to fall out of the over-buffer window.

The client must advertise the "over-buffer window" in every RTCP receiver report it sends to the server. In the absence of a non-zero window size, the server must assume the over-buffer window is zero. Though RTCP packets are specific to each stream, the value advertised applies to the window size for all streams in the current RTSP session. The server uses the most recent value received as the value for the whole session, regardless of which stream it arrived on.

The format of the RTCP over-buffer window size APP packet is shown below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P| subtype |   PT=APP=204  |             length            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           SSRC/CSRC                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          name (ASCII)  = 'QTSS'               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           SSRC/CSRC                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      field name='ob           |   version=0   |   length=4    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               Over-buffer window size in bytes                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The 'QTSS' APP packet can contain additional fields. Each field begins with a 2-octet field name, a 1-octet field version, and a 1-octet field length. There is no padding between individual fields. The other fields are documented in the QTSS APP packet documentation.

Section 4: RTSP Negotiation

Whether or not to use Reliable RTP, and the parameters of the protocol, are negotiated out of band in RTSP (Real-Time Streaming Protocol, RFC 2326).

Header 1: x-Retransmit header

If a client would like to use Reliable RTP, it should append this header in its SETUP request. The body of the header contains the retransmit protocol name, followed by a semi-colon delimited list of arguments. The name for the Reliable RTP retransmit protocol is 'our-retransmit'. There is currently one argument that can be passed from client to server, 'window'. If included, this tells the Reliable RTP server the size of the client's window, in KBytes. See section 2B on how this is used to size the congestion window.

Example:

x-Retransmit: our-retransmit;window=128

The server must echo the header and all parameters. If the header is not in the SETUP response, the client must assume that Reliable RTP will not be used for this stream. If the parameter values have been changed, the client must use the new values.

Header 2: x-Transport-Options header

x-transport-options: late-tolerance=1.5