Live streaming of DVB radio

DVB Radio live streaming

If you have Squeezebox media players and a Logitech media server in the house you probably like to be able to receive radio with it. Without problems you can listen to different webstreams, but actually you can receive a lot of radio programms via satellite or cable televion. These DVB data streams have (technical) a high quality and it scales very well. This means that the reception is guaranteed, since it is actually radio reception. However, the radio programmes on DVB-S and DVB-C are MPEG transport streams.

[view of the player]
DVB Stream received as (shoutcast compatible) stream by a Squeezebox replica. Currently played station SWR1 BW "Der Nachmittag".

MPEG transport streams are not played by the Squeezebox and most Internet and web radios. Therefore a conversion is necessary. With this conversion you can convert DVB -C/-S radio programs into something that most Webradios and the Squeezeboxes will understand.

Since you need a specialized application to use a tv card (or SAT>IP) I use tvheadend for the actual reception. I personally felt that the setup was not trivial and you also need additional software which is not available without further ado to add extended to provide program information from the MPEG transport stream in the radio program (from the MPEG "EPG"). In principle, tvheadend can also be used to convert the MPEG transport streams into "naked" MPEG audio data. With my specialized application described below, information about the stream can also be transmitted.

TV card / SAT>IP

For the reception of the DVB data streams I use a Digital Devices Cine-C2 and also a Selfsat SAT>IP antenna is used. The Linux kernel module ddbridge must not be loaded with MSI-X interrupts, since otherwise you will observe I2C bus timeouts. Digital Devices describes this in the FAQ. I therefore specified the following options:

server:/etc>cat modules |grep ddbridge
ddbridge msi=0 
server:/etc>cat modprobe.d/ddbridge.conf
options ddbridge msi=0

For the integration of the satellite antenna I write more on an extra page. Otherwise you don't need any special drivers for this.

tvheadend

After the low-level modules have been loaded without any problems, you can install the tvheadend middleware. I am using tvheadend, currently in version 4.3-1774~g0122ccb on a Debian Jessie. This is a beta version that is not in the Debian Jessie standard repository. The support of SAT>IP is not yet stabilized in all situations, however, so I'm always testing new tvheadend versions. For more info please see the corresponding wiki page of the tvheadend project.After installing tvheadend, it is noticeable that the configuration is somewhat unusual. Everything is configured via the web frontend. There is no central configuration file but rather a configuration directory which can be found in /home/hts/.hts/. Rather Linux-unusual a manual configuration is rather not possible but all settings must be done via the web frontend.
Furthermore, I assume that tvheadend is already set up so that you can watch TV with vlc or kodi or listen to radio.

To be able to listen to the streams with Squeezebox you have to convert MPEG transport stream into something the Squeezebox understands. For this I use the software ts2shout which I have developed. This converts a MPEG transport stream into a Shoutcast- or, alternatively, a native MPEG Audio data stream.

For streaming the radio programs from cable or satellite I use the tvheadend "Stream profile" "pass" which is an already predefined standard in tvheadend.

ts2shout

The conversion is done by apache2 together with a CGI- and Filter program, which is available here:

ts2shout is intentionally minimalistic, but autoconfigures. A suitable audio mpeg payload is searched In the MPEG transport stream and it will also extract the EPG information about the current broadcast. This will be converted to Shoutcast Format (started without parameters) or into a naked mpeg audio data stream (with Parameter noshout called). The crc32 checks over the MPEG control frames will be made. A "naked" mpeg audio stream can also be generated directly with tvheadend, by using the corresponding transcoding module. ts2shout is only needed if you want a "Shoutcast"-compatible audio stream, that contains broadcast information about the current radio broadcast from the EPG. This means that you can directly get information about the current radio transmission on the display.

ts2shout ... as filter

ts2shout will be installed in /usr/local/bin as default. I's a "dual-use" application. On the one hand you can use it regularly at the command line as "classic" Unix filter. You can supply a MPEG transport stream on stdin (standard input) and you'll get the naked MPEG audio stream (or Shoutcast media stream) on stdout (standard output). Typically one uses this mode for testing.

ts2shout ... as CGI application

This is the main application. When used as CGI application the relevant stream parameters are taken directly from the MPEG frames. and sent as "Shoutcast-compatible" HTTP headers as soon as they were collected. After that the regular streaming of MPEG data starts. In CGI mode, the input data is streamed with the help of libcurl. As it is usual for CGI programs the result is written to stdout. A parameter cache is used to reduce stream startup time.

There are some hacks in the code so that the EPG displayed on a Squeezebox also makes sense, but nevertheless many Information about the current program is displayed. Sometimes the EPG information about the current broadcast is over 600 characters long (mainly in the "cultural programs" e.g. SWR2 or Bayern 4 Klassik). The squeezeboxes scroll through the text and pause at the beginning of the text a short time

[LMS Ansicht eines langen Titels] [LMS Ansicht eines langen Titels]
Reception of DVB stream. Display in LMS webinterface, currently running are "SWR2 Feature" or "Bayern 4 Jazztime".

Installing of apache2 for radio streaming

I use apache2 in combination with "mod_action" as web server for the CGI program. The following change in /etc/apache2/sites-available/000-default.conf is required after installation of the module mod_action (a2enmod action). this configuration section does not have to be exactly at this place. The configuration has to be inserted at a point where it should be for this virtual host and a (virtual) subdirectory that fits your needs. You can find a minimum server configuration in /etc/apache2/sites-available/000-default.conf in a Debian default installation. Normally you can insert it just there. In the Debian world /cgi-bin/ is searched by default in /usr/lib/cgi-bin/. I have created a symlink ln -s /usr/local/bin/ts2shout /usr/lib/cgi-bin/

But now the required configuration section:

    <Location /radio>
        Order Deny,Allow
        Deny from all
        Allow from 172.16.0.0/24
        Allow from ::1
        Allow from 127.0.0.1
        <If "%{HTTP:Icy-MetaData} in {'1'}">
                SetEnv "MetaData" "1"
        </If>
        SetEnv TVHEADEND "http://localhost:9981/stream/channelname"
        # The radio stations are called with an URL like
        # "/radio/SWR1 BW" or (URL encoded) /radio/SWR1%20BW
        # if you like to listen the station "SWR1 BW" from tvheadends
        # channel list
        # Only characters supported are A-Z/a-z/0-9, space and "-" due 
        # to the regular expression. Please adjust other channel names
        # in tvheadend, otherwise it cannot work.
        # You can also use the URL http://localhost:9981/stream/channelnumber
        # in conjunction with channel numbers (if used this in the past)
        # If you put "°" in front of the channel name you'll get an AC-3 stream
        # (if it's available)
        # /radio/SWR2 - mpeg audio stream
        # /radio/°SWR2 - AC-3 audio stream
        SetEnvIf REQUEST_URI "°([-A-Za-z0-9_ ]*)$" PROGRAMMNO=$1 AC3=1
        SetEnvIf REQUEST_URI "([-A-Za-z0-9_ ]*)$" PROGRAMMNO=$1
        Options +ExecCGI
        Action ts2shout /cgi-bin/ts2shout virtual
        SetHandler ts2shout
    </Location>

This configuration block provides the following: Apache is instructed to use the CGI application ts2shout for all accesses to URLs below /radio. When called, the environment variables TVHEADEND and MetaData and PROGRAMMNO are set. These are avaluated and used by ts2shout! The URL $TVHEADEND/$PROGRAMMNO is used by ts2shout using libcurl to access tvheadend. The received MPEG transport stream is searched for streaming information (stream rate, bitrate) and the station name. If this has been received, then the HTTP headers on stdout are sent to Apache and then the MPEG data stream in a continuous data stream (in shoutcast mode or not, depending on the possibly received Icy-MetaData HTTP Header). A new stream takes about half a second until the stream starts. It starts when the CGI program has collected all the information. I implemented a streaming parameter cache to reduce the stream startup time to an absolute minimum.

By parsing Icy-MetaData in the request and setting the correct stream- and bitrate the streams should be compatible to many webradios (not only to Squeezebox).

Event errors are logged to stderr and thus to the Apache error log - usually found in /var/log/apache2/error.log. If everything is set up and configured correctly, the following is written from ts2shout to the apache error log.

[Wed Mar 27 06:55:15.036250 2019] [ts2shout:info] [pid 7931] init_structures(): Subscribing to MPEG-TS PID 0, 17, 18 (CHANNEL_TYPE_PAT, CHANNEL_TYPE_SDT, CHANNEL_TYPE_EIT)
[Wed Mar 27 06:55:15.036293 2019] [ts2shout:info] [pid 7931] Streaming with shoutcast StreamTitles in CGI mode.
[Wed Mar 27 06:55:15.037470 2019] [ts2shout:info] [pid 7931] fetch_cached_parameters(): found parameters for programme 813
[Wed Mar 27 06:55:15.061952 2019] [ts2shout:info] [pid 7931] add_channel(): Subscribing to MPEG-TS PID 1000 (Type CHANNEL_TYPE_PMT)
[Wed Mar 27 06:55:15.120455 2019] [ts2shout:info] [pid 7931] add_payload_from_pmt(): Found mp1/mp2 audio stream in PID 1001
[Wed Mar 27 06:55:15.120472 2019] [ts2shout:info] [pid 7931] add_channel(): Subscribing to MPEG-TS PID 1001 (Type CHANNEL_TYPE_PAYLOAD)
[Wed Mar 27 06:55:15.120483 2019] [ts2shout:info] [pid 7931] Synced to MP1/MP2 audio in PID 1001 stream: 0xc0
[Wed Mar 27 06:55:15.120491 2019] [ts2shout:info] [pid 7931]    MPEG-1 layer 2, 320 kbps, 48000 Hz, Stereo
[Wed Mar 27 06:55:15.161536 2019] [ts2shout:info] [pid 7931] EIT: Current transmission `Guten Morgen Baden-Württemberg'
[Wed Mar 27 06:55:15.781241 2019] [ts2shout:info] [pid 7931] SDT: Stream is station SWR1 BW from network ARD SWR.
[Wed Mar 27 08:25:38.660953 2019] [ts2shout:info] [pid 7931] curl_download: Terminated after fetching 1023.30 MB and writing 206.92 MB. Exiting.

The first 2 lines are written directly with program start. ts2shout starts reading the MPEG transport stream and searches for the PAT (Programme association table) to find the PID of the PMT (programme map table). The PMT is also read to get the PID of the actual "audio payload". This is all done automatically and read from the data stream. When the data from PAT and PMT have been processed the audio data stream is output. As soon as "Synced to MPEG audio" and the "SDT:" line have been logged (if no cached station name is found), the audio stream is forwarded in the CGI. In addition to the audio data, data from the EIT (Event information table, better known as EPG) is also transferred to determine the current program. Especially in satellite transmission there is often a big discrepancy between mpeg-ts streaming data and actual audio data.

A line that appears often when stopping streaming [cgid:error] [pid 589:tid 139682125932288] (32)Broken pipe: [client 172.16.0.4:50752] AH02550: Failed to flush CGI output to client is not an error in the narrow sense, but is caused by how the streaming is aborted by the clients. ts2shout tries to stop the streaming of tvheadend as cleanly as possible.

Preparation of tvheadend

[Bitte Bild herunterladen]
Editing users in the tvheadend web frontend

Important: Please adjust Allowed networks in any case to the own installation. This is the IP (range) from which the client, in our case ts2shout will access tvheadend. In my case it is the localhost, but Apache doesn't have to be installed on the same server wher tvheadend runs. Just configure accordingly, since version 4.3-1159 you have to use digest authentication (or reconfigure tvheadend to use "Basic" again).

Now the whole thing can be tested. I have LMS and tvheadend installed on the same machine, but it doesn't have to be like this.

In the Channel-dialog of tvheadend you have to select the channel names that are in use by the desired radio programs.

[Bitte Bild herunterladen]
tvheadend channel view

For streaming of DRS3 (or SRF3, Unitymedia has stopped in the cable BW time) you have to select channel name "SRG-DRS 3".

Squeezebox

Now you switch to the web interface of the LMS and edit the favorites (I put the radio programs there, so you can simply assign a logo to them).

[Ansicht der LMS Oberfläche] [Ansicht des Players]
View of the favorites in the LMS surface and display on the Player, currently running programme SWR1 BW "Der Nachmittag".

Since entering several radio programs manually is a bit cumbersome and you can't assign a logo, I directly edited the favorites and restarted the LMS. To do this you must edit the file /var/lib/squeezeboxserver/prefs/favorites.opml on the LMS. If you want the changes to become active, you have to restart the LMS, maybe you can do this easier, but one doesn't change it very often. An entry in the Favorites looks like this, then. Usernames or passwords are required not, this is done by Apache via an access list.

<outline URL="http://172.16.0.4/radio/SWR1 BW" icon="http://172.16.0.4/logos/SWR1-BW.png" text="SWR1 BW" type="audio" />

I downloaded the logos directly from the respective info pages about the radio program at Wikipedia as a png file. They can be delivered via Apache. In order for the logos to work correctly, the IP address of the icon URL must be "hard coded" in the default settings of the LMS. Otherwise the "image proxy" of mysqueezebox is used. But it cannot reach a webserver which only exists in the LAN. In the next section I describe a solution for this, then the station logos are also displayed much faster in the selection. Caution: If you make a syntax error in favorites.opml, the file will be reset when you start the logitechmediaserver. If you switch on debugging for the favorites, you can see the problematic sequence in the favorites.opml in the logfile.

station logo for tvheadend and Squeezebox

To ensure that the station logos can be used well both in the squeezeboxes and via tvheadend, e.g. in kodi, the following settings in tvheadend and LMS are useful Configuration->General (tvheadend) or Settings->Extended>Performance (LMS):

[picon Dialog tvheadend] [LMS Einstellungen]
Setting up the picons - for vlc, kodi etc.;Setting up the local image proxy in the LMS

To allow the piCore Player ("Squeezeboxes") to quickly load the icons in the Favorites selection, it makes sense to use the local "Imageproxy". The setting dialog is called Disk Envelopes Resize ("Plattenhüllen Größenanpassung") and should be set as shown in the screenshot above. Unfortunately Android App Squeezer has a bug where the pictures of the favorites (in my case the channel logos) are no longer displayed on the smartphone. With the following workaround in the LMS (version 7.9.0), the station logos are also displayed in the app. The bug in the app is that "//imageproxy" is called instead of "/imageproxy" (with a slash).

Advantages of this setup are the following