New release, 0.4.1

Changes:
 - added --transcode yes/no to enable transcoding .off into .mp3 for handicapped devices which do not support open codes (iOS, looking at you here)
 - added webcron endpoint to run feed updates in situations where the system scheduler can not be used
 - feed manager is now mostly a single page app with live updates
 - added -v (version) option
 - added versioned updatees for feed and index manager

Fixes:
 - direct login now works as intended

New install requirements:
 - ffmpeg-python
 - setuptools

The change to a SPA was necessitated by the introduction of the `--transcode yes/no` option which (when activated) causes feed updates to take much more time, especially on less powerful hardware. This would cause the feed manager process to timeout before the feeds were updated. This problem is mostly fixed but can still occur in the webcron update process. If this happens the php-fpm and/or web server timeout needs to be increased. This should only happen on slower hardware and/or slow links.

README.md

@@ -88,15 +88,14 @@ To use _Spodcast_ you need a (free) _Spotify_ account, if you don't have one yet
 Here's `spodcast` displaying its help message:
 ```
 $ spodcast -h
-usage: spodcast [-h] [-c CONFIG_LOCATION] [-p] [-l LOGIN] [--root-path ROOT_PATH]
+usage: spodcast [-h] [-c CONFIG_LOCATION] [-p] [-v] [-l LOGIN] [--root-path ROOT_PATH]
-                [--skip-existing SKIP_EXISTING] [--retry RETRY]
+                [--skip-existing SKIP_EXISTING] [--retry RETRY] [--max-episodes MAX_EPISODES]
-                [--max-episodes MAX_EPISODES] [--chunk-size CHUNK_SIZE]
+                [--chunk-size CHUNK_SIZE] [--download-real-time DOWNLOAD_REAL_TIME]
-                [--download-real-time DOWNLOAD_REAL_TIME] [--language LANGUAGE]
+                [--language LANGUAGE] [--credentials-location CREDENTIALS_LOCATION]
-                [--credentials-location CREDENTIALS_LOCATION] [--rss RSS]
+                [--rss-feed RSS_FEED] [--transcode TRANSCODE] [--log-level LOG_LEVEL]
-                [--log-level LOG_LEVEL]
                 [urls ...]
 
-A caching _Spotify_ podcast to RSS proxy.
+A caching Spotify podcast to RSS proxy.
 
 positional arguments:
   urls                  Download podcast episode(s) from a url. Can take multiple urls.
@@ -106,14 +105,15 @@ optional arguments:
   -c CONFIG_LOCATION, --config-location CONFIG_LOCATION
                         Specify the spodcast.json location
   -p, --prepare-feed    Installs RSS feed server code in ROOT_PATH.
+  -v, --version         show program's version number and exit
   -l LOGIN, --login LOGIN
                         Reads username and password from file passed as argument and stores
                         credentials for later use.
   --root-path ROOT_PATH
                         set root path for podcast cache
   --skip-existing SKIP_EXISTING
-                        skip files with the same name and size
+                        [yes|no] skip files with the same name and size. Defaults to "yes".
-  --retry RETRY         retry count for _Spotify_ API access
+  --retry RETRY         retry count for Spotify API access
   --max-episodes MAX_EPISODES
                         number of episodes to download
   --chunk-size CHUNK_SIZE
@@ -122,22 +122,30 @@ optional arguments:
                         simulate streaming client
   --language LANGUAGE   preferred content language
   --credentials-location CREDENTIALS_LOCATION
-                        path to credentials file
+                        path to credentials file. If a relative path is used the file will be
-  --rss RSS             add a (php) RSS feed server and related metadata for feed. To serve
+                        stored in the same directory as the configuration file (-c
-                        the feed, point a web server at the spodcast root path as configured
+                        /path/to/config.json -> /path/to).
-                        using --root-path.
+  --rss-feed RSS_FEED   [yes|no] add a (php) RSS feed server and related metadata for feed. To
+                        manage feeds, point a web server at the spodcast root path as configured
+                        using --root-path. Defaults to "yes".
+  --transcode TRANSCODE
+                        [yes|no] transcode ogg/vorbis to mp3 (where applicable) - only needed
+                        for devices which do not support open formats (e.g. iOS). Defaults to
+                        "no".
   --log-level LOG_LEVEL
                         log level (debug/info/warning/error/critical)
 ```
 
 ### Using _Spodcast_ to proxy _Spotify_ podcasts to RSS
-The following example shows how to use the `spodcast` command to prepare the feed root directory and add a _Spotify_ account to be used. It specifies the configuration file to create (`-c /mnt/audio/podcast/spodcast.json`) and the root path where podcasts will be downloaded to (`--root-path /mnt/audio/spodcast`). The `-p` option tells _spodcast_ to prepare the RSS feed server in the root directory which will also be used to store the credential file created by the `-l spotify.rc` command. That `spotify.rc` file is a plain text file containing the username and password (separated by a single space character) to use to login to _Spotify_. It is only needed to create the stored credentials file(s) so it can be deleted once _Spotcast_ is up and running.
+The following example shows how to use the `spodcast` command to prepare the feed root directory and add a _Spotify_ account to be used. It specifies the configuration file to create (`-c /mnt/audio/podcast/spodcast.json`) and the root path where podcasts will be downloaded to (`--root-path /mnt/audio/spodcast`). The `-p` option tells _spodcast_ to prepare the RSS feed server in the root directory which will also be used to store the credential file created by the `-l spotify.rc` command. That `spotify.rc` file is a plain text file containing the username and password (separated by a single space character) to use to login to _Spotify_. It is only needed to create the stored credentials file(s) so it can be deleted once _Spotcast_ is up and running. If one or more of your preferred listening devices does not supports open audio codecs - e.g. because it runs iOS - you can use `--transcode yes` to enable transcoding of such streams to `mp3`. Transcoding is performed using _ffmpeg_ which needs to be available on your _Spodcast_ host. It can take a considerable amount of time depending on the hardware you're using to run _Spodcast_ so only enable this option when there are no other options.
 ```
 spodcast -c /mnt/audio/podcast/spodcast.json --root-path /mnt/audio/spodcast -p -l /home/exampleuser/spotify.rc
 ```
 Configure the [Web server](#web-server-configuration) using the path given as root path (in this example that would be `/mnt/audio/spodcast`) as web root, making sure to exclude files with `.json` and `.info` extenstions to avoid leaking your _Spotify_ credentials (even though these are stored in hashed form using hashed file names). Now point a browser at the site you configured for _Spodcast_ and you're ready to add the first show or episode. This is done easily by entering the _Spotify_ show/episode url (e.g. `https://open.spotify.com/show/4rOoJ6Egrf8K2IrywzwOMk` for _The Joe Rogan Experience_ for the whole show, `https://open.spotify.com/episode/2rYwwE7hcpgsDo9vRVHxAI?si=24fb00294b7f40db` for a specific episode, notice the `show` and `episode` parts of these links) and either hitting _Enter_ or clicking the _Add_ button. _Spodcast_ will now create a directory under the given root path, add the `.index.php` RSS feed generator script and the `index.info` show info URL used by that script and the RSS manager script and whatever episode(s) you decided to sync.
 
-Once the initial feed has been created it can be kept up to date by enabling the feed update service found in the _Settings_ menu. Select the update frequency and the start time and click _Update_, this will create a _cron_ job for the web server which will run the _Spodcast_ manager script to update feeds. While the update frequency is configured for all shows simultaneously this is not the case for the number of episodes to _sync_ and the number to _keep_ in cache, these can be configured individually for each show. The idea here is that some shows may publish more than one episode between update intervals so by fetching the last X episodes on each update nothing will be missed. Episodes which have already been synced will not be synced again so no time or bandwidth is wasted. In the same vein the number of episodes to _keep_ can be configured to make sure your RSS clients have the opportunity to download these before they are rotated out of cache. Once more than X (being the value chosen for _keep_) episodes have been downloaded the oldest episodes will be deleted to keep the total no more than X.
+Once the initial feed has been created it can be kept up to date by enabling the feed update service found in the _Settings_ menu. Select the update frequency and the start time and click _Update_, this will create a _cron_ job for the web server which will run the _Spodcast_ manager script to update feeds. If the system scheduler can not be used for some reason - e.g. because the web server user is not allowed to create cron jobs or because the web server is run from a Docker container which does not support cron jobs - you can use the _webcron_ endpoint to initiate update runs. Just point any web client at `SPODCAST_URL/?action=update_shows` to run an update and get a json-encoded report on what has been updated. By running a command line web client - _curl_ and _wget_ are good options - as a user cron job at regular intervals the feeds can be kept up to date.
+
+While the update frequency is configured for all shows simultaneously this is not the case for the number of episodes to _sync_ and the number to _keep_ in cache, these can be configured individually for each show. The idea here is that some shows may publish more than one episode between update intervals so by fetching the last X episodes on each update nothing will be missed. Episodes which have already been synced will not be synced again so no time or bandwidth is wasted. In the same vein the number of episodes to _keep_ can be configured to make sure your RSS clients have the opportunity to download these before they are rotated out of cache. Once more than X (being the value chosen for _keep_) episodes have been downloaded the oldest episodes will be deleted to keep the total no more than X.
 
 Point your RSS clients at the _Spodcast_ feed URL for this show and you should see new episodes appear after they were published on _Spotify_ and subsequently picked up on the next update. For the example given in the [Web server requirements](#web-server-configuration) example that URL would be `http://spodcast.example.org/The_Joe_Rogan_Experience`.
 
@@ -151,21 +159,21 @@ Here's what the Spodcast feed manager looks like:
 
 The settings screen is simple and concise:
 
-![Spodcast feed manager settings](/resources/screenshots/relevant_part_of_spodcast_settings_screen.png?raw=true "Spodcast feed manager: Settings")
+![Spodcast feed manager settings](/resources/screenshots/spodcast_settings.png?raw=true "Spodcast feed manager: Settings")
 
 Each show has its own _sync_ and _keep_ settings. Use the _Delete_ button to, well, delete the show. Use _Refresh_ to retrieve the last [_sync_] episodes, skipping those which have already been synced.
 
 ![Show controls](/resources/screenshots/sync_keep_delete_refresh.png?raw=true "Configure _sync_ and _keep_ settings for each show")
 
 ### Using the _Spodcast_ CLI command to download a single episode
-_Spodcast_ can also be used stand-alone (without the need for a web server) by either just ignoring the feed-related files (`.index.php`, `index.info` plus a `*.info` file for every episode) or by disabling the RSS feed using `--rss no` on the command line. Instead of using the `-l spotify.rc` command to add _Spotify_ credentials it is possible to point _Spotcast_ at a single `credentials.json` file (which will be created if it does not exist yet`), `spotcast` wil ask for the username and password when needed. To get single episode links use the _Spotify_ web app and select _Share->Copy Episode Link_ from the episode menu (three dots in the top-right corner of the episode block). The following example shows (an already configured instance of) `spodcast` ready to download a single episode:
+_Spodcast_ can also be used stand-alone (without the need for a web server) by either just ignoring the feed-related files (`.index.php`, `index.info` plus a `*.info` file for every episode) or by disabling the RSS feed using `--rss no` on the command line. Instead of using the `-l spotify.rc` command to add _Spotify_ credentials it is possible to point _Spotcast_ at a single `credentials.json` file (which will be created if it does not exist yet`), `spodcast` wil ask for the username and password when needed. To get single episode links use the _Spotify_ web app and select _Share->Copy Episode Link_ from the episode menu (three dots in the top-right corner of the episode block). The following example shows (an already configured instance of) `spodcast` ready to download a single episode:
 ```
-spodcast -c ~/.config/spodcast/spodcast.json --credentials-location ~/.config/spodcast/credentials.json --rss no https://open.spotify.com/episode/2rYwwE7hcpgsDo9vRVHxAI?si=24fb00294b7f40db
+spodcast -c ~/.config/spodcast/spodcast.json --credentials-location ~/.config/spodcast/credentials.json --rss-feed no https://open.spotify.com/episode/2rYwwE7hcpgsDo9vRVHxAI?si=24fb00294b7f40db
 ```
 Like in the previous example _Spodcast_ will create a directory under the root path with the same name as the show from which the episode is downloaded. The episode will be downloaded into this directory under a `SHOW_NAME_-__EPISODE_NAME.[ogg|mp3]` name. Point a mediaplayer of choice at this file to play the episode.
 In "manual" mode _Spodcast_ does not do anything by itself, feeds can be kept up to date by running _Spotcast_ with the required settings for `--max-episodes` (which is the value used for _sync_ in the web UI) and the show URL. Here's how to update the _The Joe Rogan Experience_ show using the `spodcast` CLI command, syncing the last 3 episodes:
 ```
-spodcast -c ~/.config/spodcast/spodcast.json --rss no --max-episodes 3 https://open.spotify.com/show/4rOoJ6Egrf8K2IrywzwOMk`
+spodcast -c ~/.config/spodcast/spodcast.json --rss-feed no --max-episodes 3 https://open.spotify.com/show/4rOoJ6Egrf8K2IrywzwOMk`
 ```
 ## Web server configuration
 _Spodcast_ places a hidden `.index.php` file in the root path and each show directory. The one in the root directory is used to manage feeds while those in the show directories produce RSS feeds based on the information found in all `*.info` files in that directory. Configure the server to serve those `.index.php` files as index to make things work as intended. Don't forget to block all web access to files ending in `.json` and `.info` to make sure you _Spotify_ credentials (which are stored in hashed form in files named `spodcast-cred-MD5_HASH_OF_SPOTIFY_USER_NAME.json` in the root path) can not be accessed. For _nginx_ the following should suffice to produce an unencrypted (HTTP) feed under the domain name `spodcast.example.org` given a feed root directory (as configured using `--root-path`) of `/mnt/audio/spodcast` with _php-fpm 7.4_ listening on `unix:/run/php/php7.4-fpm.sock`:

setup.cfg

@@ -1,6 +1,6 @@
 [metadata]
 name = spodcast
-version = 0.3.7
+version = 0.4.1
 description = A caching Spotify podcast to RSS proxy.
 long_description = file:README.md
 long_description_content_type = text/markdown
@@ -21,6 +21,8 @@ packages =
     spodcast
 install_requires =
     librespot >= 0.0.1
+    ffmpeg-python
+    setuptools
 include_package_data =
     True
 

spodcast/__main__.py

@@ -1,11 +1,11 @@
 import argparse
+import pkg_resources
 
 from spodcast.app import client
 from spodcast.config import CONFIG_VALUES
 
 def main():
-    parser = argparse.ArgumentParser(prog='spodcast',
+    parser = argparse.ArgumentParser(prog='spodcast', description='A caching Spotify podcast to RSS proxy.')
-        description='A caching Spotify podcast to RSS proxy.')
     parser.add_argument('-c', '--config-location',
                         type=str,
                         help='Specify the spodcast.json location')
@@ -14,6 +14,10 @@ def main():
                        action='store_true',
                        help='Installs RSS feed server code in ROOT_PATH.')
 
+    parser.add_argument('-v', '--version',
+                       action='version',
+                       version = '%(prog)s ' + pkg_resources.require("Spodcast")[0].version)
+
     group = parser.add_mutually_exclusive_group(required=True)
     group.add_argument('urls',
                        type=str,
@@ -26,7 +30,6 @@ def main():
                        type=str,
                        help='Reads username and password from file passed as argument and stores credentials for later use.')
 
-
     for configkey in CONFIG_VALUES:
         parser.add_argument(CONFIG_VALUES[configkey]['arg'],
                             type=str,

spodcast/config.py

@@ -1,5 +1,6 @@
 import json
 import os
+import pkg_resources
 import sys
 from typing import Any
 
@@ -7,16 +8,20 @@ CONFIG_FILE_PATH = '../spodcast.json'
 
 CONFIG_DIR = 'CONFIG_DIR'
 CONFIG_PATH = 'CONFIG_PATH'
+VERSION = 'VERSION'
+VERSION_STR = pkg_resources.require("Spodcast")[0].version
+
 ROOT_PATH = 'ROOT_PATH'
 SKIP_EXISTING_FILES = 'SKIP_EXISTING_FILES'
 CHUNK_SIZE = 'CHUNK_SIZE'
 DOWNLOAD_REAL_TIME = 'DOWNLOAD_REAL_TIME'
 LANGUAGE = 'LANGUAGE'
 CREDENTIALS_LOCATION = 'CREDENTIALS_LOCATION'
-RETRY_ATTEMPTS = 'RETRY_ATTEMPTS'
+RETRY = 'RETRY'
 MAX_EPISODES = 'MAX_EPISODES'
 LOG_LEVEL = 'LOG_LEVEL'
-ENABLE_RSS_FEED = 'ENABLE_RSS_FEED'
+RSS_FEED = 'RSS_FEED'
+TRANSCODE = 'TRANSCODE'
 
 CONFIG_VALUES = {
     ROOT_PATH:            { 'default': '../Spodcast/',
@@ -26,16 +31,16 @@ CONFIG_VALUES = {
     SKIP_EXISTING_FILES:  { 'default': 'True',
                             'type': bool,
                             'arg': '--skip-existing',
-                            'help': 'skip files with the same name and size' },
+                            'help': '[yes|no] skip files with the same name and size. Defaults to "yes".' },
-    RETRY_ATTEMPTS:       { 'default': '5',
+    RETRY:                { 'default': 5,
                             'type': int,
                             'arg': '--retry',
                             'help': 'retry count for Spotify API access' },
-    MAX_EPISODES:         { 'default': '1000',
+    MAX_EPISODES:         { 'default': 1000,
                             'type': int,
                             'arg': '--max-episodes',
                             'help': 'number of episodes to download' },
-    CHUNK_SIZE:           { 'default': '50000',
+    CHUNK_SIZE:           { 'default': 50000,
                             'type': int,
                             'arg': '--chunk-size',
                             'help': 'download chunk size' },
@@ -50,11 +55,15 @@ CONFIG_VALUES = {
     CREDENTIALS_LOCATION: { 'default': 'credentials.json',
                             'type': str,
                             'arg': '--credentials-location',
-                            'help': 'path to credentials file' },
+                            'help': 'path to credentials file. If a relative path is used the file will be stored in the same directory as the configuration file (-c /path/to/config.json -> /path/to).' },
-    ENABLE_RSS_FEED:      { 'default': 'True',
+    RSS_FEED:             { 'default': 'True',
                             'type': bool,
-                            'arg': '--rss',
+                            'arg': '--rss-feed',
-                            'help': 'add a (php) RSS feed server and related metadata for feed. To serve the feed, point a web server at the spodcast root path as configured using --root-path.' },
+                            'help': '[yes|no] add a (php) RSS feed server and related metadata for feed. To manage feeds, point a web server at the spodcast root path as configured using --root-path. Defaults to "yes".' },
+    TRANSCODE:            { 'default': 'False',
+                            'type': bool,
+                            'arg': '--transcode',
+                            'help': '[yes|no] transcode ogg/vorbis to mp3 (where applicable) - only needed for devices which do not support open formats (e.g. iOS). Defaults to "no".' },
     LOG_LEVEL:            { 'default': 'warning',
                             'type': str,
                             'arg': '--log-level',
@@ -111,6 +120,7 @@ class Config:
 
         cls.Values[CONFIG_DIR] = os.path.dirname(true_config_file_path)
         cls.Values[CONFIG_PATH] = str(true_config_file_path)
+        cls.Values[VERSION] = str(VERSION_STR)
 
     @classmethod
     def get_default_json(cls) -> Any:
@@ -172,19 +182,24 @@ class Config:
 
     @classmethod
     def get_credentials_location(cls) -> str:
-        return os.path.join(os.getcwd(), cls.get(CREDENTIALS_LOCATION))
+        credentials_location = cls.get(CREDENTIALS_LOCATION)
+        return credentials_location if os.path.isabs(credentials_location) else os.path.join(cls.get(CONFIG_DIR), cls.get(CREDENTIALS_LOCATION))
 
     @classmethod
-    def get_retry_attempts(cls) -> int:
+    def get_retry(cls) -> int:
-        return cls.get(RETRY_ATTEMPTS)
+        return cls.get(RETRY)
 
     @classmethod
     def get_max_episodes(cls) -> int:
         return cls.get(MAX_EPISODES)
 
     @classmethod
-    def get_enable_rss_feed(cls) -> bool:
+    def get_rss_feed(cls) -> bool:
-        return cls.get(ENABLE_RSS_FEED)
+        return cls.get(RSS_FEED)
+
+    @classmethod
+    def get_transcode(cls) -> bool:
+        return cls.get(TRANSCODE)
 
     @classmethod
     def get_log_level(cls) -> str:
@@ -197,3 +212,11 @@ class Config:
     @classmethod
     def get_config_path(cls) -> str:
         return cls.get(CONFIG_PATH)
+
+    @classmethod
+    def get_version_str(cls) -> str:
+        return cls.get(VERSION)
+
+    @classmethod
+    def get_version_int(cls) -> int:
+        return int(cls.get(VERSION).replace('.',''))

spodcast/feedgenerator.py

@@ -1,17 +1,33 @@
+import re
+
 RSS_FEED_FILE_NAME = '.index.php'
 RSS_FEED_INFO_EXTENSION = 'info'
 RSS_FEED_SHOW_INDEX = 'index'
+RSS_FEED_SHOW_IMAGE = 'image.jpg'
+RSS_FEED_VERSION = '$SPODCAST_VERSION$ '
+VERSION_NOT_FOUND = 0
 
-def RSS_FEED_CODE():
+def get_index_version(filename) -> str:
+    with open(filename, 'rb') as f:
+        for line in f.readlines():
+            m = re.search(RSS_FEED_VERSION + ' (\d+.\d+.\d+)', str(line))
+            if m:
+                return int(m[1].replace('.',''))
+
+    return VERSION_NOT_FOUND
+
+
+def RSS_FEED_CODE(version):
     return r'''<?php
+/* ''' + RSS_FEED_VERSION + version + r''' */
 const SHOW_INDEX = "''' + RSS_FEED_SHOW_INDEX + r'''";
 const INFO = "''' + RSS_FEED_INFO_EXTENSION + r'''";
-$PROTOCOL = ($HTTPS) ? "https://" : "http://";
+$PROTOCOL = (empty($_SERVER['HTTPS'])) ? "http://" : "https://";
 header("Content-type: text/xml");
 $feed_name = "Spodcast autofeed";
 $feed_description = "Spodcast autofeed";
 $base_url = strtok($PROTOCOL . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI'], '?');
-$feed_logo = "$base_url/.image.jpg";
+$feed_logo = "$base_url/''' + RSS_FEED_SHOW_IMAGE r'''";
 $feed_link = $base_url;
 $allowed_extensions = array('mp4','m4a','aac','mp3','ogg');
 
@@ -21,7 +37,6 @@ if(file_exists($sinfo)) {
     $info=json_decode($json);
     $feed_name=$info->title;
     $feed_description=$info->description;
-    $feed_logo=$info->image;
     $feed_link=$info->link;
 }
 
@@ -71,12 +86,14 @@ foreach ($raw_files as &$raw_file) {
     </channel>
 </rss>'''
 
-def RSS_INDEX_CODE(bin_path, config_name):
+def RSS_INDEX_CODE(bin_path, config_name, version):
     return r'''<?php
+/* ''' + RSS_FEED_VERSION + version + r''' */
 const INFO="''' + RSS_FEED_INFO_EXTENSION + r'''";
 const SHOW_INFO="''' + RSS_FEED_SHOW_INDEX + r'''.".INFO;
 const SPODCAST="''' + bin_path + r'''";
 const SPODCAST_CONFIG="''' + config_name + r'''";
+const SHOW_IMAGE="''' + RSS_FEED_SHOW_IMAGE + r'''";
 const FEEDS_INFO="feeds.".INFO;
 const SETTINGS_INFO="settings.".INFO;
 const MAX_EPISODES=3;
@@ -86,10 +103,13 @@ const CLI_COMMANDS=['refresh'];
 const SUCCESS='success';
 const ERROR='error';
 const LOG_LEVEL='warning';
-
 const NOT_FOUND=-1;
 
+$SPODCAST_CONFIG=dirname(__FILE__)."/".SPODCAST_CONFIG;
+
+# CLI
 if (PHP_SAPI == "cli") {
+    global $SPODCAST_CONFIG;
     if (count($argv) < 2) {
         echo "use: php ".__FILE__." <command> [options...]".PHP_EOL;
         echo "     commands: ".implode("|",CLI_COMMANDS).PHP_EOL;
@@ -98,35 +118,30 @@ if (PHP_SAPI == "cli") {
 
     $settings=read(dirname(__FILE__)."/".SETTINGS_INFO);
     $feeds=read(dirname(__FILE__)."/".FEEDS_INFO);
-    $SPODCAST_CONFIG=dirname(__FILE__)."/".SPODCAST_CONFIG;
 
     $command=$argv[1];
     if($command == "refresh") {
-        foreach ($feeds as $url => ["title"=>$title,"directory"=>$directory,"max"=>$max,"keep"=>$keep]) {
+        list($retval, $result) = refresh_shows($feeds);
-            if ($max > 0) {
+        if ($retval > 0) {
-                $output = add_feed($url, $max, true);
+            echo "An error occurred during refresh, return value was $retval" . PHP_EOL;
-                echo(implode(PHP_EOL, $output));
-            }
-            $episodes=get_episodes($directory);
-            if(count($episodes) > $keep) {
-                $to_delete=array_splice($episodes, $keep);
-                foreach ($to_delete as $episode) {
-                    system("rm -f ".escapeshellarg($directory."/".$episode['filename'])." 2>&1");
-                    system("rm -f ".escapeshellarg($directory."/".$episode['filename']).".".INFO." 2>&1");
-                }
-            }
         }
+        echo implode(PHP_EOL, $result);
     }
     die();
 }
 
+# CGI/API
+$PROTOCOL = (empty($_SERVER['HTTPS'])) ? "http://" : "https://";
+$SPODCAST_URL = $PROTOCOL . $_SERVER['HTTP_HOST'] . parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH);
 $feeds=get_feeds(dirname(__FILE__));
 $settings=get_settings();
+$config=get_spodcast_config();
 $ERROR_MESSAGE=null;
 $ERROR_DETAILS=null;
+$SPODCAST_COMMAND=SPODCAST." -c ".$SPODCAST_CONFIG." --log-level ".$settings['log_level'];
 
 function get_feeds($dir) {
-    $spodcast_url=(isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] === 'on' ? "https" : "http")."://".$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
+    global $SPODCAST_URL;
     foreach(glob($dir."/*/".SHOW_INFO) as $show_info) {
         $episodes=get_episodes(dirname($show_info));
         $json=file_get_contents($show_info);
@@ -138,7 +153,7 @@ function get_feeds($dir) {
         $feeds[$info->link]['directory']=dirname($show_info);
         $feeds[$info->link]['max']=$info->max ?? 2;
         $feeds[$info->link]['keep']=$info->keep ?? 5;
-        $feeds[$info->link]['feed']=$spodcast_url.basename(dirname($show_info));
+        $feeds[$info->link]['feed']=$SPODCAST_URL.basename(dirname($show_info));
     }
     uasort($feeds, fn ($a, $b) => strnatcmp($a['title'], $b['title']));
     store($feeds, FEEDS_INFO);
@@ -146,6 +161,7 @@ function get_feeds($dir) {
 }
 
 function get_episodes($dir) {
+    $episodes=[];
     foreach(glob($dir."/*.".INFO) as $episode_info) {
         if(basename($episode_info) == SHOW_INFO) {
             continue;
@@ -159,22 +175,29 @@ function get_episodes($dir) {
 }
 
 function get_settings() {
+    global $SPODCAST_URL;
     $settings=read(SETTINGS_INFO);
-    $spodcast_url=(isset($_SERVER['HTTPS']) && $_SERVER['HTTPS'] === 'on' ? "https" : "http")."://".$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
+    $settings['spodcast_url']=$settings['spodcast_url'] ?? $SPODCAST_URL;
-    $settings['spodcast_url']=$settings['spodcast_url'] ?? $spodcast_url;
     $settings['update_start']=$settings['update_start'] ?? 0;
     $settings['update_rate']=$settings['update_rate'] ?? 1;
     $settings['update_enabled']=$settings['update_enabled'] ?? false;
+    $settings['log_level']=$settings['log_level'] ?? LOG_LEVEL;
     store($settings, SETTINGS_INFO);
     return $settings;
 }
 
+function get_spodcast_config() {
+    global $SPODCAST_CONFIG;
+    $config=read($SPODCAST_CONFIG);
+    return $config;
+}
+
 function get(&$var, $default=null) {
     return isset($var) ? $var : $default;
 }
 
 function read($file) {
-    if(file_exists($file)) {
+    if(is_readable($file)) {
         $json=file_get_contents($file);
         $info=json_decode($json, true);
     } else {
@@ -185,8 +208,9 @@ function read($file) {
 
 function store($info, $file) {
     $f = fopen($file,'w');
-    fwrite($f, json_encode($info));
+    $result = fwrite($f, json_encode($info));
     fclose($f);
+    return ($result === false) ? ERROR : SUCCESS;
 }
 
 function cron_signature($crontab, $CRON_SIGNATURE) {
@@ -207,11 +231,15 @@ function debug($var) {
 }
 
 function submit_crontab($crontab) {
-    $tempfile=system("mktemp");
+    $retval = null;
+    $output = null;
+    $tempfile=tempnam(sys_get_temp_dir(), 'spodcast');
     file_put_contents($tempfile, implode(PHP_EOL,$crontab));
     $command="crontab ".$tempfile;
-    exec($command);
+    exec($command, $output, $retval);
     unlink($tempfile);
+
+    return [$retval, $output];
 }
 
 function get_range($start, $rate) {
@@ -221,6 +249,31 @@ function get_range($start, $rate) {
     return implode(",", $arr);
 }
 
+function background_check_id($id) {
+    $runfile = md5($id).".json";
+    $log = md5($id).".log";
+    if (is_readable($runfile)) {
+        $info = read($runfile);
+        if (background_check($info['pid'])) {
+            return true;
+        }
+        unlink($runfile);
+        unlink($log);
+    }
+    return false;
+}
+
+function background_check($pid) {
+    try {
+        $result = shell_exec(sprintf("ps %d", $pid));
+        if (count(preg_split("/\n/", $result)) > 2) {
+            return true;
+        }
+    } catch(Exception $e) {}
+    return false;
+}
+
+# [ status, output ] return functions
 function update_scheduler($enable, $start, $rate) {
     $CRON_SIGNATURE="SPODCAST:".dirname(__FILE__);
     $crontab=null;
@@ -233,42 +286,102 @@ function update_scheduler($enable, $start, $rate) {
                 array_splice($crontab, $index, 1);
             }
             $crontab[]=sprintf("%d %s * * * php %s refresh # %s".PHP_EOL, rand(5,25), get_range($start, $rate),  __FILE__, $CRON_SIGNATURE);
-            submit_crontab($crontab);
+            return submit_crontab($crontab);
         } else {
             if ($index !== NOT_FOUND) {
                 array_splice($crontab, $index, 1);
                 $crontab[count($crontab)-1]=rtrim($crontab[count($crontab)-1]).PHP_EOL;
-                submit_crontab($crontab);
+                return submit_crontab($crontab);
             }
         }
+    } else {
+        return [$retval, "failed to update scheduler"];
     }
 }
 
 function login($username, $password, $return_output=false) {
+    global $SPODCAST_CONFIG;
+    global $SPODCAST_COMMAND;
     $output = null;
     $retval = null;
-    $tempfile=system("mktemp");
+    $tempfile=tempnam(sys_get_temp_dir(), 'spodcast');
-    $SPODCAST_CONFIG=dirname(__FILE__)."/".SPODCAST_CONFIG;
     file_put_contents($tempfile, "$username $password");
-    $command=SPODCAST." -c ".$SPODCAST_CONFIG." --log-level ".LOG_LEVEL." -l ".$tempfile." 2>&1";
+    $command=$SPODCAST_COMMAND . " -l ".$tempfile." 2>&1";
     exec($command, $output, $retval);
     unlink($tempfile);
-    if ($retval > 0 || $return_output) {
+
-        return $output;
+    return [$retval, $output];
-    }
-    return SUCCESS;
 }
 
-function add_feed($url, $max, $return_output=false) {
+function background_run($command, $id) {
+    $retval=null;
+    $md5 = md5($id);
+    $runfile = $md5.".json";
+    $output = $md5.".log";
+    $cmd = sprintf("nohup %s > %s 2>&1 & echo $!", $command, $output);
+    exec($cmd, $pid, $retval);
+    if ($retval == 0 && count($pid) > 0 && $pid[0] > 0) {
+        $info['command']=$command;
+        $info['log']=$output;
+        $info['pid']=(int) $pid[0];
+        store($info, $runfile);
+        return [$retval, $pid[0]];
+    } else {
+        return [$retval, 0];
+    }
+}
+
+function background_add_feed($url, $max) {
+    global $SPODCAST_CONFIG;
+    global $SPODCAST_COMMAND;
     $output = null;
     $retval = null;
-    $SPODCAST_CONFIG=dirname(__FILE__)."/".SPODCAST_CONFIG;
+    $command=$SPODCAST_COMMAND . " --max-episodes ".(int)$max." ".escapeshellarg($url);
-    $command=SPODCAST." -c ".$SPODCAST_CONFIG." --max-episodes ".$max." ".escapeshellarg($url)." 2>&1";
+    list($retval, $pid) = background_run($command, $url);
-    exec($command, $output, $retval);
+    if ($retval == 0 && (int) $pid > 0) {
-    if ($retval > 0 || $return_output) {
+        return [$retval, $pid];
-        return $output;
     }
-    return SUCCESS;
+    return [$retval, 0];
+}
+
+
+function add_feed($url, $max) {
+    global $SPODCAST_CONFIG;
+    global $SPODCAST_COMMAND;
+    $output = null;
+    $retval = null;
+    $command=$SPODCAST_COMMAND . " --max-episodes ".(int)$max." ".escapeshellarg($url)." 2>&1";
+    exec($command, $output, $retval);
+    return [$retval, $output];
+}
+
+function update_feed($url, $max, $keep, $feeds) {
+    $output = null;
+    $retval = null;
+    if ($max > 0) {
+    list($retval,$output) = add_feed($url, $max);
+        if ($retval > 0) {
+            return [$retval, $output];
+        }
+    }
+    $directory=$feeds[$url]['directory'];
+    $episodes=get_episodes($directory);
+    if(count($episodes) > $keep) {
+        $to_delete=array_splice($episodes, $keep);
+        foreach ($to_delete as $episode) {
+            $command="rm -f ".escapeshellarg($directory."/".$episode['filename'])." 2>&1";
+            exec($command, $output, $retval);
+            if ($retval > 0) {
+                return [$retval, $output];
+            }
+            $command="rm -f ".escapeshellarg($directory."/".$episode['filename']).".".INFO." 2>&1";
+            exec($command, $output, $retval);
+            if ($retval > 0) {
+                return [$retval, $output];
+            }
+        }
+    }
+    return [$retval, $output];
 }
 
 function delete_feed($url, $return_output=false) {
@@ -278,85 +391,214 @@ function delete_feed($url, $return_output=false) {
     $feed_dir=$feeds[$url]['directory'];
     $command="rm -rf ".$feed_dir." 2>&1";
     exec($command, $output, $retval);
-    if ($retval > 0 || $return_output) {
+    return [$retval, $output];
-        return $output;
-    }
-    return SUCCESS;
 }
 
 function update_show($url, $field, $value) {
     if (array_search($field, UPDATEABLE) === false) {
-        return ["$field is not an updateable field"];
+        return [1, "$field is not an updateable field"];
     } else {
         $feeds=read(FEEDS_INFO);
         $show_dir=$feeds[$url]['directory'];
         $show=read($show_dir."/".SHOW_INFO);
-        $show[$field]=(int) $value;
+        $show[$field]=$value;
         store($show, $show_dir."/".SHOW_INFO);
-        return SUCCESS;
+        return [0, "$field set to $value"];
     }
 }
 
+function refresh_shows($feeds) {
+    $result = [];
+    $status = 0;
+    foreach ($feeds as $url => ["title"=>$title, "directory"=>$directory, "max"=>$max, "keep"=>$keep]) {
+        $output = null;
+        $retval = 0;
+        list($retval, $output) = update_feed($url, $max, $keep, $feeds, true);
+        if ($retval > 0) {
+            $status = $retval;
+        }
+        $result = array_merge($result, $output);
+    }
+    return [$return, $result];
+}
+
+# terminating functions
+
 function show_error($message, $details) {
     header("Location: ./?action=error&message=".urlencode($message)."&details=".urlencode(implode(PHP_EOL,$details)));
     die();
 }
 
+function json_response($info) {
+    header('Content-Type: application/json; charset=utf-8');
+    echo json_encode($info, true);
+    exit();
+}
+
+# CGI commands
+
 switch(get($_GET['action'])) {
+
 case 'refresh':
+    $url = get($_POST['url']);
+    $max = (int) get($_POST['max']) ?? MAX_EPISODES;
+    $keep = (int) get($_POST['keep']) ?? KEEP_EPISODES;
+    $info['url']=$url;
+    $info['show']=$feeds[$url];
+    $info['id']=basename($url);
+    if ($max > $keep) {
+        $info['result'] = "It does not make sense for the number of episodes to refresh to be larger than the number of episodes to keep.";
+        $info['status'] = 'ERROR';
+    } else {
+        list($result,$output) = update_feed($url, $max, $keep, $feeds, false);
+        $info['status'] = ($result == 0) ? 'SUCCESS': 'ERROR';
+        if ($result != 0) {
+            $info['result'] = "Refresh failed";
+        }
+    }
+    json_response($info);
+
 case 'new':
-    $url = get($_GET['url']);
+    $url = get($_POST['url']);
-    $result = add_feed($url, MAX_EPISODES);
+    list($result, $output) = background_add_feed($url, MAX_EPISODES, false);
-    if ($result !== SUCCESS) {
+    $info['url']=$url;
-        show_error("Add/refresh feed failed", $result);
+    $info['show']=$feeds[$url] ?? null;
-    }
+    $info['id']=basename($url);
-    header("Location: .");
+    $info['status']= ($result == 0) ? 'SUCCESS': 'ERROR';
-    die();
+    json_response($info);
-case 'delete':
+
-    $url = get($_GET['url']);
-    $result = delete_feed($url);
-    if ($result !== SUCCESS) {
-        show_error("Delete feed failed", $result);
-    }
-    header("Location: .");
-    die();
 case 'update':
-    $url = get($_GET['url']);
+    $url = get($_POST['url']);
-    $field = get($_GET['field']);
+    $field = get($_POST['field']);
-    $value = get($_GET['value']);
+    $value = get($_POST['value']);
-    $result=update_show($url, $field, $value);
+    if (in_array(strtolower($field), UPDATEABLE)) {
-    if ($result !== SUCCESS) {
+        list($result, $output) = update_show($url, $field, $value);
-        show_error("Update failed", $result);
+        if ($result == 0) {
+            $info['result'] = $output;
+            $info['status'] = 'SUCCESS';
+        } else {
+            $info['result'] = "Update failed";
+            $info['status'] = 'ERROR';
         }
-    header("Location: .");
+    } else {
-    die();
+        $info['result'] = "$field can not be updated";
-case 'login':
+        $info['status'] = 'ERROR';
-    $username = get($_GET['username']);
-    $password = get($_GET['password']);
-    $result = login($username, $password);
-    if ($result !== SUCCESS) {
-        show_error("Login failed",$result);
     }
-    header("Location: .");
+
-    die();
+    json_response($info);
+
 case 'schedule':
-    $enable = (get($_GET['enable']) == "true" ? true : false);
+    $enable = (get($_POST['enable']) == "true" ? true : false);
-    $start = (int) get($_GET['start']);
+    $start = (int) get($_POST['start']);
-    $rate = (int) get($_GET['rate']);
+    $rate = (int) get($_POST['rate']);
     $settings['update_enabled'] = $enable;
     $settings['update_start'] = $start;
     $settings['update_rate'] = $rate;
-    store($settings, SETTINGS_INFO);
+    $result = store($settings, SETTINGS_INFO);
-    update_scheduler($enable, $start, $rate);
+    if ($result === ERROR) {
-    header("Location: .");
+        $info['status'] = 'ERROR';
-    die();
+        $info['result'] = 'Could not store scheduler preferences, giving up';
+    } else {
+        list($result, $output) = update_scheduler($enable, $start, $rate);
+        if ($result == 0) {
+            $info['status'] = 'SUCCESS';
+            $info['result'] = ($enable === true) ? "Scheduled updates enabled, $rate times per day starting at $start:00" : 'Scheduled updates disabled';
+        } else {
+            $info['status'] = 'ERROR';
+            $info['result'] = implode(PHP_EOL, $result);
+        }
+    }
+    json_response($info);
+
+case 'transcode':
+    $enable = (get($_POST['enable']) == "true" ? true : false);
+    $config['TRANSCODE']=($enable) ? 'True' : 'False';
+    $result = store($config, $SPODCAST_CONFIG);
+    if ($result === SUCCESS) {
+        $info['status'] = 'SUCCESS';
+        $info['result'] = ($enable) ? 'Transcoding enabled' : 'Transcoding disabled';
+    } else {
+        $info['status'] = 'ERROR';
+        $info['result'] = 'Could not enable transcoding: can not write to config file';
+    }
+    json_response($info);
+
+case 'logging':
+    $level = get($_POST['level']) ?? LOG_LEVEL;
+    $config['LOG_LEVEL']=$level;
+    if (in_array(strtolower($level), ['critical','error','warning','info','debug'])) {
+        $result = store($config, $SPODCAST_CONFIG);
+        if ($result === SUCCESS) {
+            $info['status'] = 'SUCCESS';
+            $info['result'] = 'Log level set to '. $level;
+        } else {
+            $info['status'] = 'ERROR';
+            $info['result'] = 'Could not change log level: can not write to Spodcast config file';
+        }
+    } else {
+        $info['status'] = 'ERROR';
+        $info['result'] = 'Invalid log level ' . $level;
+    }
+    json_response($info);
+
+case 'status':
+    $url = get($_POST['url']);
+    $info['url'] = $url;
+    $info['show']=$feeds[$url] ?? null;
+    $info['id'] = basename($url);
+    $info['status'] = background_check_id($url) ? 'ACTIVE' : 'READY';
+    json_response($info);
+
+case 'delete':
+    $url = get($_POST['url']);
+    $info['url'] = $url;
+    $info['id'] = basename($url);
+    list($result, $output) = delete_feed($url);
+    if ($result !== 0) {
+        $info['status'] = 'ERROR';
+        $info['result'] = "Delete feed failed: " . implode(PHP_EOL, $output);
+    } else {
+        $info['status'] = 'SUCCESS';
+        $info['result'] = "Deleted <i>" . $feeds[$url]['title'] . "</i>" ;
+    }
+    json_response($info);
+
+case 'login':
+    $username = get($_POST['username']);
+    $password = get($_POST['password']);
+    list($result, $output) = login($username, $password);
+    if ($result === 0) {
+        $info['status'] = 'SUCCESS';
+        $info['result'] = 'Login succeeded';
+    } else {
+        $info['status'] = 'ERROR';
+        $info['result'] = 'Login failed for user '. $username;
+    }
+    json_response($info);
+
+case 'update_shows':
+    list($result, $output) = refresh_shows($feeds);
+    $info['status'] = ($result == 0) ? 'SUCCESS' : 'ERROR';
+    $info['result'] = $output;
+    json_response($info);
+
 case 'error':
-    $ERROR_MESSAGE = get($_GET['message']);
+    $ERROR_MESSAGE = get($_POST['message']);
-    $ERROR_DETAILS = get($_GET['details']);
+    $ERROR_DETAILS = get($_POST['details']);
 default:
     break;
 }
 
+$ACTIVE=[];
+
+foreach (array_keys($feeds) as $url) {
+    if (background_check_id($url)) {
+        $ACTIVE[$url]=true;
+    }
+}
+
+$TRANSCODE_ENABLED=($config['TRANSCODE'] == "True") ? true : false ;
+$LOG_LEVEL=$config['LOG_LEVEL'];
 $UPDATE_ENABLED=$settings['update_enabled'];
 $UPDATE_START=$settings['update_start'];
 $UPDATE_RATE=$settings['update_rate'];
@@ -373,7 +615,9 @@ $UPDATE_RATE=$settings['update_rate'];
         div#loading { position: fixed; display: flex; width: 100%; height: 100%; top: 0; left: 0; opacity: 0.8; background-color: #fff; z-index: 99; justify-content: center; align-items: center; font-size: 200%; }
         div#settings, div#error { position: fixed; top: 0; left: 0; width: 100%; height: 100%; z-index: 99; opacity: 0.95; margin: 0.5em auto; }
         div.hidden { visibility: hidden; }
-        .episode, .setgroup, .error { display: flex; flex-wrap: wrap; margin: 0 0 2em 0; padding: 0; font-family: sans-serif; border: 1px solid #F0F0F0; background: #F0F0F0; border-radius: 6px; }
+        div.overlay { position: absolute; width: 100%; height: 100%; left: 0; right: 0; background: white; opacity: 0.5; z-index: 100; display: flex; justify-content: center; align-items: center; }
+        .show, .setgroup, .error { position: relative; display: flex; flex-wrap: wrap; margin: 0 0 2em 0; padding: 0; font-family: sans-serif; border: 1px solid #F0F0F0; background: #F0F0F0; border-radius: 6px; }
+        .show .placeholder { height: 4em; }
         .setgroup { border: 1px solid black; margin: 1em; }
         .error { border: 2px solid red; margin: 1em; }
         .entry, .setitem, .errorline { display: flex; box-sizing: border-box; flex-grow: 1; width: 100%; padding: 0.4em 0.6em; overflow: hidden; align-items: center; text-align: left; background: #F0F0F0; }
@@ -396,11 +640,19 @@ $UPDATE_RATE=$settings['update_rate'];
         #delete:hover { color: white; background: red; }
         #update-settings { background: inherit; }
 
-        legend { margin-left: 1.5em; font-size: 120%; border: 1px solid black; border-radius: 6px; background: white; }
+        .loader { display: inline-block; width: 4em; height: 4em; }
+        .loader:after { content: " "; display: block; width: 2em; height: 2em; margin: 8px; border-radius: 50%; border: 6px solid black; border-color: black transparent black transparent; animation: loader 1.2s linear infinite; }
+        @keyframes loader { 0% { transform: rotate(0deg); } 100% { transform: rotate(360deg); } }
+
+        #toast { position: absolute; width: 33%; left: 33%; bottom: 2em; padding: 2em; border-radius: 6px; background: #F0F0F0; font-size: 1.2em; font-family: sans-serif; z-index: 200; text-align: center;}
+        #toast.success { border: 1px solid black; }
+        #toast.error { border: 6px solid red; }
+
+        legend { margin-left: 1.5em; font-size: 100%; border: 1px solid black; border-radius: 6px; background: white; padding: 0.1em 0.5em; }
         .error legend { border: 2px solid red; }
  
         @media all and (max-width: 500px) {
-            .episode { display: block; }
+            .show { display: block; }
             .entry { width: 95% !important; }
             .last { margin-bottom: 1em; }
             .actions { flex-direction: column; align-items: flex-end; }
@@ -416,23 +668,17 @@ $UPDATE_RATE=$settings['update_rate'];
     </div>
     <div id="feed-list">
     <?php foreach($feeds as $url => ["title"=>$title,"image"=>$image,"episodes"=>$episodes,"last"=>$last,"max"=>$max,"keep"=>$keep,"feed"=>$feed]): ?>
-        <div class="episode">
+        <div class="show" id="<?php echo basename($url) ?>">
+            <div class="overlay loader <?php echo (array_key_exists($url, $ACTIVE)) ? "" : "hidden" ?>"></div>
             <div class="entry title"><?=$title?></div>
-            <div class="entry logo"><img src="<?=$image?>"/></div>
+            <div class="entry logo"><img src="<?=$feed."/".SHOW_IMAGE?>"/></div>
             <div class="entry link"><a target="_blank" href="<?=$url?>"><?=htmlspecialchars($url)?></a></div>
-            <div class="entry stats"><?=$last?>&nbsp;(<?=$episodes?>)</div>
+            <div class="entry stats" id="stats-<?php echo basename($url) ?>"><?=$last?>&nbsp;(<?=$episodes?>)</div>
             <div class="entry feed"><a target="_blank" href="<?=$feed?>"><svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-rss" viewBox="0 0 16 16"><path d="M14 1a1 1 0 0 1 1 1v12a1 1 0 0 1-1 1H2a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1h12zM2 0a2 2 0 0 0-2 2v12a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V2a2 2 0 0 0-2-2H2z"/><path d="M5.5 12a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0zm-3-8.5a1 1 0 0 1 1-1c5.523 0 10 4.477 10 10a1 1 0 1 1-2 0 8 8 0 0 0-8-8 1 1 0 0 1-1-1zm0 4a1 1 0 0 1 1-1 6 6 0 0 1 6 6 1 1 0 1 1-2 0 4 4 0 0 0-4-4 1 1 0 0 1-1-1z"/></svg>&nbsp;&nbsp;<?=$feed?></a></div>
-            <div class="entry last actions"><div></div><div>sync&nbsp;<select id="max-<?=htmlspecialchars($url)?>" onchange="showUpdate(this,'<?=$url?>','max')"><?php foreach([0,1,2,3,4,5] as $i) { printf("<option value='%d'>%d</option>",$i, $i);}?></select></div><div>keep&nbsp;<select id="keep-<?=htmlspecialchars($url)?>" onchange="showUpdate(this,'<?=$url?>','keep')"><?php foreach([1,2,5,10,25,50,100,250,1000,2500] as $i) { printf("<option value='%d'>%d</option>",$i, $i);}?></select>&nbsp;</div><div><a id="delete" onclick="return confirmDelete('<?=$title?>','<?=$url?>')" href="?action=delete&url=<?=$url?>">delete</a></div><div><a onclick="document.getElementById('loading').classList.remove('hidden'); return true;" href="?action=refresh&url=<?=$url?>">refresh</a></div></div>
+            <div class="entry last actions"><div></div><div>sync&nbsp;<select id="max-<?=basename($url)?>" onchange="showUpdate(this,'<?=$url?>','max')"><?php foreach([0,1,2,3,4,5] as $i) { printf("<option value='%d'>%d</option>",$i, $i);}?></select></div><div>keep&nbsp;<select id="keep-<?=basename($url)?>" onchange="showUpdate(this,'<?=$url?>','keep')"><?php foreach([1,2,5,10,25,50,100,250,1000,2500] as $i) { printf("<option value='%d'>%d</option>",$i, $i);}?></select>&nbsp;</div><div><a id="delete" onclick="confirmDelete('<?=$title?>','<?=$url?>')">delete</a></div><div><a id="refresh" onclick="refresh('<?=$url?>')">refresh</a></div></div>
         </div>
     <?php endforeach; ?>
     </div>
-    <div id="loading" class="hidden">
-        <div>
-            <h1>Please wait...</h1>
-            <p>This may take a while.
-            <p>Refresh the page if this takes too long
-        </div>
-    </div>
     <div id="settings" class="hidden">
         <div id="settings-header">
             <button id='closesettings' onclick="document.getElementById('settings').classList.add('hidden'); return false;">Close</button>
@@ -445,13 +691,26 @@ $UPDATE_RATE=$settings['update_rate'];
         </fieldset>
         <fieldset id="scheduler" class="setgroup">
             <legend>Feed update schedule</legend>
+            <div class="setitem comment"><span>There are two ways to keep your shows up to date:<ol><li>using the native scheduler</li><li>through webcron: <b><?=$SPODCAST_URL.'?action=update_shows'?></b></li></ol>Enable updates through the native scheduler by clicking the button. If this does not work - e.g. because the web server user is not allowed to add cron jobs or because the web server runs in a Docker container which does not support such jobs - you can point any web client (curl or wget work fine here) at the webcron endpoint (see #2) to initiate an update run. Configure a user cron job for curl/wget to access the refresh endpoint at the desired times for a user experience similar to using the native scheduler.</span>  </div>
             <div class="setitem"><button id="update-enable" name="update-enable">Enable scheduled updates</button></div>
             <div id="update-settings">
             <div class="setitem">Update feed&nbsp;<select id="update-rate"><option value="1">1</option><option value="2">2</option><option value="3">3</option><option value="4">4</option><option value="6">6</option><option value="8">8</option><option value="12">12</option><option value="24">24</option></select>&nbsp;times per day starting around&nbsp;<select id="update-start"><?php for ($i=0; $i<=24; $i++) { printf("<option value='%d'>%02d:00</option>",$i, $i);}?></select></div>
             <div class="setitem comment">Updates start 5 to 25 minutes after the hour</div>
-            <div class="setitem"><button id="do-update" name="do-update">Update</button>
+            <div class="setitem"><button id="do-update" name="do-update">Update schedule</button>
             <button id="update-disable" name="update-disable">Disable updates</button></div>
             </fieldset>
+        <fieldset id="transcode" class="setgroup">
+            <legend>Transcode</legend>
+            <div class="setitem comment"><span>Some devices - mainly Apple iOS - do not support open audio codecs like those used by Spotify. For such devices Spodcast can transcode ogg streams to mp3. This is an expensive operation which can take a substantial amount of time, especially on less powerful hardware. Only enable this option when there are no other options.</span></div>
+            <div class="setitem"><button id="transcode-enable" name="transcode-enable">Enable transcoder</button><button id="transcode-disable" name="transcode-disable">Disable transcoder</button></div>
+        </fieldset>
+        <fieldset id="logging" class="setgroup">
+            <legend>Logging</legend>
+            <div class="setitem comment"><span>The default log level shows which episodes have been downloaded. Change the log level to get more (or less) elaborate information on what Spodcast is up to. Log messages are reported in plain text format through scheduled updates and json-encoded through the webcron endpoint.</span></div>
+            <div class="setitem"><select id="log-level" onchange="logUpdate(this)"><?php foreach([['CRITICAL','Only show critical errors'],['ERROR','Show error messages'],['WARNING','Show which episodes have been downloaded'],['INFO','Show downloaded as well as skipped episodes'],['DEBUG','Show detailed internal information for debugging purposes']] as $s) { printf("<option value='%s'>%s</option>",$s[0], $s[1]);}?></select></div>
+        </fieldset>
+
+
         </div>
     </div>
     <div id="error" class="hidden">
@@ -461,25 +720,156 @@ $UPDATE_RATE=$settings['update_rate'];
         <div class="errorline"><button id="hide-error" onclick="window.location.href='.'; return false;">Close this message</button></div>
     </fieldset>
     </div>
+    <div class="hidden" id="toast"></div>
     <script>
+    function xhr(type, url, data, options) {
+        options = options || {};
+        var request = new XMLHttpRequest();
+        request.open(type, url, true);
+        if(type === "POST") {
+            request.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
+        }
+        request.onreadystatechange = function () {
+            if (this.readyState === 4) {
+                if (this.status >= 200 && this.status < 400) {
+                    options.success && options.success(parse(this.responseText));
+                } else {
+                    options.error && options.error(this.status);
+                }
+            }
+        };
+        request.send(data);
+    }
+    function parse(text) {
+        try {
+            return JSON.parse(text);
+        } catch(e) {
+            return text;
+        }
+    }
+    function basename(path) {
+        return path.replace(/.*\//, '');
+    }
     function newfeed() {
-        document.getElementById("loading").classList.remove("hidden");
+        xhr("POST", '?action=new', 'url=' + encodeURI(document.getElementById('url').value), {success:addShow,error:removeShade});
-        window.location.href = '?action=new&url=' + encodeURI(document.getElementById('url').value);
+    }
+    function addShow(info) {
+        title="Adding show (" + info.url + "), please wait...";
+        show=document.createElement('div'); show.setAttribute('class', 'show placeholder'); show.setAttribute('id', info.id);
+        shade=document.createElement('div'); shade.setAttribute('class', 'overlay');
+        logo=document.createElement('div'); logo.setAttribute('class', 'entry logo loader');
+        title=document.createElement('div'); title.setAttribute('class', 'entry title');
+        feedlist=document.getElementById('feed-list');
+        show.appendChild(shade); show.appendChild(title); show.appendChild(logo);
+        title.innterHTML = title;
+        first=feedlist.firstElementChild;
+        feedlist.insertBefore(show, first);
+        waitUpdate(info);
     }
     function login() {
-        window.location.href = '?action=login&username=' + document.getElementById('username').value + '&password=' + document.getElementById('password').value;
+        xhr("POST", '?action=login', 'username=' + document.getElementById('username').value + '&password=' + document.getElementById('password').value, {success:showToast, error:showToast});
+    }
+    function refresh(url) {
+        unhide(basename(url));
+        id = basename(url);
+        max=document.getElementById('max-' + id).value;
+        keep=document.getElementById('keep-' + id).value;
+        xhr("POST", '?action=refresh', 'url=' + encodeURI(url) + '&max=' + max + '&keep=' + keep, {success:wait, error:removeShade});
+    }
+    function waitUpdate(info) {
+        if (info.status == 'READY' || info.status == 'ERROR') {
+            window.location.href = '';
+        }
+        if (info.show != null) {
+            document.getElementById(info.id).childNodes.item(1).innerHTML = "Adding show: " + info.show.title + " (" + info.show.episodes + " episodes of 3 downloaded)...";
+        } else {
+            document.getElementById(info.id).childNodes.item(1).innerHTML = "Adding show: " + info.url + "...";
+        }
+        setTimeout(function() {
+                xhr("POST", '?action=status', 'url=' + encodeURI(info.url), {success:waitUpdate, error:removeShade});
+                }, 2000);
+    }
+    function wait(info) {
+        if (info.status == 'READY' || info.status == 'ERROR') {
+            document.getElementById('stats-' + info.id).innerHTML = info.show.last + '&nbsp;(' + info.show.episodes + ')';
+            hide(info.id);
+        } else { 
+            if (info.show != null) {
+                document.getElementById('stats-' + info.id).innerHTML = info.show.last + '&nbsp;(' + info.show.episodes + ')';
+            }
+            setTimeout(function() {
+                    xhr("POST", '?action=status', 'url=' + encodeURI(info.url), {success:wait, error:removeShade});
+                    }, 2000);
+        }
+    }
+    function removeShade(info) {
+        hide(info.id);
+    }
+    function hide(id) {
+        document.getElementById(id).firstElementChild.classList.add('hidden');
+    }
+    function unhide(id) {
+        document.getElementById(id).firstElementChild.classList.remove('hidden');
     }
     function schedule(enable) {
-        window.location.href = '?action=schedule&enable=' + enable + '&rate=' + document.getElementById('update-rate').value + '&start=' + document.getElementById('update-start').value;
+        xhr("POST", '?action=schedule', 'enable=' + enable + '&rate=' + document.getElementById('update-rate').value + '&start=' + document.getElementById('update-start').value, {success:showToast, error:showToast});
+    }
+    function transcode(enable) {
+        xhr("POST", '?action=transcode', 'enable=' + enable, {success:showToast, error:showToast});
+    }
+    function logUpdate(e) {
+        xhr("POST", '?action=logging', 'level=' + e.value, {success:showToast, error:showToast});
     }
     function selectElement(id, val) {
         document.getElementById(id).value=val;
     }
     function showUpdate(e,url,field) {
-        window.location.href = '?action=update&url=' + encodeURI(url) + '&field=' + field + '&value=' + e.value;
+        id = basename(url);
+        max=document.getElementById('max-' + id);
+        keep=document.getElementById('keep-' + id);
+        maxval = max.value;
+        keepval = keep.value;
+        if(maxval > keepval) {
+            keeper = (maxval > 2) ? 5 : 2;
+            if (confirm("Sync count " + maxval + " is higher than Keep count " + keepval + "\n\n" +
+                "It does not make sense to refresh more expisodes than are kept in cache." +
+                "Press OK to continue, Keep will be set to " + keeper + ". Otherwise press Cancel")) {
+                keep.value = keeper;
+            } else {
+                keep.value = keepval;
+                max.value = maxval;
+                return false;
+            }
+        }
+        xhr("POST", '?action=update', 'url=' + encodeURI(url) + '&field=' + field + '&value=' + e.value, {success:showToast, error:showToast});
     }
     function confirmDelete(title, url) {
-        return confirm('You are about to delete the following feed:\n\nTitle: ' + title + '\nURL: ' + url + '\n\nAre you sure you want to delete this feed?');
+        if (confirm('You are about to delete the following feed:\n\nTitle: ' + title + '\nURL: ' + url + '\n\nAre you sure you want to delete this feed?')) {
+            xhr("POST", '?action=delete', 'url=' + encodeURI(url), {success:deleteShow, error:showToast});
+        }
+        return false;
+    }
+    function deleteShow(info) {
+        document.getElementById(info.id).remove();
+        showToast(info);
+    }
+    function showToast(info) {
+        var toast=document.getElementById('toast');
+        if (info.status == 'ERROR') {
+            toast.classList.remove('success');
+            toast.classList.add('error');
+        } else {
+            toast.classList.remove('error');
+            toast.classList.add('success');
+        }
+        toast.innerHTML=info.result;
+        toast.classList.remove("hidden");
+        window.setTimeout(function() {
+            toast = document.getElementById('toast');
+            toast.classList.add("hidden");
+            toast.classList.replace('error', 'success');
+            toast.innerHTML="";
+        }, 2000);
     }
 
     document.getElementById('new').onclick = function() {
@@ -500,24 +890,39 @@ $UPDATE_RATE=$settings['update_rate'];
     var schedule_enable = document.getElementById('update-enable');
     var schedule_disable = document.getElementById('update-disable');
     var schedule_entries = document.getElementById('update-settings');
+    var transcode_enable = document.getElementById('transcode-enable');
+    var transcode_disable = document.getElementById('transcode-disable');
     schedule_entries.style.display = '<?php echo ($UPDATE_ENABLED ? 'initial' : 'none');?>';
     schedule_enable.style.display = '<?php echo ($UPDATE_ENABLED ? 'none' : 'initial');?>';
+    transcode_enable.style.display = '<?php echo ($TRANSCODE_ENABLED ? 'none' : 'initial');?>';
+    transcode_disable.style.display = '<?php echo ($TRANSCODE_ENABLED ? 'initial' : 'none');?>';
 
     schedule_enable.onclick = function() {
         schedule_entries.style.display = 'initial';
         schedule_enable.style.display = 'none';
+        schedule(true);
     };
     schedule_disable.onclick = function() {
         schedule_entries.style.display = 'none';
         schedule_enable.style.display = 'initial';
         schedule(false);
     };
-
+    transcode_enable.onclick = function() {
+        transcode_enable.style.display = 'none';
+        transcode_disable.style.display = 'initial';
+        transcode(true);
+    };
+    transcode_disable.onclick = function() {
+        transcode_enable.style.display = 'initial';
+        transcode_disable.style.display = 'none';
+        transcode(false);
+    };
     selectElement('update-rate', '<?=$UPDATE_RATE?>');
     selectElement('update-start', '<?=$UPDATE_START?>');
+    selectElement('log-level', '<?=$LOG_LEVEL?>');
     <?php foreach($feeds as $url => ["keep"=>$keep,"max"=>$max]) {
-        printf("    selectElement('keep-%s', '%d');\n", htmlspecialchars($url), $keep);
+        printf("    selectElement('keep-%s', '%d');\n", basename($url), $keep);
-        printf("    selectElement('max-%s', '%d');\n", htmlspecialchars($url), $max);
+        printf("    selectElement('max-%s', '%d');\n", basename($url), $max);
     }?>
     <?php if ($ERROR_MESSAGE != null) {
         echo "    document.getElementById('error').classList.remove('hidden');";
@@ -525,3 +930,4 @@ $UPDATE_RATE=$settings['update_rate'];
     </script>
 </body>
 </html>'''
+

spodcast/podcast.py

@@ -7,8 +7,10 @@ import urllib.parse
 
 from librespot.metadata import EpisodeId
 
+import ffmpeg
+
 from spodcast.const import ERROR, ID, ITEMS, NAME, SHOW, DURATION_MS, DESCRIPTION, RELEASE_DATE, URI, URL, EXTERNAL_URLS, IMAGES, SPOTIFY, FILE_EXISTS
-from spodcast.feedgenerator import RSS_FEED_CODE, RSS_FEED_FILE_NAME, RSS_FEED_SHOW_INDEX, RSS_FEED_INFO_EXTENSION
+from spodcast.feedgenerator import RSS_FEED_CODE, RSS_FEED_FILE_NAME, RSS_FEED_SHOW_INDEX, RSS_FEED_INFO_EXTENSION, RSS_FEED_SHOW_IMAGE, RSS_FEED_VERSION, get_index_version
 from spodcast.spotapi import EPISODE_INFO_URL, SHOWS_URL, EPISODE_DOWNLOAD_URL, ANON_PODCAST_DOMAIN
 from spodcast.utils import clean_filename
 from spodcast.spodcast import Spodcast
@@ -70,6 +72,8 @@ def download_file(url, filepath):
     import shutil
     import requests
 
+    mimetype = "audio/mpeg"
+
     r = requests.get(url, stream=True, allow_redirects=True)
     if r.status_code != 200:
         r.raise_for_status()  # Will only raise for 4xx codes, so...
@@ -83,24 +87,29 @@ def download_file(url, filepath):
         and abs(file_size - os.path.getsize(filepath)) < 1000
         and Spodcast.CONFIG.get_skip_existing_files()
     ):
-        return filepath, FILE_EXISTS
+        return filepath, FILE_EXISTS, mimetype
 
     log.info("Downloading file")
     r.raw.read = functools.partial(r.raw.read, decode_content=True)
     with open(filepath, "wb") as file:
         shutil.copyfileobj(r.raw, file)
 
-    return filepath, os.path.getsize(filepath)
+
+    return filepath, os.path.getsize(filepath), mimetype
 
 def download_stream(stream, filepath):
     size = stream.input_stream.size
+    mp3_filepath = os.path.splitext(filepath)[0] + ".mp3"
+    mimetype = "audio/ogg"
 
     if (
-        os.path.isfile(filepath)
+        ((os.path.isfile(filepath)
-        and abs(size - os.path.getsize(filepath)) < 1000
+        and abs(size - os.path.getsize(filepath)) < 1000)
+        or (Spodcast.CONFIG.get_transcode()
+        and os.path.isfile(mp3_filepath)))
         and Spodcast.CONFIG.get_skip_existing_files()
     ):
-        return filepath, FILE_EXISTS
+        return filepath, FILE_EXISTS, mimetype
  
     log.info("Downloading stream")
     time_start = time.time()
@@ -117,7 +126,18 @@ def download_stream(stream, filepath):
                 if delta_want > delta_real:
                     time.sleep(delta_want - delta_real)
 
-    return filepath, downloaded
+    if Spodcast.CONFIG.get_transcode():
+        log.info("transcoding ogg->mp3")
+        transcoder = ffmpeg.input(filepath)
+        transcoder = ffmpeg.output(transcoder, mp3_filepath)
+        ffmpeg.run(transcoder, quiet=True)
+        file.close()
+        os.unlink(filepath)
+        filepath = mp3_filepath
+        downloaded = os.path.getsize(filepath)
+        mimetype = "audio/mpeg"
+
+    return filepath, downloaded, mimetype
 
 
 def download_episode(episode_id) -> None:
@@ -140,21 +160,19 @@ def download_episode(episode_id) -> None:
             stream = Spodcast.get_content_stream(episode_stream_id, Spodcast.DOWNLOAD_QUALITY)
             basename = f"{filename}.ogg"
             filepath = os.path.join(show_directory, basename)
-            path, size = download_stream(stream, filepath)
+            path, size, mimetype = download_stream(stream, filepath)
-            mimetype="audio/ogg"
+            basename = os.path.basename(path) # may have changed due to transcoding
         else:
             basename=f"{filename}.mp3"
             filepath = os.path.join(show_directory, basename)
-            path, size = download_file(download_url, filepath)
+            path, size, mimetype = download_file(download_url, filepath)
-            mimetype="audio/mpeg"
 
         if size == FILE_EXISTS:
             log.info(f"Skipped {podcast_name}: {episode_name}")
-            return
         else:
             log.warning(f"Downloaded {podcast_name}: {episode_name}")
 
-        if Spodcast.CONFIG.get_enable_rss_feed():
+            if Spodcast.CONFIG.get_rss_feed():
                 episode_info = {
                         "mimetype": mimetype,
                         "medium": "audio",
@@ -167,21 +185,27 @@ def download_episode(episode_id) -> None:
                 info_file.write(json.dumps(episode_info))
                 info_file.close()
 
+        if Spodcast.CONFIG.get_rss_feed():
             show_index_file_name = os.path.join(show_directory, f"{RSS_FEED_SHOW_INDEX}.{RSS_FEED_INFO_EXTENSION}")
-            if not os.path.isfile(show_index_file_name):
+            if not os.path.isfile(show_index_file_name) or int(get_index_version(show_index_file_name)) < Spodcast.CONFIG.get_version_int():
                 podcast_name, link, description, image = get_info(episode_id, "show")
                 show_info = {
+                        "version": str(RSS_FEED_VERSION + Spodcast.CONFIG.get_version_str()),
                         "title": escape(podcast_name),
                         "link": link,
                         "description": escape(description),
-                        "image": image }
+                        "image": RSS_FEED_SHOW_IMAGE }
                 show_index_file = open(show_index_file_name, "w")
                 show_index_file.write(json.dumps(show_info))
                 show_index_file.close()
 
+            show_image_name = os.path.join(show_directory, f"{RSS_FEED_SHOW_IMAGE}")
+            if not os.path.isfile(show_image_name):
+                download_file(image, show_image_name)
+
             rss_file_name = os.path.join(show_directory, RSS_FEED_FILE_NAME)
-            if not os.path.isfile(rss_file_name):
+            if not os.path.isfile(rss_file_name) or int(get_index_version(rss_file_name)) < Spodcast.CONFIG.get_version_int():
                 rss_file = open(rss_file_name, "w")
-                rss_file.write(RSS_FEED_CODE())
+                rss_file.write(RSS_FEED_CODE(Spodcast.CONFIG.get_version_str()))
                 rss_file.close()
 

spodcast/spodcast.py

@@ -15,7 +15,7 @@ from librespot.core import Session
 
 from spodcast.config import Config
 from spodcast.const import CREDENTIALS_PREFIX, TYPE, USER_READ_EMAIL, OFFSET, LIMIT
-from spodcast.feedgenerator import RSS_FEED_FILE_NAME, RSS_INDEX_CODE
+from spodcast.feedgenerator import RSS_FEED_FILE_NAME, RSS_INDEX_CODE, get_index_version
 
 class Spodcast:    
     SESSION: Session = None
@@ -34,9 +34,9 @@ class Spodcast:
             os.makedirs(root_path, exist_ok=True)
             if os.path.exists(root_path):
                 index_file_name = os.path.join(root_path, RSS_FEED_FILE_NAME)
-                if not os.path.isfile(index_file_name):
+                if not os.path.isfile(index_file_name) or int(get_index_version(index_file_name)) < Spodcast.CONFIG.get_version_int():
                     rss_file = open(index_file_name, "w")
-                    rss_file.write(RSS_INDEX_CODE(Spodcast.CONFIG.get_bin_path(), os.path.basename(Spodcast.CONFIG.get_config_path())))
+                    rss_file.write(RSS_INDEX_CODE(Spodcast.CONFIG.get_bin_path(), os.path.basename(Spodcast.CONFIG.get_config_path()), Spodcast.CONFIG.get_version_str()))
                     rss_file.close()
             else:
                 sys.exit(f"Can not create root path {root_path}")
@@ -142,7 +142,7 @@ class Spodcast:
         responsejson = response.json()
 
         if 'error' in responsejson:
-            if tryCount < (cls.CONFIG.get_retry_attempts() - 1):
+            if tryCount < (cls.CONFIG.get_retry() - 1):
                 Spodcast.LOG.warning(f"Spotify API Error (try {tryCount + 1}) ({responsejson['error']['status']}): {responsejson['error']['message']}")
                 time.sleep(5)
                 return cls.invoke_url(url, tryCount + 1)

spodcast/utils.py

@@ -4,7 +4,7 @@ import unicodedata
 from enum import Enum
 from typing import List, Tuple
 
-from .spodcast import Spodcast
+from spodcast.spodcast import Spodcast
 
 valid_filename_chars = "-_.() %s%s" % (string.ascii_letters, string.digits)