Pull Mode

From SpinetiX Support Wiki

(Redirected from Pull mode)
Jump to: navigation, search
Version: 2.0 (revision history)
Release date: January 8, 2015

Summary information and/or updates to the documentation can be found on this page.


Pull Mode is a content transfer mechanism of the Hyper Media Player, mainly used for updating the player content in particular scenarios, where:

  • Players are located in private networks protected by firewall / NAT and direct network access is not allowed for updating the content.
  • Players are deployed in particular environments (slow network connection, 4G) where resilience to network failures or network usage optimization are mandatory.
  • Content is generated by third-party web applications / CMS.
  • Content must be retrieved from different locations / servers, eventually managed by different persons, and merged into one.

Pull Mode main feature is smart content update, yet it can also be used to upload certain files from the player onto a remote server, or as RPC bridge.

To use Pull Mode with the HMP, you need to set up a content server and configure the player accordingly from Control Center. For advanced uses, a iCalendar (ics) file using a special syntax must be written and configured on the player.


Smart content update

This feature offers many options for automatic update of the content from remote web servers, at predefined intervals (hourly, daily etc.) or on demand.

When activated, the HMP will do the following:

  1. Connect to the specified content server.
  2. Get the list of files present on that location and compare it with the content present on its local storage.
  3. Remove from its local storage all files that are no longer present on the web server.
  4. Download all the new and modified files onto the local storage. Unmodified files will not be touched.
  5. When finishing downloading, the project's index.svg is restarted.
Note Notes:
  • Content on HMPs behind firewall or NAT can be updated practically from anywhere.
  • The HMP will copy the content to its local storage, making the system resilient to network failures. Once the content has been downloaded, the server is no longer needed until the next update check. The downloading process was designed to work even for very poor connections - there is an initial 10 minutes time-out to establish the connection and later another 10 minutes time-out during the transmission of data.
  • The smart content pull ensures downloading only new or modified files, not everything else, thus saving network traffic / bandwidth - the Etag and/or Last-Modified HTTP directives are used to determine if a file was modified or not. See Caching page for more details.
  • The manual configuration allows for content download from one location. See also Scheduled Download feature of HMP400/W, HMP350, and HMP300 models, which allows configuring multiple locations.
  • For advanced users, the publish action offers even more options for fine-tuning the content download - for instance, the order of the last two steps above can be changed using the -deletelast option.

Files upload

This feature offers the possibility to transfer files, mainly logs (see Scheduled Upload), from the player onto a specified content server.


To use Pull Mode with the HMP, you need to set up a content server and configure the player accordingly from Control Center. For advanced uses, a iCalendar (ics) file using a special syntax must be written and configured on the player.

In most cases, the server location would be protected and you'll need to configure the credentials to be used by the player to access that location.

HMP400/W, HMP350, HMP300

On these player models, the manual settings have been grouped under Content > Scheduled Download, allowing for content retrival from a server with a click of a button or in a scheduled manner (daily or hourly). See how to configure Scheduled Download on the player.

The advanced Pull Mode functionality, driven by an iCalendar (.ics) file, can be enabled from HMP Control Center > Advanced Applications > Pull Mode - two options are available:

  • From uploaded iCalendar file (ics)
  • From remote iCalendar file (ics)
Note Notes:

HMP200, HMP130, HMP100

Pull Mode can be enabled from HMP Control Center > Content Settings > Pull Mode tab, by selecting one of the three available options:

  • Manual settings
    Enable this option to manually configure the HMP to: Update the content (daily or hourly) and to upload logs (daily or hourly).
  • From uploaded iCalendar file (ics)
  • From remote iCalendar file (ics)
Note Notes:

Content server

Pull Mode's implies using a public or private content server to host the content to be downloaded by the player via HTTP protocol. Any standard web server can be used, with or without support for WebDAV protocol. Most popular ones, like Apache or Microsoft IIS, support WebDAV, though usually it must be manually enabled; there are also some public WebDAV servers that can be used, for instance box or myDrive. Otherwise, some extra steps might be required for content servers without WebDAV support.

Note Notes:
  • Write access rights on the web server are not required, unless the upload action is used.
  • The port used by Pull Mode is the one you have specified when entering the project source / log destination - in the absence of such port, then the default port of the scheme is used, meaning 80 for http and 443 for https.
  • Although an HMP can technically be used as a WebDAV server, this configuration is not recommended because the HMP can only respond to maximum two concurrent clients (this includes player web interface, HMP Control Center, Fusion) and the performance on the host HMP is degraded while the file reconciliation is taking place.
  • See also the generic remarks when using Web servers.

Content server with WebDAV support

Web Distributed Authoring and Versioning (WebDAV) is an extension of the Hypertext Transfer Protocol (HTTP) that allows clients to perform remote content authoring operations: create, change, move, and delete. The most important features of the WebDAV protocol include the maintenance of properties (such as the creation / modification date), namespace management, collections (creation, removal, and listing of various resources), and overwrite protection.

The HMP will use WebDAV's PROPFIND requests to retrieve first the collection structure (also known as directory hierarchy) of the remote server location and then the properties (such as the creation / modification date) of each file, so that only new or modified files are downloaded.

Most popular web servers, like Apache or Microsoft IIS, support WebDAV, though usually it must be manually enabled; there are also some public WebDAV servers that can be used. Here are some examples of web servers that can be used for Pull Mode:

Web server Notes
Apache Apache HTTP Server is a free and open-source cross-platform web server, developed and maintained by an open community of developers under the auspices of the Apache Software Foundation. For details on how to enable mod_dav module on Apache HTTP Server Version 2.2, see the official Apache Module mod_dav page.
Microsoft IIS Internet Information Services (IIS) is included with all Professional versions of Windows, so it can be used for testing, usually without further installation. However, by default, IIS is heavily locked down with regard to WebDAV.
  • IIS 7 introduced a new WebDAV stack which, among other changes, restricts WebDAV functions for unauthenticated (anonymous) users to PROPFIND.
  • PUT, MKCOL, PROPPATCH, COPY, MOVE, DELETE, and WebDAV-based GET requests all require authentication (see this Microsoft page for more details).
  • Ensure that SVG is listed in the allowed MIME types, and directory browsing is enabled if you are using log uploads.
  • If using basic authentication, in the IIS site configuration, ensure that only basic is enabled. Enabling multiple types will result in the HMP not being able to connect.

See also:

Box Box (formerly Box.net), based in Redwood City, California, is a cloud content management and file sharing service for businesses. The company uses a freemium business model to provide cloud storage and file hosting for personal accounts and businesses.

To use this WebDAV-enabled public server with Pull Mode, check out this tutorial: Use box.com as content server.

Note that as of October 25, 2019, Box has ended WebDAV support.

MyDrive MyDrive is a free and secure online storage service, based in Switzerland. It offers 512 MB space as a freemium service.

To use this WebDAV-enabled, public server with Pull Mode, check out this tutorial: Use myDrive as content server.

NAS devices Network-attached storage (NAS) is a file-level computer data storage server connected to a computer network, providing data access to a group of clients. NAS systems are networked appliances which contain one or more storage drives, often arranged into logical, redundant storage containers or RAID. SpinetiX does not recommend specific NAS servers, but mid-range manufacturers such as Synology, QNAP, and Netgear have performance NAS appliances which support WebDAV and are of a much more practical form factor than rack servers. They usually avoid unnecessary license costs.

Two tutorials are available: Use Synology NAS as content server and Use QNAP NAS as content server.

Content server without WebDAV support

Content servers without WebDAV support can be used with Pull Mode, yet an additional step might be required to allow the player to get the list of files present on that location. Standard HTTP commands don't allow listing the content of a plain HTTP web server location for security reasons - the solution is to use an XML repository descriptor file and configure the player to point to this XML file instead of the root of the repository.

When using the Publish Location feature of Elementi M / X such an XML file (named "spx-listing.xml") is automatically generated within the root of the repository. Otherwise, either use a script that generates it (see sample PHP scripts below) or create it manually, following the instructions detailed on the "XML repository descriptor file" page.

Advanced uses

Control Pull Mode with an ics file

For accurate control over the Pull Mode's actions to be performed by the player, create an iCalendar file (.ics) according to the API specification and configure it on the HMP. The ics file can be directly uploaded on the HMP (from Control Center) or it can be uploaded on a remote web server from which the HMP will be instructed to retrieved it.

This ics file can be created using Elementi, Outlook, Google Calendar or manually within a text editor. It can be used to:

  • Update the player's content according to a predefined schedule, including using complex recurring rules, from one or more server locations.
  • Upload more player's logs, as well as snapshot images and the player's report onto a designated server.
  • Enable the communication with a RPC Concentrator and / or execute RPC commands.

Dynamic pull

The add_pull_action RPC command should be used to trigger a pull action dynamically, as it is more flexible than the method below.

In some rare scenarios, you may want to configure Pull mode to checks for updates very regularly (e.g. 5 mins). In a normal setup, this would consume many network resources and put unnecessary load on the HMP. To avoid this:

  • Configure the HMP to use a dummy ICS file on your remote server for the Pull schedule.
  • Set Check calendar every : to a small duration (1m or 5m for instance).
    The player will contact the server every time to check if the ICS file has been modified. It will use ETag and HEAD requests to check for a new file (see Caching page).
  • By default the server will return an empty ICS file.
  • To trigger a content Pull, change the ICS file on the server to add a single event taking place between some short time in the past, to a time in the future long enough for the Pull to complete (2 hours for instance).
    The event location should contain either the URI to the WebDAV server hosting the project, or to the XML describing the list of file to download if you are not using a WebDAV server.
  • This way, the player does not need to check the its entire storage at the interval specified, unless the ics file has changed.

Pull Mode and Fusion

Applies to HMP200, HMP130, HMP100.

When Fusion mode is enabled on the device, you can still use Pull mode (firmware 2.2.3 or later is required), thus allowing a mix of Fusion and externally managed content, however that content cannot be directly accessed from Fusion like any other media. Instead you need to reference such files within a custom skin, a custom slide template, an hypermedia project or any combination of these.

The default destination for a Download operation is no longer the root folder, but a sub-folder named "publish". In case an ics file is used to control the Pull Mode, then make sure to specify the destination folder like this: -dest publish.

Sample PHP scripts

Also check the RPC server-side application samples for remote managing of the HMP.

Files pull

Version: 1.0 (revision history)
Release date: 2013-06-13

SpinetiX is providing an automatic generation of XML description listing the content of a files folder (written in PHP) in order to make it easier for you to develop your own HTTP based Pull Mode server.

Note Note:
Make sure to check the README file before getting started!

Database pull

Version: 1.0 (revision history)
Release date: 2013-06-13

SpinetiX is providing a server side application for managing and testing Pull Mode functionality of HMP devices. Its purpose is to serve as an example and guide implementation for a simple Pull Mode server for HMP devices based on a Database. All Pull Mode server activity is done through an SQLite database.

Note Note:
Make sure to check the README file before getting started!

API specification

This section details the syntax to use within the iCalendar file (.ics) used for advanced uses of Pull Mode.


Each action of Pull mode is described by an event (item, meeting) within the ICS file, and this event must have a certain format.

The following iCalendar fields are used to define the action:

  • SUMMARY (also referred to as: Title or Subject) - specifies the action to be performed, as one of the following:
    • publish - to publish a project (or just some files) from a server to the HMP (a.k download),
    • upload - to upload different files from the HMP to a server,
    • rpc - to enable sending RPC statuses to a remote server (RPC Concentrator).
    • inline-rpc - to trigger an RPC command. See more details on Inline RPC.
  • URL (also referred to as: URI, Location, Where) - specifies the URI of the action.
  • DTSTART and DTEND (also referred to as: From & To, Start Time & End time) - specifies the starting and ending date and time for the event (or for the first event of a series if the recurring rule is used).
  • RRULE - specifies the recurrence rule applying to this event.
  • DESCRIPTION - specifies additional options for the upload or rpc actions.
  • Other fields, such as: UID, VTIMEZONE, PRIORITY etc., can also be used within the ics file.
Note Note:
In parentheses, there are the alternative names used within different applications for the above-mentioned iCalendar fields.

Duration of an event

The duration of an event represents the time period between DTSTART and DTEND, during which the player will try to execute the specified action (with other words, it's the validity period of the action, not the duration of the action itself, which can take far less or even more than the duration of the event).

  • If the action is not successful for any reasons (for instance, the server cannot be contacted), the player will retry the same action until the DTEND is reached; at that point the action is discarded even if the action has not been executed.
  • Once the action is successful, the event is removed from the Pull mode queue (event if the DTEND has not been reached).
Note Note:
If the player is restarted, the ICS file controlling the Pull mode will be reprocessed, which means that some past events might be executed again if their end-date is after the current time.

Recurrence rule

The RRULE property defines a repeating pattern for recurring events. It is used in computing the recurrence set of instances for that event, generated by considering the initial DTSTART property along with the RRULE, RDATE, and EXDATE properties contained within the recurring component. This property should not be specified more than once.

Here is an example of an event with a duration of 5m 30s, that is repeating every 15 minutes:

Recurring events limits

When using recurrence rules, make sure that the following limits are respected:

  1. No more than 100 events are generated for any interval of 8 hours!
    This means that the repeating interval should not be less than 5 minutes if there is only one event/action, while if there are two events, the repeating interval for each event should not be less than 10 minutes.
  2. Avoid overlapping an event by the other events from its series!
    This means that if an action needs to be executed every two hours, the duration of the event should not be greater than two hours - it could be exactly 2h, or it could be less (e.g., 1h 59m 59s).

Publish action

The publish action controls when the content of the device should be updated from a remote server. At the end of a successful publish action, the content of the device is an exact copy of the content on the server and the main index.svg file is reloaded.

The publish action uses the following fields:

  • SUMMARY:publish
  • URL:<server_location>
    This must point either to a remote location (folder) on an WebDAV-enabled server or to an XML repository descriptor file in the case of a standard HTTP server.
  • DESCRIPTION:<parameters>
    Optional field that can be used to specify additional parameters for the publish action, each separated by "\n" (newline character), from the following:
    • -deletelast
      By default, the files that need to be removed are deleted at the beginning of the publish action to save space on the HMP storage. When this option is specified, the file are deleted after the new / modified ones have been uploaded on the HMP.
    • -dest folder
      This option changes the destination of the publish from the content root folder (the default behavior) to an inner folder and the main index.svg file is no longer reloaded. The path entered as destination folder must be URI encoded and must not start by "/" or "\" - if the specified folder does not exist on the HMP storage, it will be created. Note that when Fusion is enabled, the publish action must be set like this: -dest publish, otherwise it won't be executed.
    • -norecurs
      Prevents recursion during the publish process; only the root folder (or the folder specified with -dest option) is modified and no other sub-folders are created or deleted.
    • -statusinterval n_seconds (since 3.0.3 release)
      Sends an update every n seconds to the RPC concentrator (if one is configured on the HMP).
    • -statusnrfiles n_files (since 3.0.3 release)
      Sends an update every n files to the RPC concentrator (if one is configured on the HMP).
    • -statussize n_bytes (since 3.1.0 release)
      Sends an update every n bytes to the RPC concentrator (if one is configured on the HMP).
Note Note:
If -statusinterval, -statusnrfiles or -statussize options are used together, an update status is (re)sent when each of the these conditions is reached.

To instruct the player to download / check for updates a project from SpinetiX server, each day starting at 6 AM (and up to 7 AM), the ICS file would look like this:

PRODID:-//Spinetix.com/NONSGML Spinetix Control Center//EN 
URL: http://demo.spinetix.com/project/default/publish.xml
Note Note:
See also the publish screen feature.

Upload action

The upload action can be used to upload different files (logs, snapshot images, report files) from the HMP to a remote server (usually a WebDAV enabled server, though a standard HTTP server can be used as well). This action doesn't affect in any way the content on the HMP.

The upload action uses the following fields:

  • SUMMARY:upload
  • URL:<server_location>
    It must point to a server location (e.g., http://webdav.spinetix.com/logs/) where the player should upload the specified files.
  • DESCRIPTION:[<options>\n]<file>[ <destination_name>]
    Contains one or more file upload instructions, eventually using a custom destination name and / or preceded by one or more options for the upload action. Each file upload instruction must be separated by "\n" (newline character).
    This can be an URI (e.g., http://localhost/status/snapshot) pointing to the file or one of the following shortcuts:
    • -accounting : to upload the accounting log files.
    • -alllogs : to upload all the logs, both current (plain text) and past (archived).
    • -content : to upload an XML repository descriptor file containing the list of files present on the player content server.
    • -currentlogs : to upload the current logs (plain text).
    • -log <name> : to upload a specific log file, being any of the regular logs (e.g., -log accounting.log, -log uploader.log etc.), but also the HMP info file (-log info) or the system log (-log syslog). Added in 3.0.0 release.
    • -logs : to upload the archived logs from the previous days; a check is done to make sure all past logs are uploaded. Note that at most 7 days of logs are kept onto the player.
    • -pastlogs : to upload the past logs (archived); no checks are done to find out whether any of the logs have already been uploaded on the server.
    • -report : to upload a standard report file of the player.
    • -snapshot : to upload the player snapshot.
    Specifies the naming format for the uploaded file through the following special placeholders:
    • %D : modification date of the uploaded file
    • %H : modification time of the uploaded file
    • %d : date of the upload
    • %h : time of the upload
    • %n : name of the source file
    • %s : serial number of the HMP
    • %x : extension of the source file
    If missing, the default file naming format is set to %n-%s-%d-%h.%x.
    One or more options can be specified for the following batch of files to upload, each option separated by "\n" (newline character).
    • -buffer true : to force the buffering of files sent to the server to ensure that the Content-Length header is valid (necessary for any upload to Amazon S3 or mydrive.ch for instance). Added in 3.0.6 release.
    • -data <key> <value> : to specify a key=value pair to be added to the POST request. This option can be specified multiple times in case more key/value pairs need to be added. Note: using -data reset will remove all the key / value pairs previously set. Added in 3.0.6 release.
    • -format <destination_name_syntax> : to specify a global file naming format, using the syntax specified above, for files specified using URIs; this doesn't apply to shortcuts.
    • -header <name> <value> : to specify additional HTTP headers to be added to the PUT / POST request. This option can be used multiple times in case more headers need to be added. Note: using -header reset will remove all the headers previously set. Added in 3.0.6 release.
    • -method post : to set the upload method as POST (make sense in the case of a regular HTTP server), instead of the default PUT method.
PRODID:-//Spinetix.com/NONSGML Spinetix Control Center//EN 
URL: http://webdav.spinetix.com/logs/ 
DESCRIPTION:-buffer true\n-currentlogs\n-snapshot %s-%n.jpg

This command instructs the player to upload every two hours the player's current log files and snapshot (using a custom name format) onto a server. The option -buffer true applies to all uploads.

Inline RPC action

See the full article about Inline RPC.


Common problems

Errors regarding Pull Mode operations are stored in the uploader.log file.
  • The error "Could not read status line: connection was closed by server " means that the server is reachable but it is returning an error on the path being used. Check the capitalisation, spelling and syntax of the URL is correct.
  • 404: File Not Found errors: Ensure that the not-found file is added as a valid MIME type for the WebDAV server you are using.
    • In most cases that's caused by IIS server not allowing the SVG files (the MIME type is: image/svg+xml).
  • IIS returns "unauthorised" errors: Check that the user IIS_IUSR is added with Read permissions on the Virtual Directory you are using.
  • Spaces in the path in Pull Mode may cause problems. Use underscores instead of spaces.

Enable extended logging

The extended logging of Pull Mode actions should only be enabled when asked by SpinetiX Support.

For HMP400/W, HMP350, and HMP300 devices, you need to apply the following config file:

<?xml version="1.0"?>
<configuration version="2.0">  
Note Note:
Replace trace by info when done.

For HMP200, HMP130, and HMP100 devices, follow this procedure:

  1. Open HMP Control Center.
  2. On the "Status" page, find the HMP picture and click over the SpinetiX logo (in case of an HMP200) or the blue button (in case of an HMP130 / HMP100).
  3. A confirmation dialog appears - click "Yes" to enable the "Advanced settings" menu on the left side.
  4. Go to "Content Settings" page > "Pull Mode" tab > "Log settings" section.
  5. Set the "Log level" option to "Trace".
  6. Click the "Apply" button.
  7. Wait for the problematic action to be executed by the HMP, then generate a report and send it to SpinetiX Support for further analysis.
Note Note:
Once the issue has been identified / solved, make sure to set the log level back to "Info".
This page was last modified on 24 January 2023, at 16:53.