Pull Mode

From SpinetiX Support Wiki

(Redirected from Pull mode)
Jump to: navigation, search

Contents

"Documentation"
SpinetiX Technical Documentation - Version 3.1.0 (2014-10-13)

Summary information and/or updates to the documentation can be found on this page.

Introduction

Pull Mode is one of the content management methods of the HMP whereby the content stored on a remote web server is periodically pulled by the HMP onto its local storage. Basically, when using the Pull Mode, the HMP is instructed to download content from a server at specific intervals and each time the HMP will do the following:

  1. Connect to the specified web server.
  2. Compare the content on the web server with the content present on the local storage.
  3. Remove from the local storage all files that are no longer present on the web server.
  4. Download the new files and the modified ones to the local storage.
    • The Etag and/or Last-Modified HTTP directives are used to determine if a file was modified or not - see Caching page for more details.
    • Unmodified files will not be downloaded.

Note: The order of the last two steps can be changed using the option -deletelast .

Advantages

  • Updates of the content can be programmed at predefined times or under request (using RPC).
  • The HMP will copy the content to its local storage, making the system resilient to network failures. Moreover, the uploading process was designed to work even for very poor connections. There is an initial 10 minutes time-out to establish the connection and later another 10 minutes time-out during the transmission of data.
  • Content on HMPs behind firewall or NAT can be updated practically from anywhere.
  • The Pull Mode mechanism can also be used to upload logs and snapshot images to a remote web server for monitoring purposes.

Pull Mode and Fusion

When Fusion mode is enabled on the device, you can still use Pull mode (firmware 2.2.3 or later is required), thus allowing a mix of Fusion and externally managed content, however that content cannot be directly accessed from Fusion like any other media. Instead you need to reference such files within a custom skin, a custom slide template, an hypermedia project or any combination of these.

The default destination for a Pull operation is no longer the root folder, but a sub-folder named "publish". In case an ics file is used to control the Pull Mode, then make sure to specify the destination folder like this: -dest publish.

  • Note: Using the manual settings to configure the publish action is currently not working. Use the ics method instead.

Bandwidth management

* See full article Bandwidth management.

Configure Pull Mode on HMP

Content Settings - Pull Mode tab

Pull Mode must be enabled on the "Pull Mode" page from Control Center, using one of the three available options.

  • Use the Manual settings section to configure the pull mode to run once per day:
    • For scheduling a daily update of the project from the server:
      1. Enable "Automatically upload project to the HMP..." option.
      2. Enter the location of the project on the web server.
      3. Enter the daily update time.
    • For scheduling a daily update of the HMP logs to the server:
      1. Enable "Automatically upload logs from the HMP..." option.
      2. Enter the destination of the logs on the web server.
      3. Enter the daily upload time.
      4. Specify if you want the "Accounting logs only" or "All logs" to be uploaded.
  • For more complex schedules, use one of the iCalendar file sections: "From uploaded iCalendar file (ics)" or "From remote iCalendar file (ics)").
    • See the section How to create an iCalendar file to control Pull Mode for more details.
    • When using a remote ics file, the option "Check calendar every" can be used to specify the maximum duration after which the HMP will recheck the calendar for updates. Note that if there is an event during the time, the player will automatically check for updates before executing that event and the timer will be reset.

Web servers

Pull Mode publish action can be used with any standard web server, with or without support for WebDAV.

  • A WebDAV compatible web server is recommended and most popular web servers, like Apache or Microsoft IIS, do support WebDAV, though usually it must be manually enabled. There are also some public WebDAV servers that can be used, for instance myDrive.ch.
  • If WebDAV cannot be used, the HMP can pull from a plain HTTP web server as well, but since there is no standard method of listing the content of a repository using standard HTTP commands, an XML repository descriptor file must be available on the web server.
    • Starting with version 3.1.0, Elementi M and Elementi X automatically generate this file within the project during publishing. See Publish Location page for more details about this feature.

Notes:

  • Write access rights on the web server are required only if the upload action is used.
  • The port used by Pull Mode is the one you have specified when entering the project source / log destination - in the absence of such port, then the default port of the scheme is used, meaning 80 for http and 443 for https.
  • Although an HMP can technically be used as a WebDAV server, this configuration is not recommended because the HMP can only respond to maximum two concurrent clients (this includes HMP Control Center and Fusion) and the performance on the host HMP will be degraded while the file reconciliation is taking place.
  • See also the generic remarks when using Web servers.

Apache

For details on how to enable mod_dav module on Apache HTTP Server Version 2.2, see the official Apache Module mod_dav page.

Microsoft IIS

  • Internet Information Services (IIS) is included with all Professional versions of Windows, so it can be used for testing, usually without further installation. However, by default, IIS is heavily locked down with regard to WebDAV.
  • For a full guide to setting up Microsoft IIS 7 as a Pull Mode source, see the Configuring IIS as server for Pull mode tutorial.
  • IIS 7 introduced a new WebDAV stack which, among other changes, restricts WebDAV functions for unauthenticated (anonymous) users to PROPFIND.
  • PUT, MKCOL, PROPPATCH, COPY, MOVE, DELETE, and WebDAV-based GET requests all require authentication (see this Microsoft page for more details).
  • Ensure that SVG is listed in the allowed MIME types, and directory browsing is enabled if you are using log uploads.
  • If using basic authentication, in the IIS site configuration ensure that only basic is enabled. Enabling multiple types will result in the HMP not being able to connect.
  • See #Troubleshooting and common errors in Pull mode below for more tips. Also see the Web servers page for more general information.

MyDrive.ch

To use the WebDAV-enabled public server myDrive.ch with Pull Mode, check out Tutorial:Using myDrive as content server.

Box.com

To use the WebDAV-enabled public server box.com with Pull Mode, follow these instructions:

  1. Create an account on box.com
  2. Create a Publish Location and then publish your project from Elementi. Alternatively, you can upload your project manually or using a WebDAV client.
  3. Open HMP Control Center > Network Settings page > Credentials tab.
  4. Add https://dav.box.com/dav/ as Server URI and your box.com username and password.
  5. Go to the Content Settings page > Pull Mode tab and set the project source to: https://dav.box.com/dav/ProjectName/.

Dropbox

To use the Dropbox public server with Pull Mode, check out Tutorial:Using Dropbox as content server.

NAS devices

  • It is often overkill to use a full enterprise-class, rackmounted server to host content for Pull mode. SpinetiX does not recommend specific NAS servers, but mid-range manufacturers such as QNAP and Netgear have performance NAS appliances which support WebDAV and are of a much more practical form factor than rack servers. They usually avoid unnecessary license costs.

Sample PHP scripts

Also check the RPC management samples for the management of the HMP

File pull

SpinetiX is providing an automatic generation of XML description listing the content of a files folder (written in PHP) in order to make it easier for you to develop your own HTTP based Pull mode server.

  • Make sure you read the README file before getting started!

Database pull

SpinetiX is providing a server side application for managing and testing Pull mode functionality of HMP devices. Its purpose is to serve as an example and guide implementation for a simple Pull mode server for HMP devices based on a Database.

All pull mode server activity is done through an SQLite database.

  • Make sure you read the README file before getting started!

Control the Pull Mode with an ics file

An iCalendar file (.ics) can be used to accurately control the actions performed by the HMP when working in Pull mode, for instance to:

  • Update the content of the HMP more (or less) than once per day and even from different locations.
  • Schedule the update using complex recurring rules.
  • Upload different files to a server: logs, snapshot images, report file.
  • Enable the communication with a remote RPC server.

The ics file for controlling the Pull mode actions can be created using different methods: Elementi, HMD, Outlook, Google Calendar or manually with any text editor.

  • Usually, it's easier to create the ics file within an application. In this case, if you want to see to contents of the ics file, simply open it within a text editor.

The ics file can be directly uploaded on the HMP (from Control Center) or it can be uploaded on a remote web server from which the HMP will be instructed to retrieved it.

Events and actions

Each action of Pull mode is described by an event (item, meeting) within the calendar file and this event must have a certain format.

The following iCalendar fields are used to define the action (in parenthesis the alternative names used within applications for these fields):

  • SUMMARY (also referred to as: Title or Subject) - specifies the action to be performed and can take one of the following values:
    • publish - to publish a project (or just some files) from a server to the HMP,
    • upload - to upload different files from the HMP to a server,
    • rpc - to enable sending RPC statuses to a remote server (RPC Concentrator).
    • inline-rpc - to trigger an RPC command.
  • URL (also referred to as: URI, Location, Where) - specifies the URI of the action.
  • DTSTART and DTEND (also referred to as: From & To, Start Time & End time) - specifies the starting and ending date and time for the event (or for the first event of a series if the recurring rule is used).
  • RRULE - specifies the recurrence rule(s) applying to this event.
  • DESCRIPTION - specifies additional options for the upload or rpc actions.
  • Other fields, such as: UID, VTIMEZONE, PRIORITY etc., can also be used within the ics file.

Duration of an event

The duration of an event represents the time period between DTSTART and DTEND, during which the player will try to execute the specified action (with other words, it's the validity period of the action, not the duration of the action itself, which can take far less or even more that the duration of the event).

  • If the action is not successful for any reasons (for instance, the server cannot be contacted), the player will retry the same action until the DTEND is reached; at that point the action is discarded even if the action has not been executed.
  • Once the action is successful, the event is removed from the Pull mode queue (event if the DTEND has not been reached).

Note Note: If the player is restarted, the ics file controlling the Pull mode will be reprocessed, which means that some past events might be executed again if their DTEND is still in the future.

Recurrence rules

When using recurrence rules, make sure that no more than 100 events are generated for any interval of 8 hours, for instance:

  • In case of only one event / action, the repeating interval should not be less than 5 minutes.
  • In case of two events, the repeating interval for each event should not be less than 10 minutes.

Note Note: A special care should be taken to avoid overlapping an event by the other events from its series.

  • For instance, for an action than needs to be executed every two hours, the duration of the event should not be more than two hours, like in the following example:
DTSTART:20130101T000000 
DTEND:20130101T010000
RRULE:FREQ=HOURLY;INTERVAL=2;WKST=SU

Publish action

The publish action controls when the content of the device should be updated from a remote server. At the end of a successful publish action, the content of the device is an exact copy of the content on the server.

  • The SUMMARY must be set to publish.
  • The URL must point to the remote location (folder) of the files for an WebDAV server or to the xml file with the repository description for a standard HTTP server.
  • The DESCRIPTION can contain: -deletelast, -dest folder, -norecurs.
  • The duration of the publish action is important in case of errors while downloading files. If an error occurs or the server cannot be reached, the Pull mode will retry the downloading process for the entire duration of the event.

Example:
To instruct the player to download / check for updates a project from SpinetiX server, each day starting at 6 AM (and up to 7 AM), the ics file would look like this:

BEGIN:VCALENDAR 
VERSION:2.0 
PRODID:-//Spinetix.com/NONSGML Spinetix Control Center//EN 
BEGIN:VEVENT 
UID:14d64d53-a926-4a71-848d-c78452b195fe 
DTSTART:20130101T060000 
DTEND:20130101T070000
RRULE:FREQ=DAILY;WKST=SU  
SUMMARY:publish URL: http://demo.spinetix.com/project/default/publish.xml
DESCRIPTION:-deletelast
END:VEVENT 
END:VCALENDAR

Note Note: See also the publish_screen feature.

Upload action

The upload action can be used to upload different files (logs, snapshot images, report file) from the HMP to a remote server (usually a WebDAV enabled server, though a standard HTTP server can be used as well). This action doesn't effect in any way the content playing on the HMP.

  • The SUMMARY must be set to upload.
  • The URL must point to the remote location (folder) where the files will be uploaded. By default the Pull mode uses the PUT command to upload files, but it can be changed to POST for a regular HTTP server, using the -method post option.
  • The DESCRIPTION field contains a list of files to be uploaded (<file_name> [destination_name]) and/or the options (-format, -method <post|put>, -buffer, -data, -snapshot, -log <name>, -logs, -alllogs, -currentlogs, -pastlogs, -accounting, -report, -content).
    • The DESCRIPTION can contain multiple options separated with a '\n' line feed.

Example:
To instruct the player to upload two log files to a server using a custom file name format, every two hours, the ics file would look like this:

BEGIN:VCALENDAR 
VERSION:2.0 
PRODID:-//Spinetix.com/NONSGML Spinetix Control Center//EN 
BEGIN:VEVENT 
UID:14d64d53-a926-4a71-848d-c78234b195fg 
DTSTART:20130101T000000 
DTEND:20130101T010000
RRULE:FREQ=HOURLY;INTERVAL=2;WKST=SU
SUMMARY:upload URL: http://webdav.spinetix.com/logs/ 
DESCRIPTION:-format %s/%d-%n.%x\n-accounting\nhttp://localhost/log/warn.log 
END:VEVENT 
END:VCALENDAR

Using Pull mode in a pseudo-push scenario

"Info"
Starting from firmware 2.2.3 the add_pull_action() RPC command should be used to trigger a pull dynamically. It is more flexible than the method described bellow.

In some rare scenarios, you may want to configure Pull mode to checks for updates very regularly (e.g. 5 mins). In a normal setup, this would consume many network resources and put unnecessary load on the HMP. To avoid this:

  • Configure the HMP to use a dummy ICS file on your remote server for the Pull schedule.
  • Configure the Check calendar every : to a small duration (1m or 5m for instance).
    The player will contact the server every time to check if the ICS file has been modified. It will use ETag and HEAD requests to check for a new file (see Caching page).
  • By default the server will return an empty ICS file.
  • To trigger a content Pull, change the ICS file on the server to add a single event taking place between some short time in the past, to a time in the future long enough for the Pull to complete (2 hours for instance).
    The event location should contain either the URI to the WebDAV server hosting the project, or to the XML describing the list of file to download if you are not using a WebDAV server.
  • This way, the player does not need to check the its entire storage at the interval specified, unless the ics file has changed.

Troubleshooting

Common problems

"Info"
Errors regarding Pull Mode operations are stored in the uploader.log file.
  • The error "Could not read status line: connection was closed by server " means that the server is reachable but it is returning an error on the path being used. Check the capitalisation, spelling and syntax of the URL is correct.
  • 404: File Not Found errors: Ensure that the not-found file is added as a valid MIME type for the WebDAV server you are using.
    • In most cases that's caused by IIS server not allowing the SVG files (the MIME type is: image/svg+xml).
  • IIS returns "unauthorised" errors: Check that the user IIS_IUSR is added with Read permissions on the Virtual Directory you are using.
  • Spaces in the path in Pull Mode may cause problems. Use underscores instead of spaces.

Enable extended logging

The extended logging of Pull Mode actions should only be enabled when asked by SpinetiX Support. To do so, follow this procedure:

  1. Open HMP Control Center.
  2. On the "Status" page, find the HMP picture and click where the SpinetiX logo (in case of an HMP200) or the blue button (in case of an HMP130 / HMP100) is displayed.
  3. A confirmation dialog will appear; click on "Yes" to enable the "Advanced settings" menu on the left side.
  4. Go to Content Settings > Pull Mode tab > Log settings section.
  5. Set the "Log level" option to "Trace".
  6. Click the "Apply" button.
  7. Wait for the problematic action to be executed by the HMP, then generate a report and send it to SpinetiX Support for further analysis.

Note Note: Once the issue has been identified / solved, make sure to set the log level back to "Info".