Data feeds

Introduction

Data feed is a mechanism to deliver structured data from one system to another, like from a server to a client. Data feeds make it possible to automatically have new content or updates delivered to the player as soon as it is available.

Data feeds are often described in terms of their methods of delivery. The data feed can be in the form of a file (product information, corporate communication, information about current or upcoming events etc.), a web server response (live news, Facebook news feed, Twitter timeline, weather forecast, stock quotes from the Internet etc.), or the result of an interactive action (push instant messages from the reception desks).

The file types can vary - formats like CSV and RSS / XML are common for structured data, plain text files can be used because of their simplicity, thus allowing non-technical people to easily create their own data feeds.

The HMP is able to integrate dynamic / static data feeds and present the desired information with a certain style on the screen. This can be resumed in three steps:

1. The HMP retrieves the raw data from the data source. Any text-based data format can be used, such as: RSS, plain-text, CSV, ICS, XML, JSON etc.
2. The HMP processes the raw data in order to retain only the meaningful data.
3. The HMP displays the final data on the screen using the predefined content layout.

Data source

The HMP can retrieve data from a variety of local or remote data sources, like:

• Social networking service

  Twitter, Facebook, Flickr, Instagram online platforms can be used as sources of user-generated content. Yammer as well, starting with Elementi 2018.

• RSS feed

  RSS (Rich Site Summary; often called Really Simple Syndication) feeds are commonly used to share frequently updated information: news headlines, blog entries, audio, video etc.) and can be easily found on the Internet. RSS feeds enable publishers to syndicate data automatically.

• Text file

  Because of their simplicity, plain text files are commonly used for storage of information, like service / product descriptions, details of a procedure / process, notifications, rules, warnings etc. There are many software tools that allows writing / modifying text files, like Notepad, WordPad, Microsoft Word, Google Docs, OpenOffice Writer, LibreOffice Writer etc.

• Spreadsheet

  Spreadsheet files are commonly used to store data in tabular form (i.e data cells arranged into rows and columns), like sport scores, employee birthdays, production results, price lists, poll results, restaurant menus, etc. There are many software tools that allows writing / modifying spreadsheet files like: Microsoft Excel, Google Sheets, OpenOffice Calc, LibreOffice Calc, Numbers (spreadsheet) etc.

• Calendar

  Calendars are commonly used to store one-time or repetitive events, like: meetings, room bookings, classes etc. Most calendar tools, like Google Calendar, Apple Calendar (formerly iCal), IBM Lotus Notes, Yahoo! Calendar etc., are able to export the data as iCalendar (.ics) file - see more details about how to get data from calendar applications.

• Media folders / locations

  A media folder is commonly used to store media files (usually images and / or videos) - could be a designated folder or a third-party provider such as: Microsoft OneDrive, Google Drive, Flickr, Instagram etc.

• Financial data

  Market data provided by Alpha Vantage and foreign exchange rates provided by the Alpha Vantage or European Central Bank.

• Weather forecast

  The weather information can be retrieved from Open Weather Map, Weather Undeground, World Weather Online, or Yahoo! Weather.

Other data formats, like JSON and XML, are also available for advanced users; in fact, any text-based format can be used as a data source, including web services and HTML, and be processed through custom JavaScript parsers.
The advanced data feed widgets can be used to display content extracted from data sources based on any type of text-based format.

Data-driven content

Data-driven widgets

The data-driven widgets included into Elementi widgets library have been designed to ease up the creation of data-driven content from common types of data feeds (e.g., RSS, Text, Spreadsheets, Calendars etc.). They can be used to automatically display data extracted from local or external data sources.

 

See the full article on data-driven widgets.

Secure data access

Security is a fundamental element to consider when giving access to your personal data and we've created our data-driven widgets with that in mind. To ensure secure access and permission control, without exposing any user credentials, Cockpit must authorize any request for third-party data. For that, you need to do the following:

1. Create a Cockpit account, if you don't have one.
2. Register your Elementi license into your Cockpit account.
3. Add your players into your Cockpit account.
4. Connect your third-party data provider account to your Cockpit account.
   • You are being redirected to the provider’s own web site, where you authenticate yourself and then you confirm which permissions you give to the SpinetiX app.
   • Once you authorize the SpinetiX app to access your data on your behalf, you are redirected back to Cockpit.
5. All set - you can now use the data-driven widgets with your data in a secured manner.

Notes:

• Your credentials are never saved on our servers, nor on the players - in fact, your credentials are not even shared with Cockpit.
• At any time you can disconnect any third-party data provider account from the "Channels" page and thus prevent access to your data to all players. Or you can remove a particular player from your Cockpit account and that player will no longer have access to your private data.
• This is part of IT security best practices and is designed to protect privacy.

Third-party data providers are using OAuth 2.0 protocol for authorization. Once you authorize the SpinetiX app to access your data on your behalf, the provider service generates a master app token, that is stored by Cockpit and later used to generate short-lived access tokens for the players. This is a security feature which guarantees that if one of the access access tokens is compromised in any way, it can’t be exploited after 15 minutes.

Data processing

The information retrieved from the data source is parsed and arranged in a table-like format, with columns and rows, using the built-in data parsers. If needed, the resulted data set can be limited to a certain number of rows of data.

Elementi X users can also use advanced parsers (XML, JSON) and add further formatting rules on the columns returned by the main parser, so that only the meaningful rows of data can be displayed on the screen:

• Each data field (column) can be further parsed using the parsers listed above, plus Date/Time and Query String parsers. See Data parsers.
• Each data field (column) can be filtered using Numeric, Date and / or String filters. See Data filters.
• The resulted data set can be sorted by one of the data fields (columns).

Programmers can also use jSignage data feed functions or getURL() function along with low-level parsing functions.

Data mapping

The mapping between the data retrieved from the data source and the layers composing the layout of the widget is done using data placeholders - a special notation having, in its simplest form, a double-pair of square brackets around the name of the field from where the data is taken - for instance [[title]], [[description]] etc.

See the full article on Data placeholders.

Sample projects

• Data feed projects - a set of Elementi sample projects demonstrating the usage of data feeds from various sources.
• Display an RSS text ticker tutorial.
• Display a JSON data feed tutorial.

Data caching

The raw data retrieved from an external data source through data-driven widgets is cached locally (in localStorage) up to 24 hours, to protect from cases when the network is down or the player reboots.

• Any of the data-driven widgets included in Elementi can be configured to automatically refresh the cashed data after a certain amount of time.
• The raw data is cached per widget - the same data source URI can be used in different widgets, in case the data processing is done differently per widget.
• The player built-in HTTP caching mechanism for external resources is not employed by default because with data feeds it is important to get the latest version of the data, not some potentially outdated copy. If needed, that can also be activated - see this tutorial for more details.

Troubleshooting

URI encoding

• URIs must contain only valid URI characters (as defined by RFC 3986). Other characters must be escaped. For example, the URI "http://www.acme.corp/rss/monday|01041^^3F" should be correctly written as "http://www.acme.corp/rss/monday%7C01041%5E%5E3F".
• Note that Elementi data feed dialog does the URI encoding automatically.

Text source encoding

• Ensure that your text file is encoded using UTF-8. Otherwise the output may be garbled or missing characters.
• When using Notepad, the encoding can be set when the file is saved from the "Save as" window, "Encoding:" drop-down box (next to the "Save" button).

MIME types

• In Elementi, the MIME type of a file is determined by the extension. If the image ('enclosure') content of your feed contains a link to an ASX or PHP file, or another file which generates the image on the fly, then Elementi will not be able to auto-determine the MIME type. Open the layer properties and select the correct MIME type from the drop-down.

Data feed not displayed on Elementi

• Elementi can open data sources that require authentication, only if the correct credentials have been added within the "Network Credentials" dialog, which can be opened from "Menu" > "Settings". In addition, the HMP must have the appropriate credentials specified in HMP Control Center.

Data feed not displayed on HMP

When the data feed is displayed on Elementi, but not on HMP, you should do the following:

• Check the path used for the data source:
  • SMB/CIFS file sharing (i.e. UNC path like "\\server\share\file_path") is not supported by the HMP - this is working on Elementi only (though it is strongly discouraged).
  • Check that you are not using data from your local computer (e.g. "C:\file_path").
• Check that the HMP has full access to the Internet (for Proxy and Gateway settings, see Network Settings page).
• Also see the generic remarks when using Web servers.