Data feed widgets in Elementi 2015

From SpinetiX Support Wiki

(Redirected from Data feed widgets)
Jump to: navigation, search
"Info"
This page is about the advanced data-driven widgets included in Elementi 2015 X.
For other Elementi versions, see Data feed widgets in Elementi 2016 and Feed widgets in Elementi 3.x pages.

Introduction

The advanced data-feed widgets can be used to display content extracted from data sources based on any type of text-based format - the most useful being JSON & XML data format, which are usually used by web services - although some might need to be processed through JavaScript parsers (like HTML). See more about Data feeds in general.

Common data formats like: RSS, Text, Spreadsheets, and Calendars, are also supported, though it's recommended to use the dedicated widgets instead - see data-driven widgets page for more details.


Note Note: The advanced data-feed widgets detailed below are available only for Elementi 2015 X users.

Widgets

Under the "Data feeds" folder of Elementi 2015 X Library, you can find seven ready-to-use widgets that automatically display data extracted from the provided data source:

Data feed widgets
  • Text Bar
    Displays text content using a fixed number of lines and applying a bottom-to-top & line-by-line scrolling effect whenever the entire text doesn't fit into the widget area.
  • Text Ticker
    Displays text content using a continuous horizontal scrolling effect. The font size is adjusted automatically for the text to fit into the widget area.
  • Text Roll
    Displays text content using a continuous vertical scrolling effect. The font size is adjusted automatically for the selected number of lines to fit into the predefined area.
  • Media Playlist
    Displays a playlist of media items (images, videos) included into the data feed.
  • Media Crawler
    Displays the media items (images, videos) included into the data feed using a continuous scrolling effect.
  • Slideshow
    Displays slides composed of text & media layers, which are automatically filled out with data retrieved from the data feed. The slides are displayed one after another, with or without a transition effect in between, similarly to a playlist.
  • Table
    Displays slides composed of multiple cells of text & media layers, which are automatically filled out with data retrieved from the data feed. The slides are displayed one after another, with or without a transition effect in between, similarly to a playlist.

Configuration

Each of the widgets above can be customized using the options provided under "Properties" tab - for more details go to the dedicated page of each widget type, linked above.

To edit the properties related to the data source, click on the Data Feed Properties button. This opens the Data Feed Properties dialog, which offers advanced options for more complex data processing.


Note Note: At any point, the data source configuration can be tested by clicking on the "Test..." button.

Parsers

The parser(s) used by these widgets can be changed from the "Data Feed Properties" dialog; the following parsers are available: RSS, XML, JSON (default), CSV, RegExp, ICS, Script, Directory Listing and Query String. Additionally, a Date/Time parser is available for data formatting.

JSON parser

JSON (JavaScript Object Notation) is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs.

A JSON source can be parsed into a table-like format, with columns and rows, using the JSON parser. Once selected, a "JSON Parser" tab is added within the "Data Feed Properties" dialog, offering the following parameters:

"JSON Parser" tab of "Data Feed Properties" dialog in Elementi 2015 X
  • Path
    Enter a path (as standard JavaScript syntax) into the object to identify the location of the element containing the useful rows of data. For instance, you can use:
    • main.rows to select the rows array inside the main object,
    • main.1.rows to select the rows array inside the second element of the main object.
  • Specify output
    Enable this to create a custom mapping between the JSON elements selected under "Path" and the output columns, by specifying:
    • Name -> enter the name of the new column; (this name will be used as data placeholder, i.e. [[column_name]])
    • Path -> enter the path (as above) within the selected element.
"Technical note"
For more details about the JSON parser, see $.parseJSON() function.

Customization example

See also this tutorial about how to display a JSON data feed.

Let's say that we would like to retrieve the transport data for Lausanne provided by Swiss public transport API, which looks like this:

{
  "station": { ... },
  "stationboard": [
    {
      "stop": {
        "station": { ... }, "departure": "2015-10-16T19:11:00+0200", ...
        "platform": "6", "prognosis": { ... }, "location": { ... }
      },
      "name": "RE 3235",  ...
      "operator": "SBB", "to": "Vevey",
      "passList": [ ... ]
    },  ... 
  ]
}
Custom JSON parser configuration

For that, follow these steps:

  1. Open the "Data Feed Properties" dialog.
  2. Set the URI to http://transport.opendata.ch/v1/stationboard?station=Lausanne&limit=10.
  3. Set the "Parser" option to "JSON".
  4. Click on the "JSON Parser" tab.
  5. Set the "Path" option to "stationboard".
  6. Enable "Specify output" option and, within the table, enter the following lines:
    1. Name: "departure" & Path: "stop.departure".
    2. Name: "name" & Attribute: "name".
    3. Name: "operator" & Attribute: "operator".
    4. Name: "to" & Attribute: "to".


Additionally, a string filter could be set on the "operator" column, so that only the trains are retrieved (i.e. operator equals SBB).

XML parser

Extensible Markup Language (XML) is a markup language that defines a set of rules for encoding documents in a format which is both human-readable and machine-readable. It is a textual data format with strong support via Unicode for different human languages. Although the design of XML focuses on documents, it is widely used for the representation of arbitrary data structures such as those used in web services.

An XML source can be parsed into a table-like format, with columns and rows, using the XML parser. Once selected, an "XML Parser" tab is added within the "Data Feed Properties" dialog, offering the following parameters:

"XML Parser" tab of "Data Feed Properties" dialog in Elementi 2015 X
  • Selector
    Enter a non-empty CSS3 selector to identify the data rows. For instance, you can use:
    • item to select all the elements of type "item" (i.e. <item>) from here;
    • row > title to select elements of type "title" and child of a "row" element;
  • Keep XML markup in columns
    Enable this to preserve the XML markup within the CDATA content inside the retrieved data.
  • Specify output
    Enable this to create a custom mapping between the XML elements selected under "Selector" and the output columns, by specifying:
    • Name -> enter the name of the new column, which can then be used as data placeholder (e.g., [[column_name]]);
    • Selector -> enter the selector (same syntax as above) that retrieves the child element.
    • Attribute -> enter the attribute that gives the output data value. It can be empty.


"Technical note"
For more details about the XML parser, see $.parseXML() and parseXML() functions.

Customization example

Let's say that we would like to retrieve the currency data from the XML daily exchange rate provided by European Central Bank, which looks like this:

<gesmes:Envelope ... >
	...
	<Cube>
		<Cube time='2015-10-14'>
			<Cube currency='USD' rate='1.1410'/>
			<Cube currency='JPY' rate='136.48'/>
			...
		</Cube>
	</Cube>
</gesmes:Envelope>
Custom XML parser configuration

For that, follow these steps:

  1. Open the "Data Feed Properties" dialog.
  2. Set the URI to http://www.ecb.int/stats/eurofxref/eurofxref-daily.xml.
  3. Set the "Parser" option to "XML".
  4. Click on the "XML Parser" tab.
  5. Set the "Selector" option to "Cube > Cube > Cube".
  6. Enable "Specify output" option and, within the table, enter the following two lines:
    1. Name: "currency" & Attribute: "currency".
    2. Name: "rate" & Attribute: "rate".


The custom XML parser configuration assures that the third level Cube element (the one containing the currency data) is selected and that two columns are created containing the currency and the rate values retrieved from the child element attributes.

Script parser

The Script parser can be used whenever any other parser cannot be applied over the input data and / or additional transformations need to be performed.

Once selected, a "Script Parser" tab is added within the "Data Feed Properties" dialog, offering the following parameters:

"Script Parser" tab of "Data Feed Properties" dialog in Elementi 2015 X
  • External file
    Select this option if the code of the parsing function is present within an external file.
    • URI
      Enter the name of the JavaScript file containing the custom parser. If the JS file is not in the same place as the widget, include the path as well.
      Default value: "custom.js".
    • Function
      Specify the name of the function, implemented in the JS file specified above, which needs to be used to process the input data.
      Default value: "parse".
  • Inline function
    Enable this option and enter the parsing function code directly in the dialog.

Parsing function

The parsing function is called differently depending on how the Script parser is used:

  • when used as main parser, the parsing function is executed once and the raw data is passed as parameter.
  • when used as formatting parser, the parsing function is executed once for each row of data from the resulted dataset.


The value returned by the parsing function can be one of the following types:

  • String
    The string value (e.g., 'foo') is assigned to the title column when used as the main parser or it will replace the value of input column value when formatting an existing column.
  • Plain Object
    When returning an object (e.g., { col1: 'foo', col2: 'bar', col3: 5, ... }), each value (e.g., 'foo', 'bar' etc.) is assigned to the column matching the key (e.g., col1, col2 etc.). If these columns don't exists in the result set, they are created.
    Returning an empty object (i.e. { }) means that there's no change to perform.
  • Array
    When returning an array (e.g., [ v1, v2, ... ]), each value of the array generates a new row of data. The values included into the array can be of type String (e.g., [ 'foo', 'bar' ]) or Plain Object (e.g., [ { col1: 'foo', col2: 'bar', col3: 5 } , { col1: 'baz', col2: 'qux', col3: 15 } ]).


Example 1 - check the input data to see if this is a fruit or a vegetable, and create a new column called type within the result set.

function parse( data ) {
  if ( data=="Bananas" || data=="Oranges" )
    return { type: "Fruits"};
  if ( data=="Carrots" || data=="Potatoes" )
    return { type: "Vegetable"};
  return { type: "I cannot eat this" };
}

Note Note: This kind of structure can be used to replace the switch template used in HMD. See also how to create a switch widget tutorial.

Example 2 - split the input data lines (i.e. the new-line character) into multiple rows, each having 3 columns.

function parse( data ) {
  var result = [];
  var lines = data.split( '\n' );
  for (var i=0; i<lines.length; i+=3 )
    result.push( { l1: lines[i], l2: lines[i+1], l3: lines[i+2]	} );
  return result;
}

Example 3 - convert the input data from an UNIX / JavaScript timestamp into a date.

function dateFromUnixTimestamp( data ) {
    return new Date( data * 1000 );
}

function dateFromJSTimestamp( data ) {
    return new Date( parseFloat( data ) );
}

Note Note: Other custom parsing functions returning a date can be found within dateParser.js file.

Query String parser

"Query String Parser" tab of "Data Feed Properties" dialog in Elementi 2015 X

See also Query String source type.

A query string type of data (e.g., field1=value1&field2=value2&field3=value3&...) can be parsed into a table-like format, with columns and rows, using the Query String parser. Although such parsing can be done with the RegExp parser, the Query String parser also performs a URL decoding in the process.

Once "Query String" is selected as main parser or formatting parser, a "Query String Parser" tab is added within the "Data Feed Properties" dialog, offering the following parameters:

  • Support tabular data
    Enable this when the query string contains indexed keys (e.g., key[0]=v0&key[1]=v1&...) that should generate rows under the "key" column, rather than one row with multiple columns (e.g., key[0], key[1]).
  • Specify output
    This option is not taken into account (for now).

Date/Time parser

The Date/Time parser is available only under the Format option, and can be used to parse the input data and format it as a date. Once selected for a particular entry column, a "Date/Time Parser" tab is added within the "Data Feed Properties" dialog, offering the possibility to select which date format to use to parse the entry column, from the following:

"Date/Time Parser" tab of "Data Feed Properties" dialog in Elementi 2015 X
  • Unix
    Select this when the input contains the number of seconds since Jan 1st, 1970 UTC, which is the standard Unix time representation.
  • Javascript
    Select this when the input contains the number of milliseconds since Jan 1st, 1970 UTC, as returned by the JavaScript Date.prototype.getTime() function.
  • RFC822
    Select this when the input contains a human readable date format, as described by the RFC-822 protocol (e.g., "Mon, 25 Dec 1995 13:30:00 GMT").
  • ISO-8601
    Select this when the input contains an ISO 8601 date (e.g., "2011-10-10T14:48:00").
  • Custom
    Select this when the input contains a custom date described with Unicode CLDR date-time patterns (e.g., "dd/MM/yyyy HH:mm", "MM-dd-yy h:mm a" etc.).

Filters

To allow filtering the resulted data based on some criteria, the following filters are available under the "Format" option within the "Data Feed Properties" dialog: Num. Filter, Date Filter, and String Filter.

Num. Filter

Added in Elementi 3.1.0.

"Num. Filter" tab of "Data Feed Properties" dialog in Elementi 2015 X

The Numerical Filter can be used to filter the resulted data based on numeric values.

Once selected under the "Format" option, a "Num. Filter on 'entry'" tab is added within the "Data Feed Properties" dialog, offering the possibility to filter the entry column using the following criteria:

  • Equals
    Enter the number against which the data in the entry column is compared - if the result is true, then the respective data row is kept in the data set, otherwise it is removed.
  • Not Equals
    Enter the number against which the data in the entry column is compared - if the result is false, then the respective data row is kept, otherwise it is removed.
  • In Range
    Enter the minimum and / or maximum numerical values (inclusive or not), against which the data in the entry column is compared - if the result is true, then the respective data row is kept in the data set, otherwise it is removed.

Date Filter

Added in Elementi 3.1.0.

The Date Filter can be used to filter the resulted data based on date values.

Once selected under the "Format" option, a "Date Filter on 'entry'" tab is added within the "Data Feed Properties" dialog, offering the possibility to filter the entry column using the following criteria:

"Date Filter" tab of "Data Feed Properties" dialog in Elementi 2015 X
  • From
    Specify the starting date and time used for filtering - if the date in the entry column is after, then the respective data row is kept in the data set, otherwise it is removed. The following options are possible:
    • Date
      Specify an absolute date / time for filtering the data.
    • Relative
      Specify a date / time relative to the moment when the widget is opened. Possible values includes "Now", "Today", "Tomorrow", "Next Month", "Last Year" etc.
    • Custom
      Specify a starting date using a custom date string.
  • To
    Specify the ending date and time used for filtering - if the date in the entry column is before, then the respective data row is kept in the data set, otherwise it is removed. The following options are possible:
    • Date, Relative, Custom
      Same as above.
    • Duration
      Specify a duration relative to the starting time specified under "From".
    • Indefinite
      Select this in order to use only the starting date in the filter.

String Filter

Added in Elementi 3.1.0.

"String Filter" tab of "Data Feed Properties" dialog in Elementi 2015 X

The Date Filter can be used to filter the resulted data based on string values.

Once selected under the "Format" option, a "String Filter on 'entry'" tab is added within the "Data Feed Properties" dialog, offering the possibility to filter the entry column using the following criteria:

  • Equals
    Enter the string against which the data in the entry column is compared - if the result is true, then the respective data row is kept in the data set, otherwise it is removed.
  • Not Equals
    Enter the string against which the data in the entry column is compared - if the result is false, then the respective data row is kept, otherwise it is removed.
  • Matches
    Enter a regular expression (RegExp) to be applied over the data in the entry column - if there's a match, then the respective data row is kept, otherwise it is removed.
This page was last modified on 6 September 2016, at 15:48.