Nuxeo Server

Data Visualization

Updated: December 12, 2017 Page Information Edit on GitHub

Data Visualization support in Nuxeo leverages Web Components and the Polymer framework in order to allow users to easily build their own custom dashboards, thus helping them understand how data is distributed in the repository as well as monitoring workflow activity.

A set of custom elements are available for bootstrapping custom graphical dashboards. These elements are built using our Nuxeo Elements, in particular they rely on nuxeo-connection to configure the Nuxeo instance to use.

Technical Overview

Elasticsearch Passthrough

Since Nuxeo 7.4, the platform provides an Elasticsearch Passthrough that allows using the HTTP REST API provided by the Elasticsearch back end. This API provides a comprehensive and powerful search DSL based on JSON to define queries which we can leverage to execute advanced aggregated searches.

These aggregations provide analytic information over a set of documents and allow us to build complex search requests which can provide computed metrics over this set of documents through filtering and "bucketing".

For more information about Elasticsearch's Search DSL you can take a look at its online  documentation.

Data Visualization Elements

Non visual elements are the standard way to expose remote services while sticking to Polymer's “everything is an element” metaphor so we created a set of custom elements to query data from Elasticsearch. The goal is to hide complexity from the user by providing a simple, concise declarative way to build complex queries which cover most use cases.

Like what has been done for our core Nuxeo Elements focus was on a clear separation between data and the graphical elements to allow users to use existing UI elements for building their own custom dashboards.

You will find more information about the Nuxeo Dataviz Elements on this page.

Workflow Data

The Workflow Data element (nuxeo-workflow-data) allows querying Nuxeo's Workflow Audit Log which, since Nuxeo 7.3, is stored by default as an Elasticsearch index.

It allows users to retrieve data from this index by defining a set of attributes in this custom element which provide a simple and fluent syntax to build complex aggregate searches.

Under the hood each query is built as a nested aggregation with a fixed number of aggregation types and levels. The final query structure is fully configurable by the set of optional attributes available in our element:

  • grouped-by: Top level term aggregations (comma-separated values)

  • with-*: Nested multi-bucket aggregation (with-ranges | with-date-intervals)

  • metrics: leaf single-value metrics aggregation (max, min, avg, sum)

For example, by using our nuxeo-workflow-data element with the following attributes:

<nuxeo-workflow-data workflow="worflowModelName"
                     grouped-by="action"
                     with-ranges='{
                      "duration":[
                         {"key":"quick", "to":"1500"},
                         {"key":"slow", "from":"1500"}
                       ]}'
                     metrics="sum(amount)"
                     data="{{data}}">
</nuxeo-workflow-data>

Our element would produce a request like this:

{
  query: { bool: { must: [...] },
  aggs: {
    by: { // aggregated-on
      terms: {
        field: "extended.action"
      },
      aggs: {
        by: {  // with-ranges | with-date-intervals
          range : {
            field : "extended.duration",
            ranges: [...]
          },
          aggs: {
            metric: { // metric
              sum: {
                field : "extended.workflowVariables.amount"
              }
            }
          }
        }
      }
    }
  }
}

By building a custom element to help users query the Workflow Audit Log, we not only provide a simpler more fluent declarative syntax, but we can also decouple from Elasticsearch's specificities in most use cases.

Data returned from Elasticsearch is also processed for simplification and consists of either a single scalar value, for non aggregate queries, or a nested list of {key: key, value: value} entries where value can itself be a similar list or single scalar value.

Note: Before using this data for producing tables or charts it usually must undergo another transformation process which is specific to the charting and table elements used. For sample transformation helpers we recommend looking at existing dashboards which showcase integration with third-party libraries.

Sample Dashboards

Review Workflows Dashboards

GitHub projecthttps://github.com/nuxeo/nuxeo-review-workflows-dashboards

Serial Review Dashboardhttps://github.com/nuxeo/nuxeo-review-workflows-dashboards/blob/master/src/main/elements/nx-workflow-dashboard/nx-serial-workflow-dashboard.html

Parallel Review Dashboard: https://github.com/nuxeo/nuxeo-review-workflows-dashboards/blob/master/src/main/elements/nx-workflow-dashboard/nx-parallel-workflow-dashboard.html  

Travel Expenses Sample Dashboard

GitHub project: https://github.com/nuxeo/nuxeo-travel-expenses

Dashboardhttps://github.com/nuxeo/nuxeo-travel-expenses/blob/master/src/main/yo/app/elements/nx-workflow-dashboard/nx-workflow-dashboard.html

6 hours ago Manon Lumeau add tags for doc days
a month ago manonlumeau Added content-review-lts2017 label
2 months ago manonlumeau NXDOC-1346-FT review screenshot
2 years ago Bertrand Chauvin 21 | ixed link
2 years ago Bertrand Chauvin 20 | Add link to video course
2 years ago Bertrand Chauvin 19 | Add link to video course
2 years ago Bertrand Chauvin 18 | Added video
2 years ago Bertrand Chauvin 17 | Added videos
2 years ago Nelson Silva 16 | Update prefix of Nuxeo Elements
2 years ago Solen Guitter 14
2 years ago Solen Guitter 15
2 years ago Solen Guitter 13
2 years ago Solen Guitter 12 | Title capitalization, step formatting
2 years ago Manon Lumeau 11
2 years ago Manon Lumeau 10 | Formatting
2 years ago Nelson Silva 9
2 years ago Nelson Silva 8
2 years ago Nelson Silva 7
2 years ago Nelson Silva 6
2 years ago Nelson Silva 5
2 years ago Nelson Silva 4
2 years ago Nelson Silva 3
2 years ago Nelson Silva 2
2 years ago Nelson Silva 1
History: Created by Nelson Silva