Nuxeo and Kibana

Reporting with Kibana is superseded by the Data Visualization module from LTS 2015.

We have added support for Elasticsearch to the Nuxeo Platform since version LTS 2014 (6.0), bring it greater scalability. We use it to index all the documents in your instance. The results outside of the scalability enhancements is a much better response time when browsing Nuxeo. One of the other benefits of this integration is that we now have an easy to use reporting solution with Kibana.

Documentation will give you examples of how to setup and use it.

Installing Kibana 4

Kibana 4 requires to access directly the Elasticsearch database used by Nuxeo, and also requests to be protected (so, installed on an AWS instance for example, no hacker can access it).

Some thoughts:

• All this protocol is for an installation on a Linux box (tested and checked on AWS instances)
  • Never done it on Windows, nor on Mac
• If your are doing your demos on your localhost, maybe you can skip the creation of the "kibana" user, etc.)

Recommendations

When opening Kibana 4, the "No results found" page is probably displayed by default. This is because the analysis period is set to "Last 15 Minutes" by default (which is probably cool when analyzing the server metrics), so change it (top-right item of the page)

Also, Kibana evolves at light speed, so some of the screenshots you'll find here may already be obsolete. For example, it is likely that the search interval will be saved by Kibana in future small releases, because it is something that, practically speaking, annoys everybody.

Main Principles

There are basically three steps:

1. Create and save a Search
2. Create and save 1-n _Visualization_s (Pie Chart, Bars, ...) based on this search
3. Add the visualization(s) to a Dashboard (and save it)

Saving a search is useful when you want to display different vizualizations for the same search. For example, you could have:

• A search that finds all "Picture" documents
• And several visualizations based on this search: One using the lifecycle state, one using a field (dc:nature, ...), etc.

Still, you can create a new visualization with a search that will be used only by this visualization. In this chapter, we describe the use of a saved search.

Creating and Saving a Search

Make sure you have the index pattern you want to use (nuxeoand/oraudit): Go to "Settings" (top menu)

If you don't see the index you need, you must add it:

1. In the Indices sub tab, click the "+ Add New" button
2. In "Index name or pattern" enter an existing indice: Either nuxeo or audit
3. Once the name of the indice is entered, you must select a "Time-field name"
   • We generally select  dc:modified  for nuxeo
   • And eventDate for the audit

Go to "Discover" in the top menu, and click the "New Search" button

Then, click on the "Settings" button

 

In Index Pattern, select your index (nuxeo or audit). You can now create a search:

1. Enter the search statement
   1. WARNING: This is Lucene syntax, so basically:
      • : is the equals sign
      • Which means : in Nuxeo's fields path ( schema:field ) must be escaped
   2. Example: Search all documents of type "Picture":
      • ecm\:primaryType: "Picture"
   3. Run (to check you find something)
   4. Save the search

Creating and Saving 1-n Visualizations Based on This Search

Select Visualization in the top menu. In Create a new visualization, select the kind of visualization you want. In the next step ("Select a search source"), select "From a saved search" and select the search you saved previously.

Now, set up your visualization (See the example(s) below). Click "Save" and give a name to this visualization.

Adding the Visualization(s) to a Dashboard

Go to Dashboard. If you already had a dashboard, you can either:

• Click the "+" toolbar button (on the right)
• Or the "New Dashboard" button

In all cases, you can use the "+" button and add the visualization you saved

Example: Pie Chart On "Top 10 Downloads"

This example is using the "audit" index (not "nuxeo")

Creating and Saving a Search

Select Discover in the top menu, click on the New Search button. In the Index pattern, select Audit

1. In the search field, enter the following expression: extended.downloadReason:"download" AND NOT extended.[blobFilename:storyboard-0*](http://blobfilenamestoryboard-0%2A/)

   With current version of Nuxeo (7.3), the audit logs as "download" a lot of things that are actually not a real download. For example, when displaying a video, there is a download entry for each picture of the storyboard. On the other hand, displaying thumbnails is filtered (thumbnail event in the log, displaying the preview of a picture too, ...)

2. Save this search: "Audit: Downloads"

Creating and Saving 1-n Visualizations Based on This Search

1. Select "Visualization" in the top menu
2. In "Create a new visualization", select "Pie Chart"
3. In the next step ("Select a search source"), select "From a saved search" and select the search you saved previously ("Audit: Downloads")
4. Now, set up your visualization
   1. Click "Split Slices" (below "Select buckets type")
   2. In "Aggregation", select "Terms"
   3. Field: extended.blobFilename
   4. Order: Top
   5. Size: 10
5. Click "Apply" to check all is good
6. Click "Save" and give a name to this visualization: "Audit: Top 10 Downloads"

 

Here it is:

Adding the Visualization(s) to a Dashboard

Go to Dashboard, if you already had a dashboard, you can either

• Click the "+" toolbar button (on the right)
• Or the "New Dashboard" button

In all cases, you can use the "+" button and add the visualization you saved. Possibly, add other visualizations. Once it's done, save your Dashboard, so you can reuse it for a demo.

Tada!

Setting up Kibana 4 on Mac

This will explain how to run Kibana 4 with a Nuxeo running on localhost on your Mac.

Settings

1. Download Kibana 4
   • Just the final application (no need to build or do some command line scripts. Just download the binaries)
   • Go to https://www.elastic.co/downloads/kibana, find the Mac OS version, download it
   • At the time of this writing (2015-08-24), we downloaded kibana-4.1.1-darwin-x64.tar-2.gz
2. Extract kibana-4.1.1-darwin-x64.tar-2.gz
   • You now have the "kibana-4.1.1-darwin-x64" folder
   • You can leave the default values
   • Just in case if you are stressed you can check kibana-4.1.1-darwin-x64/config/kibana.yml
     • It must listen on port 5601, and server "0.0.0.0".
     • If you change that or if a future version of Kibana changes that, you'll adapt (see below)

3. Have a nuxeo-tomcat and change its nuxeo.conf file

   • {path/to/nuxeo}/bin/nuxeo.conf
   • Uncomment the line elasticsearch.httpEnabled=true

   • So basically, you now have:

     . . .
     elasticsearch.httpEnabled=true
     . . .
     

     Nuxeo FastTrack 7.4

     Since Nuxeo FastTrack 7.4, this option is enabled by default, just check it is there

4. Start nuxeo 

5. Start Kibana:

   • Command:

     cd /path/to/kibana-4.1.1-darwin-x64/bin
     ./kibana
     

   • The terminal must ends with a line telling you Kibana is listening ("Listening on 0.0.0.0:5601")

   • Test it's ok:
     • In another terminal, curl localhost:5601 must return something valid (not a connection error)

It's ok, you are all setup. When done with this, do not forget to quit Nuxeo and Kibana. 

---