Nuxeo Server

Upgrade from LTS 2017 to FT 10.3

Updated: December 11, 2018 Page Information Edit on GitHub

For the general upgrade process, see the page Upgrading the Nuxeo Platform.

This chapter highlights some major information about upgrade from Nuxeo Platform LTS 2017 (9.10) to Nuxeo Platform LTS 2019 (10.10). We strongly encourage you to also have a quick read of the upgrade notes.

If you had already upgraded to previous 10.x Fast Track versions, check out the page Upgrade from LTS 2017 following Fast Tracks for upgrade instructions from 10.1 to 10.2, 10.2 to 10.3 or 10.3 to LTS 2019.

Installation and Configuration

New Parameters

Parameter Modification Reference
nuxeo.server.hsts.enabled New Default Value to false NXP-24030
userResolverCreateIfNeeded Default value to true NXP-25062
userResolverUpdate Default value to true NXP-25062

Requirements

  • Elasticsearch >= 6.x is required.
  • Kafka >= 2.0.0 is required.

Data

Elasticsearch

An Elasticsearch RestClient Trust Store can now be configured.

The following nuxeo.conf properties can be set to define an appropriate TLS/SSL configuration for the Elasticsearch RestClient:

  • elasticsearch.restClient.truststore.path
  • elasticsearch.restClient.truststore.password
  • elasticsearch.restClient.truststore.type
  • elasticsearch.restClient.keystore.path
  • elasticsearch.restClient.keystore.password
  • elasticsearch.restClient.keystore.type

The following properties are deprecated (they were misnamed and are actually referring to the trustStore, not the keyStore):

  • elasticsearch.restClient.keystorePath
  • elasticsearch.restClient.keystorePassword
  • elasticsearch.restClient.keystoreType

If a more fine-grained configuration is needed than properties, the following extension point can be used instead:

  <require>org.nuxeo.elasticsearch.defaultConfig</require>
  <extension target="org.nuxeo.elasticsearch.ElasticSearchComponent" point="elasticSearchClient">
    <elasticSearchClient class="org.nuxeo.elasticsearch.client.ESRestClientFactory">
      ...
      <option name="trustStorePath">/path/to/cacerts.jks</option>
      <option name="trustStorePassword">changeit</option>
      <option name="trustStoreType">jks</option>
      <option name="keyStorePath">/path/to/keystore.jks</option>
      <option name="keyStorePassword">changeit</option>
      <option name="keyStoreType">jks</option>
    </elasticSearchClient>
  </extension>

See NXP-26074

MongoDB

A MongoDB Client TrustStore can be configured for in-flight encryption.

The following nuxeo.conf properties can be set to define appropriate TLS/SSL configuration for MongoDB:

  • nuxeo.mongodb.ssl=true
  • nuxeo.mongodb.truststore.path
  • nuxeo.mongodb.truststore.password
  • nuxeo.mongodb.truststore.type
  • nuxeo.mongodb.keystore.path
  • nuxeo.mongodb.keystore.password
  • nuxeo.mongodb.keystore.type

If a more fine-grained configuration is needed than properties, the following extension point can be used instead:

  <require>org.nuxeo.mongodb.connection</require>
  <extension target="org.nuxeo.runtime.mongodb.MongoDBComponent" point="connection">
    <connection id="default">
      ...
      <ssl>true</ssl>
      <trustStorePath>/path/to/cacerts.jks</trustStorePath>
      <trustStorePassword>changeit</trustStorePassword>
      <trustStoreType>jks</trustStoreType>
      <keyStorePath>/path/to/keystore.jks</keyStorePath>
      <keyStorePassword>changeit</keyStorePassword>
      <keyStoreType>jks</keyStoreType>
    </connection>
  </extension>

See NXP-26072

Redis

The following nuxeo.conf properties can be set to define appropriate TLS/SSL configuration for Redis:

  • nuxeo.redis.ssl=true
  • nuxeo.redis.truststore.path
  • nuxeo.redis.truststore.password
  • nuxeo.redis.truststore.type
  • nuxeo.redis.keystore.path
  • nuxeo.redis.keystore.password
  • nuxeo.redis.keystore.type

If a more fine-grained configuration is needed than properties, the following extension point can be used instead:

  <require>org.nuxeo.ecm.core.redis.config</require>
  <extension target="org.nuxeo.ecm.core.redis" point="configuration">
    <server>
      ...
      <ssl>true</ssl>
      <trustStorePath>/path/to/cacerts.jks</trustStorePath>
      <trustStorePassword>changeit</trustStorePassword>
      <trustStoreType>jks</trustStoreType>
      <keyStorePath>/path/to/keystore.jks</keyStorePath>
      <keyStorePassword>changeit</keyStorePassword>
      <keyStoreType>jks</keyStoreType>
    </server>
  </extension>

See NXP-26073

Code Changes

Replace json-lib with jackson

You should now use com.fasterxml.jackson.core:jackson-*, instead of net.sf.json-lib:json-lib.

See NXP-24093.

CSRF Protection

CSRF protection is activated by default and based on the CORS configuration, along with its allowOrigin and supportedMethods parameters, which by default doesn't allow any cross origin.

See NXP-24331.

Activation

To activate CSRF Token verification, use the following configuration:

  <extension target="org.nuxeo.runtime.ConfigurationService" point="configuration">
    <property name="nuxeo.csrf.token.enabled">true</property>
  </extension>

When this is activated, all clients accessing Nuxeo will need to get a token and provide it on all requests that are not GET/HEAD.

Note that even when the CSRF Token is not activated, other CSRF checks not using a token are still being done (using the Origin/Referer headers).

Getting the token initially

The client must use the following request with the header CSRF-Token: fetch:

GET /nuxeo
CSRF-Token: fetch

The response will contain the token in the header:

200 OK
CSRF-Token: uNTIwv3oEImb3singqJKSuJDNjM9ldVOjnwtxmFh

Passing the token

Then on every request that is not a GET/HEAD (so this applies to POST/PUT/DELETE/etc.) the client must provide the same token, either in the CSRF-Token request header or in the csrf-token request parameter:

POST /nuxeo/something
CSRF-Token: uNTIwv3oEImb3singqJKSuJDNjM9ldVOjnwtxmFh

or

POST /nuxeo/something?csrf-token=uNTIwv3oEImb3singqJKSuJDNjM9ldVOjnwtxmFh

Missing, expired or invalid token

If the token is missing, expired or invalid, the client will get a 403 Forbidden error, and a CSRF-Token: invalid header will be set:

403 Forbidden
CSRF-Token: invalid

Skipping certain endpoints

Some authentication endpoints need to be available with a POST without CSRF token checks. This can be done using for example:

  <extension target="org.nuxeo.runtime.ConfigurationService" point="configuration">
    <property name="nuxeo.csrf.token.skip" list="true">/login</property>
  </extension>

See NXP-25903

Video Conversion Listener

The video info (duration, format, etc.) is now computed by an asynchronous work to avoid loading the blob and running ffmpeg-info synchronously. This work, schedules two asynchronous works to process the video storyboard and conversions.

Class removed: VideoAutomaticConversionListener, VideoStoryboardListener and org.nuxeo.ecm.platform.video.VideoConstants#VIDEO_CHANGED_EVENT (videoChanged).

Class added: VideoInfoWork and VideoStoryboardWork.

See NXP-24316.

Automation Scripting

The import constraints have been relaxed, it's now possible to allow specific Java classes to be used via Automation Scripting, by default we add:

  • java.util.ArrayList
  • java.util.Arrays
  • java.util.Collections
  • java.util.UUID
  • org.nuxeo.runtime.transaction.TransactionHelper
  • org.nuxeo.ecm.core.api.Blobs
  • org.nuxeo.ecm.core.api.impl.blob.StringBlob
  • org.nuxeo.ecm.core.api.impl.blob.JSONBlob
  • org.nuxeo.ecm.core.api.NuxeoException

See NXP-25020 and NXP-25211.

Previous names New names
WebUI.AddErrorMessage Seam.AddErrorMessage
WebUI.AddInfoMessage Seam.AddInfoMessage
WebUI.AddMessage Seam.AddMessage
WebUI.ClearClipboard Seam.ClearClipboard
WebUI.ClearSelectedDocuments Seam.ClearSelectedDocuments
WebUI.ClearWorklist Seam.ClearWorklist
WebUI.ShowCreateForm Seam.CreateDocumentForm
WebUI.DownloadFile Seam.DownloadFile
WebUI.NavigateTo Seam.NavigateTo
WebUI.AddToClipboard Seam.AddToClipboard
WebUI.PushDocumentToSeamContext Seam.PushDocument
WebUI.AddToWorklist Seam.AddToWorklist
WebUI.RaiseSeamEvents Seam.RaiseEvents
WebUI.Refresh Seam.Refresh
WebUI.SetJSFOutcome Seam.SetOutcome
WebUI.DestroySeamContext Seam.DestroyContext
WebUI.InitSeamContext Seam.InitContext
WebUI.RunOperationInSeam Seam.RunOperation

See NXP-22232

Log4j 2

Nuxeo now uses Log4j 2 as its logging backend instead of Log4j. Two logging APIs are generally available in Nuxeo:

  • Commons Logging
  • Log4j 2 API

SLF4J is still available, but Log4j 2 API are preferred by reason of its lambda support.

Breaking change

  • Log4jHelper.setQuiet and Log4jHelper.setDebug was removed due to its usage of Log4j API which are now impossible with Log4j 2. These APIs were replaced by Log4jHelper.setLevel. Compared to previous implementation, setLevel API doesn't change appender's threshold, it just acts on logger's levels.
  • LogCaptureFeature.Filter has also changed without backward compatibility because it exposed a Log4j internal class. This has been replaced by corresponding Log4j 2 internal class.
  • File rollover has changed. Files are now compressed using gzip and pattern has been generalised to NAME-YYYY-MM-dd.log.gz.
    For instance: server.log.2018-01-01 becomes server-2018-01-01.log.gz
  • Watch log4j file for reload has been removed in favor of built-in logic in Log4j2. These properties are no longer read:
    • org.nuxeo.runtime.log4jwatch.disabled
    • org.nuxeo.runtime.log4jwatch.delay
      See Automatic Reconfiguration in Log4j 2 documentation to have watch file behavior.

Code behavior change

  • LogCaptureFeature behavior has changed. Now log Result is cleaned between each tests. You'll have an isolated result to assert for @Before/@After/test methods.
  • You may need to add the slf4j dependency if you were using it in your source code:
    <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    </dependency>
    

New

  • LogCaptureFeature.FilterOn now accepts a logger class instead of a logger name
  • You can now use Log4j 2 API in your code

See NXP-23863

Use NuxeoPrincipal Instead of Principal in Most APIs

The general upgrade rule is the Nuxeo code should not deal with a java.security.Principal anymore (except for internal authentication-related classes).

API changes

NuxeoPrincipal is now used instead of Principal in the methods and constructors for these classes:

  • CoreSession
  • DocumentEventContext
  • EventContext
  • EventContextImpl
  • InlineEventContext
  • OperationContext
  • QueryFilter
  • SQLQuery.Transformer
  • SecurityPolicy
  • SecurityPolicy.QueryTransformer
  • SecurityService
  • TrashService
  • UnboundEventContext
  • UserWorkspaceService
  • WebContext

Removed APIs

These APIs were previously deprecated or internal and have been removed.

  • CoreInstance.getInstance()
  • CoreInstance.getNumberOfSessions()
  • CoreInstance.getRegistrationInfos()
  • CoreInstance.openCoreSession(repositoryName, Map context)
  • CoreInstance.openCoreSession(repositoryName, Principal principal)
  • DetachedNuxeoPrincipal
  • SimplePrincipal (use UserPrincipal instead)

See NXP-25910

REST API Endpoints

The REST API now produces and consumes only application/json as content type:

  • application/json+nxentity is now never returned as content type response; the server does not expect it as content type request.
  • application/json+nxrequest content type should not be used anymore. It still works but it's deprecated server side. application/json should be used instead when POSTing to automation.
  • application/json+esentity which was never used was also removed.

Breaking Change:

  • org.nuxeo.ecm.automation.jaxrs.io.documents.JsonESDocumentWriter, located in nuxeo-automation-io has moved to org.nuxeo.elasticsearch.io.JsonESDocumentWriter, located in nuxeo-elasticsearch-core. The JsonESDocumentWriter does not implement anymore MessageBodyWriter<DocumentModel>, you may need to update your code.

See NXP-25036

Thumbnail resolution

In ThumbnailDocumentConverter, removed unnecessary constants:

THUMBNAIL_CONVERTER_NAME

THUMBNAIL_SIZE_PARAMETER_NAME // use ThumbnailConstants#THUMBNAIL_SIZE_PARAMETER_NAME instead

THUMBNAIL_DEFAULT_SIZE // use ThumbnailConstants#THUMBNAIL_DEFAULT_SIZE instead

See NXP-24717

Code Behavior Changes

  • ecm:versionVersionableId is now indexed by Elasticsearch.
    See NXP-24114.
  • Exception stack trace is no longer written in the response by default. Use parameter org.nuxeo.rest.stack.enable to enable it.
    See NXP-23861.
  • Some test classes have been moved under nuxeo-core-io test part. But nuxeo-core-io test module is a dependency of nuxeo-core-test (former location) to enforce backward compatibility.

    TransactionalFeature has been moved from nuxeo-core-test to nuxeo-runtime-test:
    org.nuxeo.ecm.core.test.TransactionalFeature becomes org.nuxeo.runtime.test.runner.TransactionalFeature

See NXP-25197.

  • New APIs are available on KeyValueStore.

    • Optimized storage of Long values:

      • put(String key, Long value)
      • put(String key, Long value, long ttl)
      • getLong(String key)
      • getLongs(Collection keys)
    • Atomic increment:

      • addAndGet(String key, long delta)

    See NXP-23745.

  • You can add the skipAggregates=true http header when invoking the search rest endpoint to skip any aggregation computation on a page provider query.

    See NXP-25158.

  • To activate OAuth 1, an <authenticationChain> must be defined to include <plugin>OAUTH1_AUTH</plugin>.
    See NXP-25975
  • Fulltext maximum size is now 128 KB by default. To change this, the repository configuration can be updated to use another fieldSizeLimit value:

    <fulltext ... fieldSizeLimit="1048576">
      ...
    </fulltext>
    

    A value of 0 means no limit.

    See the full text documentation for more information.

    See NXP-25716

Operation Changes

SuggestUserEntries operation

Since Nuxeo 10.2 and 9.10-HF03, the SuggestUserEntries operation performs a full name user search, e.g. typing "John Do" returns the user with first name "John" and last name "Doe".

See NXP-24583.

Previous names New names
WebUI.AddErrorMessage Seam.AddErrorMessage
WebUI.AddInfoMessage Seam.AddInfoMessage
WebUI.AddMessage Seam.AddMessage
WebUI.ClearClipboard Seam.ClearClipboard
WebUI.ClearSelectedDocuments Seam.ClearSelectedDocuments
WebUI.ClearWorklist Seam.ClearWorklist
WebUI.ShowCreateForm Seam.CreateDocumentForm
WebUI.DownloadFile Seam.DownloadFile
WebUI.NavigateTo Seam.NavigateTo
WebUI.AddToClipboard Seam.AddToClipboard
WebUI.PushDocumentToSeamContext Seam.PushDocument
WebUI.AddToWorklist Seam.AddToWorklist
WebUI.RaiseSeamEvents Seam.RaiseEvents
WebUI.Refresh Seam.Refresh
WebUI.SetJSFOutcome Seam.SetOutcome
WebUI.DestroySeamContext Seam.DestroyContext
WebUI.InitSeamContext Seam.InitContext
WebUI.RunOperationInSeam Seam.RunOperation

See NXP-22232

Relocate nuxeo-platform-collections in nuxeo-service

Since Nuxeo Platform 10.3, Collection Core types (Collection, Favorites, etc.), schema and facets have been merged in CoreExtensions.xml. As a result:

  • nuxeo-platform-collections-jsf has disappeared. Resources and classes have been dispatched in nuxeo-jsf/nuxeo-platform-webapp-base and nuxeo-dm/nuxeo-platform-webapp-types
  • All Collection-related automation operations have been moved to nuxeo-automation-features
  • Collection and Favorites services have been kept but moved under nuxeo-services/nuxeo-collections
  • UserWorkspaces service now contributes where should be located User's collections and Favorites by implementing CollectionLocationService.java
  • You can use CollectionFeature.java to test a feature relying on Collections.
    You need to add the following pom dependency:
    <dependency>
     <groupId>org.nuxeo.ecm.platform</groupId>
     <artifactId>nuxeo-platform-collections-core</artifactId>
     <type>test-jar</type>
     <scope>test</scope>
    </dependency>
    

See NXP-25543

Image Recompute

Added the Picture.RecomputeViews operation for administrators.

See NXP-25791

Deprecated APIs

@LocalDeploy

@LocalDeploy is now deprecated it must not be used anymore, use @Deploy instead that now allows you to deploy local contributions.

See NXP-22544.

CoreSession#close

CoreSession.close() is deprecated and should not be used anymore.

See NXP-24089.

TrashService

API Changes

The new TrashService fires dedicated events documentTrashed and documentUntrashed (hold by TrashService interface) instead of lifecycle_transition_event. The document model passed in the event won't be saved by Nuxeo at the end.

Keeping Old Trash implementation

The trash implementation has changed in 10.2. If you want to keep previous implementation relying on life cycle state, add the following contribution:

  <require>org.nuxeo.ecm.core.trash.service.migrator</require>
  <extension target="org.nuxeo.runtime.migration.MigrationService" point="configuration">

    <migration id="trash-storage">
      <defaultState>lifecycle</defaultState>
    </migration>

  </extension>

If you want to migrate trash state to the new property model (ecm:isTrashed), follow the Trash migration steps.

Trash Migration

To migrate trash states to the new property model:

  1. Follow the step from section Keeping old trash implementation.
  2. In the Nuxeo Platform's JSF UI, go to Admin > System Information > Migration, click the button Migrate trashed state from lifecycle to property and wait until migration is completed.
  3. Then perform an Elasticsearch re-indexation of all repository, in the Nuxeo Platform's JSF UI, go to Admin > Elasticsearch > Admin, click the button Re-index repository and wait until re-indexation is completed.
  4. Remove the contribution added at step 1.

Migration Note - 10.2
During migration, documents in state deleted will receive the isTrashed property set to true but migrator will leave document in deleted state.

Migration Note - 10.3
Migrator behavior has changed in 10.3, now documents in state deleted will receive the isTrashed property set to true and migrator will follow transition undelete if possible, if not it will set project state.

See NXP-24850.

org.nuxeo.isTrashed.from.deleteTransition introduced in 10.2 has been removed. Now, if you have migrated your Nuxeo instance to use the new dedicated isTrashed property, calls to CoreSession#followTransition with the delete/undelete transition are forwarded to TrashService without following the transition.

Also, these transitions are deprecated as we DO NOT follow them anymore. As a consequence, after migration or on a fresh instance, documents can't have deleted state anymore.

As a consequence of this backward mechanism, following these transitions on a proxy will remove them.

See NXP-25761

Addons

Nuxeo Drive - Server APIs Changes

Added

  • Added to FileManager:

    DocumentModel createDocumentFromBlob(CoreSession documentManager, Blob input, String path, boolean overwrite, String fullName, boolean noMimeTypeCheck, boolean excludeOneToMany) throws IOException;
    

    to allow excluding "one-to-many" importers (ie. the ones creating more than one document for the given blob, typically CSVZipImporter or ExportedZipImporter) when selecting the importer. This is used by Drive to not try to explode a CSV import ZIP for instance.

  • Added to FileImporter:

    boolean isOneToMany();
    

    to flag an importer as "one-to-many", ie. it creates more than one document for the given blob, typically CSVZipImporter or ExportedZipImporter.

See NXP-25797

Deprecated

  • The following operations have been deprecated:

    GetRepositoriesOperation
    NuxeoDriveAddToLocallyEditedCollection
    NuxeoDriveCanMove
    NuxeoDriveGenerateConflictedItemName
    NuxeoDriveGetClientUpdateInfo
    NuxeoDriveWaitForAsyncCompletion
    
  • The following operation parameters has been deprecated:

    NuxeoDriveGetChangeSummary#lastSyncActiveRootDefinitions
    

See NXP-24885

Remove Usage of Deprecated Code in Nuxeo-Drive-Server

Removed as deprecated since 7.10 or less:

FileSystemItemFactory#getDocumentByFileSystemId(String id, Principal principal)

FileSystemItemManager#getSession(String repositoryName, Principal principal)

NuxeoDriveManager#setChangeFinder(FileSystemChangeFinder changeFinder)

NuxeoDriveActions#UPDATE_SITE_URL_PROP_KEY
NuxeoDriveActions#SERVER_VERSION_PROP_KEY

NuxeoDriveCreateFile#name

NuxeoDriveGetChangeSummary#lastSyncDate

NuxeoDriveGetTopLevelChildren

Removed as should have been deprecated since 7.10 or less, see NXP-14826:

FileSystemChangeFinder#getFileSystemChangesIntegerBounds(CoreSession session, Set<IdRef> lastActiveRootRefs, SynchronizationRoots activeRoots, Set<String> collectionSyncRootMemberIds, long lowerBound, long upperBound, int limit)

FileSystemChangeFinder#getCurrentDate();

NuxeoDriveManager#getChangeSummaryIntegerBounds(Principal principal, Map<String, Set<IdRef>> lastSyncRootRefs, long lowerBound)

and changed

FileSystemChangeFinder#getFileSystemChanges(CoreSession session, Set<IdRef> lastActiveRootRefs, SynchronizationRoots activeRoots, long lastSuccessfulSyncDate, long syncDate, int limit)

NuxeoDriveManager#getChangeSummary(Principal principal, Map<String, Set<IdRef>> lastSyncRootRefs, long lastSuccessfulSync)

to

FileSystemChangeFinder#getFileSystemChanges(CoreSession session, Set<IdRef> lastActiveRootRefs, SynchronizationRoots activeRoots, Set<String> collectionSyncRootMemberIds, long lowerBound, long upperBound, int limit)

NuxeoDriveManager#getChangeSummary(Principal principal, Map<String, Set<IdRef>> lastSyncRootRefs, long lowerBound)

See NXP-25844

Nuxeo CSV

Dates in CSV files must be formatted using the W3C format. More information on the W3C documentation.

It is possible to use the old legacy date format (without time information) by setting the configuration property nuxeo.csv.import.legacyDateFormat to true:

<require>org.nuxeo.ecm.csv.core.properties</require>
<extension point="configuration" target="org.nuxeo.runtime.ConfigurationService">
  <property name="nuxeo.csv.import.legacyDateFormat">true</property>
</extension>

See NXP-25219

Nuxeo Diff

Deprecated ContentDiffHelper#getHtmlConversionBlackListedMimeTypes.

Added:

org.nuxeo.ecm.diff.content.MimeTypesDescriptor
org.nuxeo.ecm.diff.content.MimeTypeDescriptor
ContentDiffAdapterManager#getHtmlConversionBlacklistedMimeTypes()

Added the htmlConversionBlacklistedMimeTypes extension point to the org.nuxeo.ecm.diff.content.adapter.ContentDiffAdapterManagerComponent component:

  <extension-point
    name="htmlConversionBlacklistedMimeTypes">
    <documentation>
      @since 10.10

      Allows to contribute the list of blacklisted mime types for HTML
      conversion.

      By default, contributing a list merges its mime types
      with the existing ones.
      To remove a mime type, use enabled=false.
      <code>
        <extension
          target="org.nuxeo.ecm.diff.content.adapter.ContentDiffAdapterManagerComponent"
          point="htmlConversionBlacklistedMimeTypes">
          <mimeTypes>
            <mimeType>application/msword</mimeType>
            <mimeType>application/rtf</mimeType>
            <mimeType enabled="false">application/pdf</mimeType>
          </mimeTypes>
        </extension>
      </code>

      To override the whole list, use override="true".
      <code>
        <extension
          target="org.nuxeo.ecm.diff.content.adapter.ContentDiffAdapterManagerComponent"
          point="htmlConversionBlacklistedMimeTypes">
          <mimeTypes override="true">
            <mimeType>application/msword</mimeType>
          </mimeTypes>
        </extension>
      </code>
    </documentation>
    <object
      class="org.nuxeo.ecm.diff.content.MimeTypesDescriptor" />
  </extension-point>

See NXP-25208

Complementary Information

a day ago manonlumeau Upgrade Notes 10.3
History: Created by manonlumeau

We'd love to hear your thoughts!

All fields required