The system replication feature aims to clone entire collection of data existing in a Nuxeo system.
Consequently the clone is importable in another system leading to a complete replication of the system.
Such feature is obviously an important gain because it allows:
- complete backup of system
- complete data migration
- replication of complex systems
The feature is in fact an export – import tool. Three projects are designed to accomplish the objectives:
- a common module,
- an export module and
- an import module.
The export module is actually ported on 5.1 also, allowing migration from older storages to current supported Nuxeo deployment.
In this current stage, only the documentary base is replicated. The relations, vocabularies, users replications are left for later implementation. That because:
- relations are stored usually in Jena storage: this can be moved as it is. As long as replication preserves the document ID the relations can be moved independently.
- vocabularies are also stored in directory structures which can be moved independently.
- users and groups are usually stored outside the Nuxeo repository (LDAP) which supposedly remain the same. If it is not the case, the Customer needs to previously ensure the same user / groups structure is in place.
Another important constraint is that the import on H2 database could crash.
The resume feature is not yet implemented.
The selection of a part of repository to be exported, respectively imported is not yet implemented.
More can be found at http://doc.nuxeo.org/5.2/books/nuxeo-book/html-single/#admin-replication
The services (export on 5.1+, import on 5.2+) are made available either though JSF UI or through a MBean visible in JMX console. The UI is available only if the web component is also deployed.
The feature code is currently located in addon project nuxeo-platform-replication (as also nuxeo-platform-importer feature, which is used inside replication). In order to have it deployed the following jars needs to be deployed:
- Export (5.1.6+)
- nuxeo-platform-replication-common-api-$version.jar
- nuxeo-platform-replication-common-core-$version.jar
- nuxeo-platform-replication-exporter-api-$version.jar
- nuxeo-platform-replication-exporter-core-$version.jar
- nuxeo-platform-replication-exporter-web-$version.jar
- nuxeo-platform-replication-exporter-mbean-5.1.6.sar (only for 5.1.6)
with applicable versions: 5.1.6, 5.3.0, 5.3.2, 5.4.2
- Import (5.2+)
- nuxeo-platform-importer-core-$version.jar
- nuxeo-platform-replication-common-api-$version.jar
- nuxeo-platform-replication-common-core-$version.jar
- nuxeo-platform-replication-importer-api-$version.jar
- nuxeo-platform-replication-importer-core-$version.jar
- nuxeo-platform-replication-importer-web-$version.jar
with applicable versions: 5.2.1-SNAPSHOT, 5.3.0, 5.3.2, 5.4.2
Export instructions – JSF UI
The link “Export” is present in the top list of actions. The page allows selecting the destination of the archive. The exported artifacts are stored here, so enough space must be ensured.
Once the export launched (pressing once on the link “Export”) the page displays the progress, updating the number of documents exported in top of the page. At the end of export, status “Done” is displayed.
Please be patient and don't press twice the link.
Also, the status and information about the export can be seen in the server log (see bellow).
Export instructions – JMX console MBean
Both releases 5.1.6 and 5.2 benefits the existence of MBeans. The beans interface can be found in JMX console, under the “nx” section, as service “Exporter” on 5.1.6, respectively “ExporterService” on 5.2. The method “export” requires two parameters: the repository name (usually “default”) and the path to save the archive (the third boolean one is not effective).
For the results, watch the server logs (see bellow).
Import instructions – JSF UI
The link “Import” is present in the top list of actions. The page allows selecting the source of the archive.
Once the import launched (pressing once on the link “Import”) the page displays the progress, updating the number of documents imported in top of the page. At the end of import, status “Done” is displayed.
Please logout and login in order to refresh the view.
Please be patient and don't press twice the link.
Also, the status and information about the import can be seen in the server log.
Import instructions – JMX console MBean
Release 5.2 benefits the existence of MBeans. The beans interface can be found in JMX console, under the “nx” section, as service “ImporterService”. The method “importDocuments” requires one parameter: the path to save the archive.
For the results, watch the server logs.
The export or import results are best viewed in the logging system, as configured in JBoss. Attention: the level of logging for replication tool need to be at least INFO to see the summary. The summary contains
- the number of documents attempted to be exported
- the type of errors encountered, each one with
- a short description
- and the list of documents in fault.
An example of how this summary looks:or if no error recorded the message
is displayed.
Exporting
The following errors are possible:
- unknown system error, with the message
A number of documents exported ended in error. It can't be told if they are available for import or not. Logs needs to be consulted for more details.
- document structure is corrupted, with the message
A number of documents have the XML structure corrupted and can't be recovered. These documents can't be exported / imported.
- missing children / versions, with the message
respectively
These documents were searched for children / versions, but due to errors reading, the lists couldn't be retrieved. Obviously, the children / versions (is any) are not registered for export and not attempted to be exported. Well, at least not from this try, but maybe as version published, or some other situations.
- a particular version is missing, with the message
Usually, for various documents (proxy or usual documents) a version is registered but can't be found. The export treats the document as no version attached (the import will act accordingly).
- head of a version is missing, with the message
The versions are exported, but no live document is found. Well, the versioned documents will be available (as published versions for instance) but with no living head.
- blobs are missing, with the message
The documents are exported with a fake blob file instead the expected one. The fake blob is a text file containing "The original blob could not be found, this is a fake replacement one." Attention: the mime type and file name are not modified!
Importing
The following errors are possible:
- unknown system error, with the message
A number of documents imported ended in error. It can't be told if they are available for import or not. Logs needs to be consulted for more details.
- document structure is corrupted, with the message
A number of documents have the XML structure corrupted and can't be recovered. These documents were imported, but with no schemes or data inside.
- repository import of a document failed, with the message
The atomic operation of importing the document in repository failed (because wrong configuration of repository / available types; because the exported document structure is corrupted or obsolete, etc). The import can't be performed.
- applying the custom schema change failed, with the message
The custom defined schema change was applied but it threw exception. The changes were discarded and the initial document structure is imported instead.
- document type denied to be imported, with the message
The documents were not imported as they are denied by type choice to be imported.
- the ACL couldn't be applied on a document, with the message
The documents were imported with no local ACL changes.
1.The export facility was tested and proved working fine on 5.1.6 and 5.2 servers with various backend storages (JCR, H2, PostgreSQL). The import was tested on 5.2 only with PostgreSQL backend. It is known that H2 doesn't support the import. The error thrown is timeout trying to lock a table. Nor JCR supports the import, as the node ID can't be created voluntarily in JCR system.
2.When exporting proxies, the targeted documents are also exported, increasing redundantly the size of archive: http://jira.nuxeo.org/browse/NXP-3825.
3.The multi-threaded import didn't work correctly: http://jira.nuxeo.org/browse/NXP-3826 (fixed since 5.3.2), the feature was disabled in UI.
4.Sometimes the UI could crash after or during the import, because the documents are changed massively in repository. Just login again.
5.The visual reports are sometime broken. The data recorded in log are the one to be considered as real.