Package org.nuxeo.ecm.csv.core
Class CSVImporterWork
- java.lang.Object
-
- org.nuxeo.ecm.core.work.AbstractWork
-
- org.nuxeo.ecm.core.transientstore.work.TransientStoreWork
-
- org.nuxeo.ecm.csv.core.CSVImporterWork
-
- All Implemented Interfaces:
Serializable
,Work
public class CSVImporterWork extends TransientStoreWork
Work task to import form a CSV file. Because the file is read from the local filesystem, this must be executed in a local queue. Since NXP-15252 the CSV reader manages "records", not "lines".- Since:
- 5.7
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.nuxeo.ecm.core.work.api.Work
Work.Progress, Work.State
-
-
Field Summary
-
Fields inherited from class org.nuxeo.ecm.core.transientstore.work.TransientStoreWork
entryKey, KEY_SUFFIX, STORE_NAME
-
Fields inherited from class org.nuxeo.ecm.core.work.AbstractWork
callerThread, completionTime, docId, docIds, FAILURE_EXCEPTION, FAILURE_MSG, GLOBAL_DLQ_COUNT_REGISTRY_NAME, id, isTree, loginContext, originatingUsername, progress, RANDOM, repositoryName, schedulePath, schedulingTime, session, startTime, state, status, suspended, suspending, traceContext, WORK_FAILED_EVENT, WORK_INSTANCE
-
-
Constructor Summary
Constructors Constructor Description CSVImporterWork(String id)
CSVImporterWork(String repositoryName, String parentPath, String username, Blob csvBlob, CSVImporterOptions options)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected StringList
buildRecipientsList(String userEmail)
void
cleanUp(boolean ok, Exception e)
This method is called afterAbstractWork.work()
is done in a finally block, whether work completed normally or was in error or was interrupted.protected Map<String,Serializable>
computePropertiesMap(org.apache.commons.csv.CSVRecord record, CompositeType compositeType, Map<String,Integer> header)
protected Serializable
convertValue(CompositeType compositeType, String fieldName, String headerValue, String stringValue, long lineNumber)
protected Blob
createBlobFromFilePath(String fileRelativePath)
Creates aBlob
from a relative file path.protected Blob
createBlobFromStringBlob(Blob stringBlob)
Creates aBlob
from aStringBlob
.protected boolean
createDocument(long lineNumber, String newParentPath, String name, String type, Map<String,Serializable> properties)
protected void
doImport(org.apache.commons.csv.CSVParser parser)
protected Blob
getBlob()
String
getCategory()
Gets the category for this work.protected String
getDocumentUrl(DocumentModel doc)
List<CSVImportLog>
getImportLogs()
protected static List<CSVImportLog>
getLastImportLogs(String id)
protected long
getLineNumber(org.apache.commons.csv.CSVParser parser)
protected long
getLineNumber(org.apache.commons.csv.CSVRecord record)
protected static CSVImportStatus
getStatus(String id)
String
getTitle()
Gets a human-readable name for this work instance.protected String
getUserUrl()
protected boolean
importRecord(org.apache.commons.csv.CSVRecord record, Map<String,Integer> header)
Import a line from the CSV file.protected String
launch()
protected static String
loadTemplate(String key)
protected void
logError(long lineNumber, String message, String localizedMessage, String... params)
protected BufferedReader
newReader(Blob blob)
protected void
replaceBlobs(List<Object> list)
Recursively replaces allBlob
s withBlob
s created from Files stored in the folder registered by thenuxeo.csv.blobs.folder
property.protected void
replaceBlobs(Map<String,Object> map)
Recursively replaces allBlob
s withBlob
s created from Files stored in the folder registered by thenuxeo.csv.blobs.folder
property.protected void
sendMail()
static Throwable
unwrapException(Throwable t)
protected boolean
updateDocument(long lineNumber, DocumentRef docRef, Map<String,Serializable> properties)
void
work()
This method should implement the actual work done by theWork
instance.-
Methods inherited from class org.nuxeo.ecm.core.transientstore.work.TransientStoreWork
computeEntryKey, computeEntryKey, containsBlobHolder, getBlobHolder, getEntryKey, getStore, isIdempotent, putBlobHolder, putBlobHolder, removeBlobHolder
-
Methods inherited from class org.nuxeo.ecm.core.work.AbstractWork
appendWorkToDeadLetterQueue, buildWorkFailureEventProps, closeSession, commitOrRollbackTransaction, equals, getCompletionTime, getDocument, getDocuments, getId, getOriginatingUsername, getPartitionKey, getProgress, getRetryCount, getSchedulePath, getSchedulingTime, getSpanFromContext, getStartTime, getStatus, getWorkInstanceState, hashCode, initSession, initSession, isDocumentTree, isSuspending, isWorkInstanceSuspended, newDocumentLocation, openSystemSession, openUserSession, run, runWorkWithTransaction, setCompletionTime, setDocument, setDocument, setDocuments, setOriginatingUsername, setProgress, setSchedulePath, setStartTime, setStatus, setWorkInstanceState, setWorkInstanceSuspending, startTransaction, suspended, toString, workFailed
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.nuxeo.ecm.core.work.api.Work
isCoalescing, isGroupJoin, onGroupJoinCompletion
-
-
-
-
Field Detail
-
NUXEO_CSV_MAIL_TO
public static final String NUXEO_CSV_MAIL_TO
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD
public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE
public static final String LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_EXISTING_FILE
public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_FILE
- See Also:
- Constant Field Values
-
NUXEO_CSV_BLOBS_FOLDER
public static final String NUXEO_CSV_BLOBS_FOLDER
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS
public static final String LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE
public static final String LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_DOCUMENT_UPDATED
public static final String LABEL_CSV_IMPORTER_DOCUMENT_UPDATED
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_UNABLE_TO_CREATE
public static final String LABEL_CSV_IMPORTER_UNABLE_TO_CREATE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST
public static final String LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_DOCUMENT_CREATED
public static final String LABEL_CSV_IMPORTER_DOCUMENT_CREATED
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE
public static final String LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_UNABLE_TO_SAVE
public static final String LABEL_CSV_IMPORTER_UNABLE_TO_SAVE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE
public static final String LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE
public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE
public static final String LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_MISSING_NAME_VALUE
public static final String LABEL_CSV_IMPORTER_MISSING_NAME_VALUE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN
public static final String LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_EMPTY_FILE
public static final String LABEL_CSV_IMPORTER_EMPTY_FILE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT
public static final String LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_EMPTY_LINE
public static final String LABEL_CSV_IMPORTER_EMPTY_LINE
- See Also:
- Constant Field Values
-
TEMPLATE_IMPORT_RESULT
public static final String TEMPLATE_IMPORT_RESULT
- See Also:
- Constant Field Values
-
CATEGORY_CSV_IMPORTER
public static final String CATEGORY_CSV_IMPORTER
- See Also:
- Constant Field Values
-
CONTENT_FILED_TYPE_NAME
public static final String CONTENT_FILED_TYPE_NAME
- See Also:
- Constant Field Values
-
AUTHORIZED_HEADERS
protected static final List<String> AUTHORIZED_HEADERS
CSV headers that won't be checked if the field exists on the document type.- Since:
- 7.3
-
parentPath
protected String parentPath
-
username
protected String username
-
options
protected CSVImporterOptions options
-
hasTypeColumn
protected boolean hasTypeColumn
-
startDate
protected Date startDate
-
importLogs
protected ArrayList<CSVImportLog> importLogs
-
computeTotal
protected boolean computeTotal
-
total
protected long total
-
docsProcessedCount
protected long docsProcessedCount
- Since:
- 11.1
-
docsCreatedCount
protected long docsCreatedCount
-
EMPTY_LOGS
protected static final Serializable EMPTY_LOGS
-
-
Method Detail
-
getCategory
public String getCategory()
Description copied from interface:Work
Gets the category for this work.Used to choose an execution queue.
- Specified by:
getCategory
in interfaceWork
- Overrides:
getCategory
in classAbstractWork
- Returns:
- the category, or
null
for the default
-
getTitle
public String getTitle()
Description copied from interface:Work
Gets a human-readable name for this work instance.- Returns:
- a human-readable name
-
getImportLogs
public List<CSVImportLog> getImportLogs()
-
work
public void work()
Description copied from interface:Work
This method should implement the actual work done by theWork
instance.It should periodically update its progress through
Work.setProgress(org.nuxeo.ecm.core.work.api.Work.Progress)
.To allow for suspension by the
WorkManager
, it should periodically callWork.isSuspending()
, and iftrue
callWork.suspended()
return early with saved state data.Clean up can by implemented by
Work.cleanUp(boolean, Exception)
.- Specified by:
work
in interfaceWork
- Specified by:
work
in classAbstractWork
- See Also:
Work.isSuspending()
,Work.suspended()
,Work.cleanUp(boolean, java.lang.Exception)
-
cleanUp
public void cleanUp(boolean ok, Exception e)
Description copied from class:AbstractWork
This method is called afterAbstractWork.work()
is done in a finally block, whether work completed normally or was in error or was interrupted.- Specified by:
cleanUp
in interfaceWork
- Overrides:
cleanUp
in classAbstractWork
- Parameters:
ok
-true
if the work completed normallye
- the exception, if available
-
launch
protected String launch()
-
getStatus
protected static CSVImportStatus getStatus(String id)
-
getLastImportLogs
protected static List<CSVImportLog> getLastImportLogs(String id)
-
newReader
protected BufferedReader newReader(Blob blob) throws IOException
- Throws:
IOException
- Since:
- 7.3
-
doImport
protected void doImport(org.apache.commons.csv.CSVParser parser)
-
importRecord
protected boolean importRecord(org.apache.commons.csv.CSVRecord record, Map<String,Integer> header)
Import a line from the CSV file.- Returns:
true
if a document has been created or updated,false
otherwise.- Since:
- 6.0
-
getLineNumber
protected long getLineNumber(org.apache.commons.csv.CSVRecord record)
-
getLineNumber
protected long getLineNumber(org.apache.commons.csv.CSVParser parser)
-
computePropertiesMap
protected Map<String,Serializable> computePropertiesMap(org.apache.commons.csv.CSVRecord record, CompositeType compositeType, Map<String,Integer> header)
- Since:
- 6.0
-
convertValue
protected Serializable convertValue(CompositeType compositeType, String fieldName, String headerValue, String stringValue, long lineNumber)
-
createBlobFromFilePath
protected Blob createBlobFromFilePath(String fileRelativePath) throws IOException
Creates aBlob
from a relative file path. The File will be looked up in the folder registered by thenuxeo.csv.blobs.folder
property.- Throws:
IOException
- Since:
- 9.3
-
createBlobFromStringBlob
protected Blob createBlobFromStringBlob(Blob stringBlob) throws IOException
Creates aBlob
from aStringBlob
. Assume that theStringBlob
content is the relative file path. The File will be looked up in the folder registered by thenuxeo.csv.blobs.folder
property.- Throws:
IOException
- Since:
- 9.3
-
replaceBlobs
protected void replaceBlobs(Map<String,Object> map) throws IOException
Recursively replaces allBlob
s withBlob
s created from Files stored in the folder registered by thenuxeo.csv.blobs.folder
property.- Throws:
IOException
- Since:
- 9.3
-
replaceBlobs
protected void replaceBlobs(List<Object> list) throws IOException
Recursively replaces allBlob
s withBlob
s created from Files stored in the folder registered by thenuxeo.csv.blobs.folder
property.- Throws:
IOException
- Since:
- 9.3
-
createDocument
protected boolean createDocument(long lineNumber, String newParentPath, String name, String type, Map<String,Serializable> properties)
-
updateDocument
protected boolean updateDocument(long lineNumber, DocumentRef docRef, Map<String,Serializable> properties)
-
logError
protected void logError(long lineNumber, String message, String localizedMessage, String... params)
-
sendMail
protected void sendMail()
-
getBlob
protected Blob getBlob()
- Since:
- 9.1
-
getDocumentUrl
protected String getDocumentUrl(DocumentModel doc)
-
getUserUrl
protected String getUserUrl()
-
buildRecipientsList
protected StringList buildRecipientsList(String userEmail)
-
-