Package org.nuxeo.ecm.csv.core
Class CSVImporterWork
- java.lang.Object
-
- org.nuxeo.ecm.core.work.AbstractWork
-
- org.nuxeo.ecm.core.transientstore.work.TransientStoreWork
-
- org.nuxeo.ecm.csv.core.CSVImporterWork
-
- All Implemented Interfaces:
Serializable,Work
public class CSVImporterWork extends TransientStoreWork
Work task to import form a CSV file. Because the file is read from the local filesystem, this must be executed in a local queue. Since NXP-15252 the CSV reader manages "records", not "lines".- Since:
- 5.7
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface org.nuxeo.ecm.core.work.api.Work
Work.Progress, Work.State
-
-
Field Summary
-
Fields inherited from class org.nuxeo.ecm.core.transientstore.work.TransientStoreWork
entryKey, KEY_SUFFIX, STORE_NAME
-
Fields inherited from class org.nuxeo.ecm.core.work.AbstractWork
callerThread, completionTime, docId, docIds, FAILURE_EXCEPTION, FAILURE_MSG, GLOBAL_DLQ_COUNT_REGISTRY_NAME, id, isTree, loginContext, originatingUsername, progress, RANDOM, repositoryName, schedulePath, schedulingTime, session, startTime, state, status, suspended, suspending, traceContext, WORK_FAILED_EVENT, WORK_INSTANCE
-
-
Constructor Summary
Constructors Constructor Description CSVImporterWork(String id)CSVImporterWork(String repositoryName, String parentPath, String username, Blob csvBlob, CSVImporterOptions options)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected StringListbuildRecipientsList(String userEmail)voidcleanUp(boolean ok, Exception e)This method is called afterAbstractWork.work()is done in a finally block, whether work completed normally or was in error or was interrupted.protected Map<String,Serializable>computePropertiesMap(org.apache.commons.csv.CSVRecord record, CompositeType compositeType, Map<String,Integer> header)protected SerializableconvertValue(CompositeType compositeType, String fieldName, String headerValue, String stringValue, long lineNumber)protected BlobcreateBlobFromFilePath(String fileRelativePath)Creates aBlobfrom a relative file path.protected BlobcreateBlobFromStringBlob(Blob stringBlob)Creates aBlobfrom aStringBlob.protected booleancreateDocument(long lineNumber, String newParentPath, String name, String type, Map<String,Serializable> properties)protected voiddoImport(org.apache.commons.csv.CSVParser parser)protected BlobgetBlob()StringgetCategory()Gets the category for this work.protected StringgetDocumentUrl(DocumentModel doc)List<CSVImportLog>getImportLogs()protected static List<CSVImportLog>getLastImportLogs(String id)protected longgetLineNumber(org.apache.commons.csv.CSVParser parser)protected longgetLineNumber(org.apache.commons.csv.CSVRecord record)protected static CSVImportStatusgetStatus(String id)StringgetTitle()Gets a human-readable name for this work instance.protected StringgetUserUrl()protected booleanimportRecord(org.apache.commons.csv.CSVRecord record, Map<String,Integer> header)Import a line from the CSV file.protected Stringlaunch()protected static StringloadTemplate(String key)protected voidlogError(long lineNumber, String message, String localizedMessage, String... params)protected BufferedReadernewReader(Blob blob)protected voidreplaceBlobs(List<Object> list)Recursively replaces allBlobs withBlobs created from Files stored in the folder registered by thenuxeo.csv.blobs.folderproperty.protected voidreplaceBlobs(Map<String,Object> map)Recursively replaces allBlobs withBlobs created from Files stored in the folder registered by thenuxeo.csv.blobs.folderproperty.protected voidsendMail()static ThrowableunwrapException(Throwable t)protected booleanupdateDocument(long lineNumber, DocumentRef docRef, Map<String,Serializable> properties)voidwork()This method should implement the actual work done by theWorkinstance.-
Methods inherited from class org.nuxeo.ecm.core.transientstore.work.TransientStoreWork
computeEntryKey, computeEntryKey, containsBlobHolder, getBlobHolder, getEntryKey, getStore, isIdempotent, putBlobHolder, putBlobHolder, removeBlobHolder
-
Methods inherited from class org.nuxeo.ecm.core.work.AbstractWork
appendWorkToDeadLetterQueue, buildWorkFailureEventProps, closeSession, commitOrRollbackTransaction, equals, getCompletionTime, getDocument, getDocuments, getId, getOriginatingUsername, getPartitionKey, getProgress, getRetryCount, getSchedulePath, getSchedulingTime, getSpanFromContext, getStartTime, getStatus, getWorkInstanceState, hashCode, initSession, initSession, isDocumentTree, isSuspending, isWorkInstanceSuspended, newDocumentLocation, openSystemSession, openUserSession, run, runWorkWithTransaction, setCompletionTime, setDocument, setDocument, setDocuments, setOriginatingUsername, setProgress, setSchedulePath, setStartTime, setStatus, setWorkInstanceState, setWorkInstanceSuspending, startTransaction, suspended, toString, workFailed
-
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface org.nuxeo.ecm.core.work.api.Work
isCoalescing, isGroupJoin, onGroupJoinCompletion
-
-
-
-
Field Detail
-
NUXEO_CSV_MAIL_TO
public static final String NUXEO_CSV_MAIL_TO
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD
public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_FIELD
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE
public static final String LABEL_CSV_IMPORTER_CANNOT_CONVERT_FIELD_VALUE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_EXISTING_FILE
public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_FILE
- See Also:
- Constant Field Values
-
NUXEO_CSV_BLOBS_FOLDER
public static final String NUXEO_CSV_BLOBS_FOLDER
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS
public static final String LABEL_CSV_IMPORTER_DOCUMENT_ALREADY_EXISTS
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE
public static final String LABEL_CSV_IMPORTER_UNABLE_TO_UPDATE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_DOCUMENT_UPDATED
public static final String LABEL_CSV_IMPORTER_DOCUMENT_UPDATED
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_UNABLE_TO_CREATE
public static final String LABEL_CSV_IMPORTER_UNABLE_TO_CREATE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST
public static final String LABEL_CSV_IMPORTER_PARENT_DOES_NOT_EXIST
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_DOCUMENT_CREATED
public static final String LABEL_CSV_IMPORTER_DOCUMENT_CREATED
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE
public static final String LABEL_CSV_IMPORTER_NOT_ALLOWED_SUB_TYPE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_UNABLE_TO_SAVE
public static final String LABEL_CSV_IMPORTER_UNABLE_TO_SAVE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE
public static final String LABEL_CSV_IMPORTER_ERROR_IMPORTING_LINE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE
public static final String LABEL_CSV_IMPORTER_NOT_EXISTING_TYPE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE
public static final String LABEL_CSV_IMPORTER_MISSING_TYPE_VALUE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_MISSING_NAME_VALUE
public static final String LABEL_CSV_IMPORTER_MISSING_NAME_VALUE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN
public static final String LABEL_CSV_IMPORTER_MISSING_NAME_COLUMN
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_EMPTY_FILE
public static final String LABEL_CSV_IMPORTER_EMPTY_FILE
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT
public static final String LABEL_CSV_IMPORTER_ERROR_DURING_IMPORT
- See Also:
- Constant Field Values
-
LABEL_CSV_IMPORTER_EMPTY_LINE
public static final String LABEL_CSV_IMPORTER_EMPTY_LINE
- See Also:
- Constant Field Values
-
TEMPLATE_IMPORT_RESULT
public static final String TEMPLATE_IMPORT_RESULT
- See Also:
- Constant Field Values
-
CATEGORY_CSV_IMPORTER
public static final String CATEGORY_CSV_IMPORTER
- See Also:
- Constant Field Values
-
CONTENT_FILED_TYPE_NAME
public static final String CONTENT_FILED_TYPE_NAME
- See Also:
- Constant Field Values
-
AUTHORIZED_HEADERS
protected static final List<String> AUTHORIZED_HEADERS
CSV headers that won't be checked if the field exists on the document type.- Since:
- 7.3
-
parentPath
protected String parentPath
-
username
protected String username
-
options
protected CSVImporterOptions options
-
hasTypeColumn
protected boolean hasTypeColumn
-
startDate
protected Date startDate
-
importLogs
protected ArrayList<CSVImportLog> importLogs
-
computeTotal
protected boolean computeTotal
-
total
protected long total
-
docsProcessedCount
protected long docsProcessedCount
- Since:
- 11.1
-
docsCreatedCount
protected long docsCreatedCount
-
EMPTY_LOGS
protected static final Serializable EMPTY_LOGS
-
-
Method Detail
-
getCategory
public String getCategory()
Description copied from interface:WorkGets the category for this work.Used to choose an execution queue.
- Specified by:
getCategoryin interfaceWork- Overrides:
getCategoryin classAbstractWork- Returns:
- the category, or
nullfor the default
-
getTitle
public String getTitle()
Description copied from interface:WorkGets a human-readable name for this work instance.- Returns:
- a human-readable name
-
getImportLogs
public List<CSVImportLog> getImportLogs()
-
work
public void work()
Description copied from interface:WorkThis method should implement the actual work done by theWorkinstance.It should periodically update its progress through
Work.setProgress(org.nuxeo.ecm.core.work.api.Work.Progress).To allow for suspension by the
WorkManager, it should periodically callWork.isSuspending(), and iftruecallWork.suspended()return early with saved state data.Clean up can by implemented by
Work.cleanUp(boolean, Exception).- Specified by:
workin interfaceWork- Specified by:
workin classAbstractWork- See Also:
Work.isSuspending(),Work.suspended(),Work.cleanUp(boolean, java.lang.Exception)
-
cleanUp
public void cleanUp(boolean ok, Exception e)Description copied from class:AbstractWorkThis method is called afterAbstractWork.work()is done in a finally block, whether work completed normally or was in error or was interrupted.- Specified by:
cleanUpin interfaceWork- Overrides:
cleanUpin classAbstractWork- Parameters:
ok-trueif the work completed normallye- the exception, if available
-
launch
protected String launch()
-
getStatus
protected static CSVImportStatus getStatus(String id)
-
getLastImportLogs
protected static List<CSVImportLog> getLastImportLogs(String id)
-
newReader
protected BufferedReader newReader(Blob blob) throws IOException
- Throws:
IOException- Since:
- 7.3
-
doImport
protected void doImport(org.apache.commons.csv.CSVParser parser)
-
importRecord
protected boolean importRecord(org.apache.commons.csv.CSVRecord record, Map<String,Integer> header)Import a line from the CSV file.- Returns:
trueif a document has been created or updated,falseotherwise.- Since:
- 6.0
-
getLineNumber
protected long getLineNumber(org.apache.commons.csv.CSVRecord record)
-
getLineNumber
protected long getLineNumber(org.apache.commons.csv.CSVParser parser)
-
computePropertiesMap
protected Map<String,Serializable> computePropertiesMap(org.apache.commons.csv.CSVRecord record, CompositeType compositeType, Map<String,Integer> header)
- Since:
- 6.0
-
convertValue
protected Serializable convertValue(CompositeType compositeType, String fieldName, String headerValue, String stringValue, long lineNumber)
-
createBlobFromFilePath
protected Blob createBlobFromFilePath(String fileRelativePath) throws IOException
Creates aBlobfrom a relative file path. The File will be looked up in the folder registered by thenuxeo.csv.blobs.folderproperty.- Throws:
IOException- Since:
- 9.3
-
createBlobFromStringBlob
protected Blob createBlobFromStringBlob(Blob stringBlob) throws IOException
Creates aBlobfrom aStringBlob. Assume that theStringBlobcontent is the relative file path. The File will be looked up in the folder registered by thenuxeo.csv.blobs.folderproperty.- Throws:
IOException- Since:
- 9.3
-
replaceBlobs
protected void replaceBlobs(Map<String,Object> map) throws IOException
Recursively replaces allBlobs withBlobs created from Files stored in the folder registered by thenuxeo.csv.blobs.folderproperty.- Throws:
IOException- Since:
- 9.3
-
replaceBlobs
protected void replaceBlobs(List<Object> list) throws IOException
Recursively replaces allBlobs withBlobs created from Files stored in the folder registered by thenuxeo.csv.blobs.folderproperty.- Throws:
IOException- Since:
- 9.3
-
createDocument
protected boolean createDocument(long lineNumber, String newParentPath, String name, String type, Map<String,Serializable> properties)
-
updateDocument
protected boolean updateDocument(long lineNumber, DocumentRef docRef, Map<String,Serializable> properties)
-
logError
protected void logError(long lineNumber, String message, String localizedMessage, String... params)
-
sendMail
protected void sendMail()
-
getBlob
protected Blob getBlob()
- Since:
- 9.1
-
getDocumentUrl
protected String getDocumentUrl(DocumentModel doc)
-
getUserUrl
protected String getUserUrl()
-
buildRecipientsList
protected StringList buildRecipientsList(String userEmail)
-
-