This documentation applies to target versions of the Nuxeo Platform up to 5.5. It also applies to later versions if the JBPM addon (nuxeo-platform-jbpm) is installed.
By default, the workflow is not enabled on your custom document types (this is done by default from Studio version 2.4).
You need to declare two distinct extensions from the Advanced Settings > XML Extension menu.
Copy the content of the extensions below and replace "CUSTOM DOCTYPE ID" by your document type ID.
The first extension lists the workflows you want to be available for your custom document types.
The second extension says that the default workflow is available for your custom document type. Note that since Nuxeo version 5.4.3, this contribution is no more needed:
This will be useful if you want to use the default workflows. Default workflows enable you to control the lifecycle state through a serie or a set of human validation tasks configured by the the workflow launcher himself. If you want to implement a workflow that is more "controlled", not built by the end user, you can leverage the "CreateTask" Operation, such as it is done in this simple tutorial. Tasks created with the CreateTask Operation appear on the Summary tab.