Migration Troubleshooting

This section regroups the problems you can encounter when you migrate a P6 Core instance to a different version.

Warning

This guide is not complete, so contact the R&D team if something goes wrong during the migration.

Version `6.0.0-alpha-2`¶

Cluster Removed from DSL

In order to simplify the use of the transaction DSL, the notion of cluster was removed. The value of cluster parameters passed to lower level layers is now hardcoded to B2BOX. Please remove the first parameter when calling transaction.buildPK().

Version `6.0.0-alpha-3`¶

One key pair per application publisher is needed to sign an application package, therefore the format of the application profile file (.app) has changed. Old ones can no longer be used, ask the R&D team to generate new ones for you. In the future, you will be able to create your .app files in the P6 Management Console.

Also, the format of the application package has changed and (it’s compressed). Therefore old applications can no longer be imported and need to be re-packaged.

Version `6.0.0-alpha-4`¶

Old Database Tables Removed

In the database, the following tables were deleted:

universe
role
menu
routingrule
transformer

Please update your stored procedures accordingly.

Libraries Upgraded

This should not have an impact unless you rely on specific features in your scripts:

Apache POI for manipulating Microsoft Office documents was upgraded to version 4.0.1.
Apache Camel was upgraded to version 2.23.1.

App Profile File Simplified

Application profile files were simplified. You need to re-generate you .app files.

B2GUID Removed

If you use B2GUID in your scripts, please use java.util.UUID instead.

B2RequiredEndpointScope Removed

The B2RequiredEndpointScope annotation was removed, it’s been deprecated for a long time and you can safely delete it from your scripts.

message DSL Binding Removed

Please use transaction in your scripts instead.

_DEFAULT_ Variable Replaced

When working with smart documents, the _DEFAULT_ pipeline variable was renamed to text.

Version `6.0.0-alpha-5`¶

Before the migration¶

1/ We integrate in this version some declarative table partitioning. That feature is only available in PostgreSQL 11.

The database version needs to be upgraded from 9.6.1 to 11.3. The upgrade procedure is explained here https://www.postgresql.org/docs/11/upgrading.html

If you use Platform 6 Install, follow these steps and the migration will be done for you:

Pull the latest version of the project in your instance folder.
While your instance is still running on the old database, run the script pgsql_export.sh. Your dumps are stored in a folder called database_dumps.
Stop your instance by running stop_platform6.sh and edit your .env file, set PGSQL_VERSION=11.3.
Import you data by running pgsql_import.sh, that’s it! Then you can delete the database dump files if you wish.

2/ The database schema was renamed from b2head to p6core. Renaming is handled dynamically by our migration scripts (including the migration of all stored procedures). But you still need to apply manually the following statements BEFORE starting the migration:

DROP TABLE b2head.flyway_schema_history;
ALTER SCHEMA b2head RENAME TO p6core;

3/ You need to ensure that the cluster column of the Item table only contains the following values:

.work
.crossreferencing
B2BOX

You can fetch these values by running the following query:

SELECT DISTINCT cluster FROM p6core.item;

Warning

All other values will be deleted from item table

If you still have entries with the values .configuration or PROVISIONING, you can safely delete them because configuration is managed in table serviceconfig in P6 and old UI users are no longer used, by running this query:

DELETE FROM p6core.item WHERE cluster in ('.configuration', 'PROVISIONING');

4/ The number of key columns on the Item table has been decreased. The columns iid5 and iid6 were removed from this table. If you use these columns, please concatenate old values BEFORE ATTEMPTING THE MIGRATION and store them using columns from iid1 to iid4 and rework your indexes accordingly.

5/ The views configuration are stored in the serviceconfig table. The only allowed values for the <DataCluster> xml node are:

.work
B2BOX

To list all values, launch this query:

SELECT xpath('//DataCluster/text()', xmlparse(document (content ->> 'xmlView')))
FROM p6core.serviceconfig
WHERE id3 = 'views';

If you come across any other value, make sure to replace it with the value ‘.work’ as shown below:

Migration example:

UPDATE p6core.serviceconfig
SET content = replace(content::varchar, '<DataCluster>DummyValue</DataCluster>', '<DataCluster>.work</DataCluster>')::jsonb
WHERE id3='views';

To migrate from a DummyValue to the .work cluster.

6/ Indexes

Please take a look at the indexes defined on the item table and remove all the ones you think are no longer used. The automatic migration process will try to upgrade the remaining indexes. Deleting indexes frees disk space and most importantly speeds up migration and reduces chances of failure.

Migration duration: Configuration: 2 CPU / 6 GiB memory

300k entries on item table takes ~25min 850k entries on item table takes ~1h 15min 3,5M entries on item table takes ~3h

Durations may vary depending on the environment and the configuration.

6/ Configuration update

In the application configuration file application.conf, service.workflow.datamodel property has been renamed to service.workflowsteps.datamodel and the default value has changed from MessageInfo to TransactionInfo.

7/ Instance Base Context and Swagger(OpenApi)

The Instance record created via the OAuth Settings portal menu requires an update.

The Base context of a p6 instance is now: http//hostname:8080/p6 (not http://hostname:8080/b2box/rest/resource)

Swagger support has been upgraded to V3 which is now a standard know as OpenAPI. The V1 Swagger user interface is no longer embedded in the product. The latest Swagger user interface can be downloaded from https://swagger.io/tools/swagger-ui/

The Swagger UI should use the endpoint: http://hostname:8080/p6/openapi

Note

At this stage, you can finally start your instance in version 6.0.0-alpha-5. Change the version in the Docker Compose file and start your instance.

DSL¶

fileFromUri DSL Variable Removed

Please use fileFromUrl from the uri DSL

DSL access

To access to the DSL you need to prefix them by p6., for example:

log.debug p6.counter.list()

Transaction DSL

This version makes TransactionInfo the default model for transactions rather than MessageInfo. Also, in order to ease P6 learning curve, the term ‘project’ was replaced with ‘save’, therefore the following functions were renamed:

putMI -> putTI
projectMI -> saveTI which validates against a TransactionInfo and not a MessageInfo
project -> save
p6projectAndRouteMI -> saveAndRouteTI which validates against a TransactionInfo and not a MessageInfo
p6projectAndRoute -> saveAndRoute
p6route -> route which is not to be confused with the old function route which used routing rules!

In P6 version 6.0.0 all support for the old routing rules was removed, hence the following functions disappeared:

projectAndRouteMI
projectAndRoute
route

DataModel¶

The autoloaded data model WFWorkItem has been renamed to WorkflowTask. Make sure to update your stored procedures, XSLT resources, scripts…

Integrations¶

Each Platform 6 instance should now be defined and configured as an Integration using the new Administration panel.

See: Adding Your Instance ‘Integration’

Routes¶

Web3j

The component is no longer called P6Web3j, because we now rely on the standard Web3j Camel component. Therefore, all routes with endpoint URI p6web3j://... should be renamed to web3j://....

The producer query parameter transactionIndex was renamed to indexto comply with the official Web3j Camel component.

Observable operations were renamed to align with Web3j version 4.3:

ETH_LOG_OBSERVABLE -> ETH_LOG_FLOWABLE
ETH_JSON_LOG_OBSERVABLE -> ETH_JSON_LOG_FLOWABLE
ETH_BLOCK_HASH_OBSERVABLE -> ETH_BLOCK_HASH_FLOWABLE
ETH_PENDING_TRANSACTION_HASH_OBSERVABLE -> ETH_PENDING_TRANSACTION_HASH_FLOWABLE
TRANSACTION_OBSERVABLE -> TRANSACTION_FLOWABLE
PENDING_TRANSACTION_OBSERVABLE -> PENDING_TRANSACTION_FLOWABLE
BLOCK_OBSERVABLE -> BLOCK_FLOWABLE
REPLAY_BLOCKS_OBSERVABLE -> REPLAY_PAST_BLOCKS_FLOWABLE
REPLAY_TRANSACTIONS_OBSERVABLE -> REPLAY_PAST_TRANSACTIONS_FLOWABLE
CATCH_UP_TO_LATEST_BLOCK_OBSERVABLE -> REPLAY_ALL_PAST_BLOCKS_FLOWABLE
CATCH_UP_TO_LATEST_TRANSACTION_OBSERVABLE -> REPLAY_ALL_PAST_TRANSACTIONS_FLOWABLE
CATCH_UP_TO_LATEST_AND_SUBSCRIBE_TO_NEW_BLOCKS_OBSERVABLE -> REPLAY_PAST_AND_FUTURE_BLOCKS_FLOWABLE
CATCH_UP_TO_LATEST_AND_SUBSCRIBE_TO_NEW_TRANSACTIONS_OBSERVABLE -> REPLAY_PAST_AND_FUTURE_TRANSACTIONS_FLOWABLE

Pipeline Variables

The following automatically filled pipeline variables were renamed:

platform6.request.cluster -> platform6.request.dataPartition
platform6.request.concept -> platform6.request.dataType

Routing Order Attribute Changes¶

Routing orders were designed to store calls to adapters and their parameters. This was later extended to also include service calls via the svc:\\ service URL syntax.

Since adapters have been completely removed in Platform 6, the Routing Order attributes: Adapter and Adapter Parameters have been re-purposed to Service and Service Parameters

Warning

The svc:\\ URL syntax is no longer supported.

The Service attribute must now contain the name of the target service. For example platform6.scripts (or scripts for short)

The Service Parameters attribute now contains a comma separated list of URL encoded name/value pairs

The change to format and use of attributes means that any routing orders created prior to this release will not be able to be reprocessed ( Unless they are first heavily edited first using the Routing Orders UI.)

Windows support¶

Installing Platform 6 natively on Windows is no longer supported. If you want to use Platform 6 on Windows it has to be done using Docker through the Platform 6 install scripts on Github.

Dependency Upgrades¶

Bouncy Castle from 1.59 to 1.61
Flying Saucer PDF from 9.0.7 to 9.1.18
Groovy from 2.5.6 to 2.5.7
Web3j from 3.5.0 to 4.3.1

log4j.properties¶

If you have made a copy of the log4j.properties file and stored it as an override in your instance p6core_data/conf folder you must update it.

The line:

log4j.appender.b2box = io.platform6.core.impl.logging.B2boxAppender

should be changed to

log4j.appender.platform6 = io.platform6.core.impl.logging.Platform6Appender

And the corresponding reference at the top of the file to appender b2box must be changed to platform6

log4j.rootCategory=ALL, platform6, rfile

Version `6.0.0-beta-1`¶

ApplicationId vs. InstanceId¶

A terminology change during development of Platform 6 led to a confusing mix of terms where applicationId and instanceId where used interchangeably to refer the same thing!

We have now standardised on InstanceId

Warning

This effects a mandatory attribute in the application.conf of every instance

The application.conf attribute:

applicationid = myp6instance

must be changed to:

instance.id = myp6instance

before Platform 6 will start.

DSL¶

The package name containing the DSL has been renamed from com.amalto.b2box.groovy to io.platform6.core.dsl and the DSL are no longer accessible without using the prefix p6..

Annotations

The following annotations was renamed:

B2GroovyBinding -> P6GroovyBinding
B2ExecutionContext -> P6ExecutionContext
B2User -> P6User

and also moved from the package com.amalto.b2box.groovy.dsl.bindingto io.platform6.core.impl.application.scripting.annotations

Transaction DSL

The getSerialForm method has been removed

Table DSL

The following function was renamed:

upsertRecords -> upsert

Note: Existing scripts will be updated to use the new DSL function so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Utils DSL

The following bindings has been moved (for both scripts and routes):

escapeXml -> p6.utils.escapeXml
unescapeXml -> p6.utils.unescapeXml
pause -> p6.utils.pause

Note: Existing scripts will be updated so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Routes¶

The following bindings has been moved for route definition:

utils -> p6.utils
configuration -> p6.configuration
camel -> p6.camel
service -> p6.service
appconfig -> p6.appconfig

Note: Existing routes will be updated so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Exception¶

The use of io.platform6.core.api.B2boxException has been replaced by io.platform6.common.util.P6Exception.

Note: Existing routes will be updated to use the new exception so no manual action is required.

Warning: If you are using P6Sync don’t forget to retrieve the online modification using the -pull option

Configuration files¶

The following file was renamed:

b2boxcontext.xml -> p6context.xml

The configuration entries starting with b2audit. have been renamed to audit.. And configuration entries starting with b2auth. have been renamed to p6auth..

Log appender renamed¶

io.platform6.core.impl.logging.B2boxAppender was renamed to io.platform6.core.impl.logging.Platform6Appender.

Services¶

Transactions¶

The endpoint [DELETE] /apis/v2/platform6.transactions/transaction/{view}/{ids:.*} has been removed. You should use the endpoint [DELETE] /apis/v2/platform6.transactions/transactions instead.

Endpoint permissions¶

The endpoints to deal with file submission have change and you need the following permissions:

Service	createSession, addFile	deleteFile, deleteSession	listFiles
transactions	[transactions=edit]	[transactions=delete]	[transactions=read]
bundledresources	[bundledresources=edit]	[bundledresources=delete]	[bundledresources=read]
workflowsteps	[workflowsteps=edit]	[workflowsteps=delete]	[workflowsteps=read]

Version `6.0.0-beta-2`¶

Endpoint permissions¶

Endpoint permissions have been simplified and the ones that have been modified are listed bellow.

REST API	Old permissions	New permissions
Add an item	SERVICE_ID=read && SERVICE_ID=edit	SERVICE_ID=edit
Delete an item	SERVICE_ID=read && SERVICE_ID=edit	SERVICE_ID=read && SERVICE_ID=delete
Delete items	SERVICE_ID=read && SERVICE_ID=edit	SERVICE_ID=read && SERVICE_ID=delete
Export items	SERVICE_ID=read && SERVICE_ID=edit	SERVICE_ID=read

Applications¶

The following services will be automatically restarted following application installation and un-installation (assuming the application contains service items deployed against each service)

Workflow Steps
Routes
Transactions

No manual action is required, except cleaning up your install and uninstall scripts in your applications from calls that restart the above services.

Configuration¶

All the following configuration keys have been replaced by p6.realm:

groovy.b2.realm
default.p6.realm
b2box.b2.realm
service.workflow.b2.realm

The following configuration entries have been renamed:

counters.* -> p6.service.counters.*
service.workflow.* -> p6.service.workflowsteps.*
service.* -> p6.service.*
com.amalto.service.<name.serviceName>.limiter.* -> p6.service.<name>.limiter.*
io.platform6.core.impl.application.camel.p6log.LogConsumer -> p6.camel.p6log.LogConsumer
audit.* -> p6.service.auditing.*
store.* -> p6.store.*
swagger.ui -> p6.swagger.ui
io.platform6.core.impl.platform.jobcontrol.JobControlManagerImpl.* -> p6.job.*
job.control.execution.messages.length -> p6.job.control.execution.messages.length
disable.get.cache.headers -> p6.cache.annotation
cmb.service.runner.pool.size.<name> -> p6.service.<name>.runner.pool.size

DSL Changes¶

The method defaultTransactionManager in the ethereumrpc DSL was renamed to noOpTransactionManager.

The DSL p6.control was renamed to p6.script. The method moduleNames in the script DSL was renamed to list.

Note

The renaming is automatically handled in scripts, no manual action is required.

The method getCompiled in the resource DSL was removed.

Version `6.0.0`¶

Before the migration¶

The default redirect URI for OAuth clients was changed from http://www.b2box.com to http://www.platform6.io. Therefore if your instance still uses the default b2box client to connect to P6 Auth, add the following line to application.conf:

p6auth.client.redirect.uri="http://www.b2box.com"

Variables¶

The following variables have been renamed in the application:

B2BOX_DATA -> P6_DATA
B2BOX_HOME -> P6_HOME

This affects:

Scripts (main script and resources)
Routes (template and script segments)
URL (any use of an URL starting with file:)
Bundled resources (target root)
Workflow steps (URL inside the definition of thr xml step)

Note

The renaming is automatically handled in the previous cases by SQL migration scripts

Use official Camel Web3j routes¶

This release is aligned with the official Camel Web3j module. Therefore many custom headers that were added in previous versions were removed. Here is the list of removed headers:

logAddress
logBlockHash
logBlockNumber
logData
logIndex
logTransactionHash
logTransactionIndex
logType
logTopics

These headers are the properties of a org.web3j.protocol.core.methods.response.Log that were extracted. To migrate your blockchain listener routes, you need to serialize every incoming Log from the blockchain to JSON, then parse the JSON string and read the property you want. A full example of such a route is provided in the Demo App.

Version `6.0.1`¶

Before the migration¶

Make sure to change volume mapping in Docker to INSTANCE_DATA_PATH/p6core.data:/opt/p6core.data (rather than b2box5).

DSL Changes¶

After all enhancements made to the AppConfig DSL, the Ethereum DSL no longer relies on the Tables DSL to store credentials. Therefore, the following functions were removed:

String addNewCredentials(String tableName, Map keys, String fieldName, String password, boolean useFullScrypt)
Credentials getCredentials(String tableName, Map keys, String fieldName, String password)

Some Changes To Defaults¶

The default p6auth.client.id in reference.conf is no longer b2box. It is now p6core. You may need to change you current Client definition using OAuth setting or switch to the more current Integrations way of integrating p6core instances with P6 Auth.
The default log file name has changed in log4j.properties from b2box.log to p6core.log.
The default install location of the product within the docker container has changed from /opt/b2box to /opt/p6core.
The default TMP folder used within the docker container has changed from /tmp/b2box to /tmp/p6core.

New JVM¶

For licencing reasons and for future independence we have stopped using the Oracle distribution of Java. We now use the official OpenJDK provided Docker container image:

openjdk:8-jre-alpine

This is a significant change. Not only are we moving away from the Oracle distribution, but we are also moving from a JDK to a JRE. In combination with the Alpine linux distribution the overall size of the P6 Core container has been significantly reduced.

Note: We no longer extend the original b2base image that contained the S6 service manager. We no longer configure the postfix service or run P6 Core as a service.

Services¶

Bundled resources¶

Endpoints¶

The endpoint [DELETE] /apis/v2/platform6.bundledresources/bundledresource/{name}/{appKey: .*} has been removed. You should use the endpoint [POST] /apis/v2/platform6.bundledresources/bundledresources/delete instead.

The endpoint [PUT] /apis/v2/platform6.bundledresources/bundledresource has been removed. You should use the endpoint [PUT] /apis/v2/platform6.bundledresources/bundledresources/{sessionToken: .*} instead. The body of the request has to be the BundledResource item and the sessionToken can be empty.

The endpoint [POST] /apis/v2/platform6.bundledresources/bundledresource has been removed. You should use the endpoint [POST] /apis/v2/platform6.bundledresources/bundledresources/{sessionToken: .*} instead. The body of the request has to be the BundledResource item and the sessionToken can be empty.

Version `6.0.3`¶

Before the migration¶

All class packages were renamed from com.amalto.b2box... to io.platform6.... Therefore, make sure class names are correct in your properties files in the conf folder. For instance, com.amalto.b2box.core.store.impl.dialect.pgsqlxml.mappers.proc.ReportingDataSourceProcMapper was renamed to io.platform6.core.store.impl.dialect.pgsqlxml.mappers.proc.ReportingDataSourceProcMapper.

Pipeline Changes¶

The content type of variables injected in the pipeline changed as follows:

application/b2box.itempk -> application/p6core.itempk
application/b2box.dom -> application/p6core.dom
application/b2box.i18n -> application/p6core.i18n

Note

Renaming is automatically handled by SQL migration scripts, no manual action required.

Version `6.0.6`¶

Services¶

Audit¶

The configuration keys moved from audit to p6.audit.

Only the following providers are now available:

LOG4J
ELASTICSEARCH

Example:

p6.service.auditing.provider=ELASTICSEARCH

Version `6.1.0`¶

DSL¶

ethereumrpc¶

The name of this DSL has been changed to web3ethereum.

Note

Renaming is automatically handled by SQL migration scripts, no manual action required.

Version `6.1.1`¶

DSL¶

Log¶

The log binding inside the scripts is now deprecated and calls should be replaced by p6.log.

The p6.log binding is also available inside route definitions.

Note

The old binding is kept for backward compatibility but will be removed in a next release.

Service¶

Workflow step¶

The use of the node <AllowMessageEdit> was deprecated and has been removed. You must now use the node <AllowTransactionEdit> instead as describe in the workflow step configuration

Note

This is automatically handled by SQL migration scripts, no manual action required.

When invoking a workflow step, the header of the request named platform6.request.dataPartition is now deprecated and can be removed. If set, the value will be ignored.

Version `6.2.1`¶

Database¶

The table item has been removed. Data are now split in dedicated tables:

log
table_data
transaction

On each table the column datapartition has also been removed.

The table_data table has new iid columns:

iid5
iid6
iid7
iid8

Version `6.3.3`¶

Monitoring¶

The following monitoring endpoints have been removed:

http://:8080/p6/vmstats
http://:8080/p6/log4jstats

Sensu alerts need to be migrated to target the “/p6” endpoint which returns 200 OK to signal that the instance is up and running.

Views¶

The <Searchable> type values have been renamed:

IStringEquals -> CaseInsensitiveString
StringEquals -> String
StringContains -> Words

Note

This is automatically handled by SQL migration scripts, no manual action required.

Dependencies¶

Saxon-b¶

The saxon-b dependency has been removed from Platform 6’s code base.

If you still need Saxon-b either add it using the Bundled Resources service or add the JAR to the classpath manually.

Warning

Due to a security issue it is not recommended to use that dependency. You should migrate to Saxon-HE that is already shipped inside Platform 6.

XML/XSL security¶

Explanation¶

Protections have been added to prevent XXE attacks in XML and XSL files.

The following parameters have been disabled:

external general entities
external parameter entities
external dtd
XInclude markup

If an XSL file uses a reference to a file or a URL

<!DOCTYPE transform [<!ENTITY xxe SYSTEM "file:///etc/passwd" >]>
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
<xsl:template match="/">
...

a P6Exception will be thrown Error reported by XML parser processing null: URI scheme 'file' has been disallowed

If an XML file uses a reference to a file or a URL, the content of the variable will be an empty string.

Impact¶

DSL
- Table
- Transaction
- XmlSplit
- XSLT
Services

Workflow step WorkItem node¶

The node <WorkItem> (not the one under <ViewNames>) has been renamed to <WorkflowTaskEnhancer>

Note

This is automatically handled by SQL migration scripts, no manual action required.

Workflow step Parameters node¶

Schema validation has been added to the Workflow Step items. Exceptions will be thrown while trying to save invalid XML definitions.

The nodes <Parameter> have to be wrapped inside a new <Parameters> node.

Before

<WorkflowStep>
    <Actions>
        <Parameter>...</Parameter>
        <Parameter>...</Parameter>
    </Actions>
</WorkflowStep>

After

<WorkflowStep>
    <Actions>
        <Parameters>
            <Parameter>...</Parameter>
            <Parameter>...</Parameter>
        </Parameters>
    </Actions>
</WorkflowStep>

Warning

This change has to be done manually because it cannot be automated!

Version `6.4.0`¶

Database¶

Since the version 12, PostgreSQL integrates a new progress view for index creation.

Note

Although not mandatory, it’s highly recommended migrating to the latest version of PostgreSQL

A new index management has been implemented for Views and TableData. Now, when editing one of those items you can generate all the indexes needed with one click. However, it will not deal with existing indexes because the process has its own naming convention.

Example (for table data):

before: xraxsszuvdomvnag6s
after: idx_p6_demo_items_8xknhcfpft7lvxbt

Example (for transactions):

before: item_ti_fts_idx01
after: idx_p6_demo_transactions_c6gnmiidbbam2fij

To avoid duplicate indexes and unused indexes, please remove all previous indexes that are not required anymore after running the index creation for the first time.

Note

For items bundled with an application, indexes are created automatically at application installation. Indexes are also dropped when uninstalling the application.

Configuration¶

The configuration key p6.realm has been replaced by p6auth.realm.default provided by P6 Auth.

No need to add it in application.conf unless you want to override the value of P6 Auth.

Version `6.4.3`¶

Docker Image Changes

For security reasons, P6 Core no longer runs as the root user within Docker.

A new linux user:p6user and group:p6 must now be created. (not required on MacOS)

addgroup -gid 10102 p6
adduser --system --gid 10102 --uid 10101 --no-create-home --disabled-login p6user

Warning

It is essential that the ownership of the filesystem used by P6 Core is configured to allow the new user p6user:p6 r/w access.

When the new P6 Core container starts it will automatically attempt to set the permissions of path /opt/p6core.data. So no manual updates should be required.

However, the permissions check may take some time to run on a file system containing large numbers of files/folders. Use docker logs p6core to monitor progress.

Please ensure your filesystem does not accumulate large numbers of unnecessary files as this wastes disk space and will slow P6 Core start time.

If P6 Core attempts to access files outside the /opt/p6core.data folder it will fail. This is not recommended and P6 Core processing should be changed to be constrained within the p6core.data folder only.

Docker Image Location

The location of Docker images has changed from repo.amalto.com to docker.io.

Docker images will now be managed via the official Docker Hub.

The change allows our images to be digitally signed and regularly scanned for security vulnerabilities.

Warning

A change is required to the docker-compose.yaml file to reference the new repository location.

Edit the docker-compose.yaml and change:

image: repo.amalto.com/p6core:6.4.2 (or amalto/platform6:6.4.2)

to

image: amalto/p6core:6.4.3

Version `6.5.0`¶

Dependency Upgrades¶

Apache POI from 4.1.0 to 5.0.0
getCellTypeEnum() to getCellType()
getCachedFormulaResultTypeEnum() to getCachedFormulaResultType()

Version `6.6.0`¶

Script Package Structure¶

Scripts are now packaged in io.platform6.app.[core|App Key].scripts.[Normalized Script Module Name]
If you use p6sync and maintain your scripts via an IDE you will need to update the package structure manually or recreate using the p6sync ‘-pull’ option
If you only work with scripts via the Portal no change is required.

Disabling Data Permission Validation On Startup¶

It is now possible (but not recommended) to disable the automatic permission and ownership checking of all files in /opt/p6core and /tmp/p6core each time p6core starts
Simply set the environment variable P6_DISABLE_FIXATTRS=true in your docker-compose.yml

Script ‘main’ via p6sync¶

When using p6sync to create a filesystem project and that project included more than one ‘auto packaged’ script, the IDE was unable to build the project due to duplicate class definitions:

More than one Main.class in the same default package in the same project!

Note

An IDE such as IntelliJ must be able to build a project before it can run any Groovy script or test Specification

To avoid this issue, p6sync now creates all main script classes with a unique name:

[application key]_[script name]_Main.groovy

Existing application source projects must be pushed to an instance BEFORE upgrading to 6.6.0. Once upgrade is complete, the source tree can be re-created using a pull

Camel component p6cmb no longer forwards WrappedFile content¶

After experiencing Out Of Memory exceptions processing large files with the Camel file component and passing the results to a script via the p6cmb component, the default behaviour of the p6cmb component has changed.

The p6cmb component no longer passes a WrappedFile camel body (such as that generated by the file component) as a CMB attachment.

For example, a script that is triggered as a result of a file being detected by the file component will no longer have the content of the file made available via p6.pipeline.get('body')

In this case, if the script cannot be re-written to load the file content from the URI available in the pipeline then a new parameter can be added to the route definition:

allowWrappedFileContent

For example:

  from("file:///opt/b2box5.data/resources/sshd/repository/suez_drop/in?antInclude=*.ZIP,*.zip&move=../processed&moveFailed=../error&readLock=changed&readLockCheckInterval=5000")
  .to( "p6cmb://scripts?allowWrappedFileContent=true&platform6.request.action=execute&id=Handle_DataOne_Zip" )

Camel context default attribute changes¶

By default, Tracing, Message History and Breadcrumbs are now disabled

Version `6.7.1`¶

Configuration¶

All entry key starting with p6.event or p6.transaction.event can be removed from configuration files.

Version `6.8.0`¶

Unstable version

Please skip 6.8.0 and upgrade straight to 6.8.1

Script Execution Sandbox Disallows .execute()¶

Breaking change

packaged scripts now disallows the use of the execute method of the Groovy extended String and ArrayList classes

Explanations

The script execution sandbox that is deployed for all packaged scripts now disallows the use of the execute method of the Groovy extended String and ArrayList classes The sandbox also prohibits the use of the AntBuilder class.

The sandbox was extended after a recent penetration test highlighted the security vulnerabilities exposed by these methods

Groovy 3 Upgrade (rolled back in 6.8.1)¶

Groovy Release notes

Release notes here: ttps://groovy-lang.org/releasenotes/groovy-3.0.html

Rollback

This was rolled back in 6.8.1.

Version `6.9.0`¶

Java 17¶

Platform 6 now compiles and runs with Java 17.

This is a significant upgrade from the previous Java 8 but every attempt has been made to ensure backward compatibility.

P6 Auth¶

Breaking change

P6Core application configuration should point to a p6auth url finishing by p6auth. New endpoints are not retro-compatible.

Not supported

p6auth.remote.provider.url: "https://login.amalto.io/apis/v2/b2auth"

Supported

p6auth.remote.provider.url: "https://login.amalto.io/apis/v2/p6auth"

URI Protocol Handler Changes¶

Breaking change

The use of file://${P6_DATA}/... is now deprecated and will no longer work when P6 Core is migrated to Java 17 and should be replaced by p6file protocol instead.

Backward compatible

The file protocol with URLs will continue to be valid and can continue to be used. It is only the keyword expansion capability that will soon stop working!

Explanations

Keyword expansion within URIs with the file protocol has long been the recommended and portable way to reference the file system without hard-coding particular paths. Unfortunately, it is not possible to enhance the file protocol handler in newer version of the P6 runtime: Java.
We have made a protocol available and should be used as an alternative: p6file.

Example

p6file://${P6_DATA}/resources/certificates

Automatic migration

The services scripts, workflowsteps, routes, appconfig and storedprocedures were automatically migrated to the new syntax.

Views¶

Breaking change

SmartTags nodes are removed.
SDOC and SURI features are replaced by buttons with respectively the DISPLAY and DOWNLOAD action.

Migration steps

Before

<Viewable>
    ...
    <Type key="View_Document_${BusinessDocumentName}_${CurrentDocumentFormat}">SURI</Type>
</Viewable>

After

Create a button named View_Document in the Button service item.
Create the script used on the On click using the migration example
- From SURI
- From SDOC

Change the Type definition inside the view

<Viewable>
    ...
    <Type key="View_Document">Button</Type>
</Viewable>

Camel Version 3¶

Breaking change

Due to migration to Camel 3, component quartz2 should be renamed to quartz

Automatic migration

A migration script will automatically rename quartz2 to quartz in all found camel URIs

Workflow steps¶

Breaking change

The following deprecated nodes have been removed:

FormGenerator
Parameter

Manual migration

Use FormId instead of FormGenerator
Use Parameters.Parameter instead of Parameter

Homepage¶

Breaking change

Support for reports has been removed
Structure of service item has changed (cf. Homepage changes)
Service items will be displayed on distinct lines

Report migration

Reports have to be converted into Chart before updating to that version

Do not forget to also replace, in your homepage configuration, the reports items with the new charts.

Homepage changes

The structure of the Homepage service item has changed.

In the previous version of the service, the services items were displayed based on order. Depending on the screen dimension, their disposition could change.
In the new version, services items have a determined position using x, y. Where x and y correspond to the columns and lines where it should be placed.

Automatic migration

The service item conversions are automatically handled by SQL migration scripts, no manual action required.

During the migration, all the services items will be placed in distinct lines (y = order and x = 0). It is up to you to change the size and position to fit the new display mechanism.

Code modification

Before 6.9.0

{
    ...
    "type": "charts",
    "order": 1,
    "width": "small",
    "height": "medium"
}

Since 6.9.0

{
    ...
    "type": "charts",
    "x": 0,
    "y": 1,
    "width": "small",
    "height": "medium"
}

Additional information

If you are developing applications, synchronize your application after updating to retrieve the updated data. If you want to create a homepage by hand, use the new syntax according to the example above.

Reports service¶

Breaking change

Reports service support is dropped.

Manual migration

You must convert your old reports to the new Chart service.

Groovy Version 4¶

Breaking change

GPathResult and XMLSlurper have been moved from package: groovy.util to groovy.xml

Groovy Release notes

Release notes here: https://groovy-lang.org/releasenotes/groovy-4.0.html

P6 Sync Meta JSON¶

Breaking change

The meta.json attribute contentMode has been removed from all service files when using P6 Sync.

Additional information

The attribute contentMode is now ignored when pushing changes

Permissions DSL Method Removal & Changes¶

New features

A new, more powerful method has been made available in this release: - p6.permissions.appUpsert()

Breaking change

The method p6.permissions.appInstall() has been removed and is no longer supported.
The methods p6.permissions.appUpsert(), p6.permissions.appUninstall() and p6.permissions.appRemove() no longer rely on the permissions assigned to the instance Integration

Additional information

Instead, the permissions of The Currently Logged In User are used. This means that the user installing/uninstalling the p6app must have the permsets=install and permsets=edit permissions.

Warning

Please ensure the permsets=install permission is removed from the instance Integration

Change of Behaviour¶

New features

The new DSL method p6.application.appRemove and the method p6.application.appUninstall will auto remove the app permission sets from any users who were assigned them. This makes these methods more useful as it avoids the need to manually unassign permission sets from many users!

Breaking change

Because appRemove and appUninstall now update users, The Currently Logged In User must have users=edit or users=update permissions.

Corda Support dropped¶

Breaking change

Corda has been removed from Platform6.
DSL p6.cordarpc has been removed
Route component corda has been removed

Java Security and AS4¶

Breaking change

The SMP lookup part does not work with Java 17 out of the box

Explanations

From https://github.com/phax/phase4/wiki/Known-Limitations “The SMP lookup part does not work with Java 17 out of the box, because the Peppol SMP enforces the usage of SHA-1 as the signature method and digest method”

Manual migration

To work around this, it is now possible to override java.security on a per-instance basis by adding the file:

$P6_DATA/conf/security/java.security

There are many security entries that can be overridden using this file, however the specific entry required to allow SHA-1 XML signature methods is as follows:

$P6_DATA/conf/security/java.security

jdk.xml.dsig.secureValidationPolicy=\
    disallowAlg http://www.w3.org/TR/1999/REC-xslt-19991116,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#rsa-md5,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#hmac-md5,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#md5,\
    disallowAlg http://www.w3.org/2000/09/xmldsig#dsa-sha1,\
    disallowAlg http://www.w3.org/2007/05/xmldsig-more#sha1-rsa-MGF1,\
    disallowAlg http://www.w3.org/2001/04/xmldsig-more#ecdsa-sha1,\
    maxTransforms 5,\
    maxReferences 30,\
    disallowReferenceUriSchemes file http https,\
    minKeySize RSA 1024,\
    minKeySize DSA 1024,\
    minKeySize EC 224,\
    noDuplicateIds,\
    noRetrievalMethodLoops

This simply omits the following lines which are present in the default:

Defaults values

disallowAlg http://www.w3.org/2000/09/xmldsig#sha1,\
disallowAlg http://www.w3.org/2000/09/xmldsig#rsa-sha1,\

Workflow DSL Deprecation and Changes¶

Breaking change

p6.workflow.uncheckedAssignees has been removed
p6.workflow.isAssignee(String userEmail) has been changed

Changes

The inline boolean parameter has been removed and this method now always examines all current workflow tasks.

Deprecated

method p6.workflow.syncInlineAssignees() is deprecated, and you should use p6.workflow.syncAssignees() instead.

Workflow steps¶

Breaking change

FormGenerator support has been dropped. FormId should be used instead. Cf. FormGenerator migration

Table¶

New features

When displaying table data, underscores are replaced by spaces in the column names. (UI feature only)

Indexation¶

Warning

Tables and Views may not be indexed.

Manual migration

If the instance has Tables or Views names containing - you have to force indexation from the service item detail.

Version `6.9.1`¶

Workflow steps¶

Breaking change

For the UNIT and BRANCH assignee resolution types, a limit is set to 500.

Warning

For large organisations, please consider using the SCRIPT type

Configuration

This limitation can be modified in the configuration entry p6.service.workflowsteps.user.api.limit.

Configuration¶

Breaking change

The property p6.tmp.folder is now used to define the folder where temporary files and folders are going to be created.

Configuration

Delete or leave the p6.tmp.folder property empty to keep the default /tmp value.

Version `6.9.2`¶

Database¶

Shared database¶

New features

Platform6 can now be installed in a shared database. Please refer to Database for more information.

It’s no longer required to specify the schema on the stored procedures queries.

Automatic migration

The migration to remove the default p6core schema from the stored procedures is automatically handled, no manual action is required.

Configuration¶

Breaking change

JDBC connection used to be configured inside pgsqljdbc.properties and reportingjdbc.properties files. The database configuration is now located inside application.conf file.

Backward compatible

This change is backward compatible for the properties jdbc.username and jdbc.password.

Data source tables¶

Breaking change

Generated tables from the DSL method p6.sproc.buildDataSource() are no longer attached to the public schema.

Automatic migration

The migration from public.ds_<tablename> to <schema>.ds_<tablename> inside the stored procedures is automatically handled, no manual action is required.

This change is backward compatible with the properties username and password.

Scripts¶

Breaking change

The Bytes resources type has been removed.

Automatic migration

Existing scripts using this type will be automatically updated to use TEXT instead.

Tables¶

Breaking change

Data exported from table data is no longer sorted in the output file.

The DSL methods p6.table.toList and p6.table.lookup are now returning a limited number of records. (cf. p6.service.tables.batch.size)

Exporting table data will now produce a CSV file by default instead of an Excel spreadsheet

URL¶

Breaking change

The URL resolution of p6file:// replace ${p6.home} and ${P6_HOME} by the value of p6.home system variable.

If missing, the default value is now /opt/p6core instead of /opt/b2box

Views¶

Breaking change

Searchable type RangeOfMillis has been removed.

Permissions¶

Documentation issue

The transaction permission based on the position of the user in the organization tree with FILTER_EXPR USER should be replaced by UNIT (cf. documentation). Filtering by USER was never implemented and wrongly documented.

Expected syntax

transactions=allow('WfView1'(UNIT))

P6Sync¶

Breaking change

P6sync HISTORY operation support has been removed.

Tables¶

Breaking change

Due to p6sync HISTORY operation removal, table data are no longer exported using this command.

Bundled Resources¶

Breaking change

${TMP} has been replaced by ${P6_TMP} in the targetRoot values

Automatic migration

Existing bundled resources will be automatically updated. No manual action is required.

Version `6.9.4`¶

Counters¶

Breaking change

Counters based on tables using SQL mode won’t work and should be migrated using another strategy.

Version `6.9.6`¶

Views¶

The indexing policy changed in this release. P6 used to automatically generate indexes for all searchables in a view. Therefore, if the same XPath expression was used in multiple views, it would be indexed multiple times, adding bloat and slowing updates in the database. Moreover, when a view was deleted, associated indexes were automatically deleted.

With the introduction of viewables and searchables based on SQL column names rather than XPath expressions, this policy changed. The new policy guarantees that an XPath expression or column is only indexed once, and generated indexes are no longer dropped when a view is deleted.

Breaking change

Existing indexes on views have been dropped due to large number of duplicates created during the indexation process. You must go to each views to manually launch the indexation.
Unused indexes are no longer removed by the indexation process.

Migration Troubleshooting

Version 6.0.0-alpha-2¶

Version 6.0.0-alpha-3¶

Version 6.0.0-alpha-4¶

Version 6.0.0-alpha-5¶

Before the migration¶

DSL¶

DataModel¶

Integrations¶

Routes¶

Routing Order Attribute Changes¶

Windows support¶

Dependency Upgrades¶

log4j.properties¶

Version 6.0.0-beta-1¶

ApplicationId vs. InstanceId¶

DSL¶

Routes¶

Exception¶

Configuration files¶

Log appender renamed¶

Services¶

Transactions¶

Endpoint permissions¶

Version 6.0.0-beta-2¶

Endpoint permissions¶

Applications¶

Configuration¶

DSL Changes¶

Version 6.0.0¶

Before the migration¶

Variables¶

Use official Camel Web3j routes¶

Version 6.0.1¶

Before the migration¶

DSL Changes¶

Some Changes To Defaults¶

New JVM¶

Services¶

Bundled resources¶

Endpoints¶

Version 6.0.3¶

Before the migration¶

Pipeline Changes¶

Version 6.0.6¶

Services¶

Audit¶

Version 6.1.0¶

DSL¶

ethereumrpc¶

Version 6.1.1¶

DSL¶

Log¶

Service¶

Workflow step¶

Version 6.2.1¶

Database¶

Version 6.3.3¶

Monitoring¶

Views¶

Dependencies¶

Saxon-b¶

XML/XSL security¶

Explanation¶

Impact¶

Workflow step WorkItem node¶

Workflow step Parameters node¶

Version 6.4.0¶

Database¶

Configuration¶

Version 6.4.3¶

Version 6.5.0¶

Dependency Upgrades¶

Version 6.6.0¶

Script Package Structure¶

Disabling Data Permission Validation On Startup¶

Script ‘main’ via p6sync¶

Camel component p6cmb no longer forwards WrappedFile content¶

Camel context default attribute changes¶

Version 6.7.1¶

Version `6.0.0-alpha-2`¶

Version `6.0.0-alpha-3`¶

Version `6.0.0-alpha-4`¶

Version `6.0.0-alpha-5`¶

Version `6.0.0-beta-1`¶

Version `6.0.0-beta-2`¶

Version `6.0.0`¶

Version `6.0.1`¶

Version `6.0.3`¶

Version `6.0.6`¶

Version `6.1.0`¶

Version `6.1.1`¶

Version `6.2.1`¶

Version `6.3.3`¶

Version `6.4.0`¶

Version `6.4.3`¶

Version `6.5.0`¶

Version `6.6.0`¶

Version `6.7.1`¶

Version `6.8.0`¶

Version `6.9.0`¶

Version `6.9.1`¶

Version `6.9.2`¶

Version `6.9.4`¶

Version `6.9.6`¶