Introduction
This is the second part of the two-part series on achieving file content conversion (FCC) for deep structure with a reusable custom adapter module. The first part covered the conversion of deep XML to flat file at a receiver channel.
This second part will focus on conversion of flat file to deep XML at a sender channel.
Source Code
The full source code can be found in the GitHub repository listed in the first part of this series.
Prerequisite
- Each substructure type must contain a field containing the key value to enable the parser to identify lines of this substructure type
- Each substructure type must belong to one and only one parent substructure type. The parent can be either the root element or other substructure types defined
Module Parameter Reference
Below are the parameters for configuration of the module for flat file to deep XML conversion (conversionType = 'DeepPlain2XML'). Certain parameters will automatically inherit the default values if it is not configured.
The parameters are separated into parameters that are common for this conversion type, and also parameters that are specific for field separator based or fixed length based conversion.
1) Common Parameters
| Parameter Name | Allowed values | Default value | Remarks |
|---|---|---|---|
| conversionType | DeepPlain2XML | Required field. Determines conversion type | |
| encoding | UTF-8 | Defines encoding of output plain text file | |
| documentName | Required field. Document name of root element of XML output | ||
| documentNamespace | Required field. Namespace of root element of XML output | ||
| recordsetStructure | Required field. List of substructure names separated by comma | ||
| keyFieldName | Required field. Name of the key field that must occur in all substructures | ||
| indentXML | Y, N | N | Determines if XML output will be indented or not |
| rowOffset | Integer values beginning from 1 | 0 | Starting row to begin extracting content from (i.e. 0 = start from first row, 1 = start from second row) |
| trimContents | Y, N | Y | Determines if all the leading and subsequent blanks for a field will be trimmed or not |
| <StructureName>.fieldNames | Required field. Names of fields in this substructure | ||
| <StructureName>.keyFieldValue | Required field. Key field value to identify this substructure | ||
| <StructureName>.parent | Required field. Specifies the parent of this substructure. Value should correspond to a substructure in recordsetStructure or the value 'Root' to indicate that it belongs directly under the root element | ||
| debug | Displays audit log messages of substructure and parent-child determination of each line. WARNING: Use this only for debugging in non-productive systems | ||
| messageLog | pre, post | Saves a log version of the message that is viewable in Message Monitor
| |
| logLocation | default | Name of log version when messageLog is populated |
2) Field Separator Conversion Parameters
| Parameter Name | Allowed values | Default value | Remarks |
|---|---|---|---|
| defaultFieldSeparator | Defines a default field separator for all substructures that do not have <StructureName>.fieldSeparator explicitly configured | ||
| <StructureName>.fieldSeparator | Either <StructureName>.fieldFixedLengths or <StructureName>.fieldSeparator must be populated | ||
| <StructureName>.enclosureSignBegin | Specify a string that acts as a text delimiter. Field separators within such texts are ignored | ||
| <StructureName>.enclosureSignEnd | If the text delimiters for the beginning and end of the text are different, specify the text delimiter for the end of the text here. If it is not populated, it uses value specified in <StructureName>.enclosureSignBegin | ||
| <StructureName>.enclosureSignBeginEscape | Specify a string that replaces the text delimiter if it occurs within a text that it delimits. When the text is transferred the string is replaced by the value specified in <StructureName>.enclosureSignBegin | ||
| <StructureName>.enclosureSignEndEscape | Specify a string that replaces the text delimiter for the end of the text if it occurs within a text that it delimits. When the text is transferred the string is replaced by the value specified in <StructureName>.enclosureSignEnd. If it is not populated, it uses value soecified in <StructureName>.enclosureSignBeginEscape | ||
| <StructureName>.enclosureConversion | Y, N | Y |
|
3) Fixed Length Conversion Parameters
| Parameter Name | Allowed values | Default value | Remarks |
|---|---|---|---|
| <StructureName>.fieldFixedLengths | Integer separated by commas | Either <StructureName>.fieldFixedLengths or <StructureName>.fieldSeparator must be populated. The number of lengths must match the number of fields listed in <StructureName>.fieldNames |
Similar to standard FCC functionality, fieldSeparator parameter supports non-printable ASCII characters as follows:
- characters encoded as hexadecimal values in the form of '0xHH' (including quotation marks)
Example Scenarios
Here are some example scenarios of the behavior of the conversion based on different configuration options.
Scenario 1
3 substructures with 3 level deep structure
Field separator based conversion
Field separator - Order and Item using default comma. Delivery using tab (specified in hexadecimal)
Order uses " as enclosure sign, and "" as enclosure escape
Item uses " as enclosure sign but enclosure sign is not removed when transferred
Row offset provided to skip first line
Output XML indented
Module parameters
| Parameter Name | Parameter Value |
|---|---|
| conversionType | DeepPlain2XML |
| documentName | MT_DeepPlain2XML |
| documentNamespace | urn:equalize:com |
| recordsetStructure | Delivery,Order,Item |
| keyFieldName | Type |
| Delivery.fieldNames | Type,DeliveryNo |
| Order.fieldNames | Type,DeliveryNo,OrderNo |
| Item.fieldNames | Type,OrderNo,ItemNo,Quantity |
| Delivery.keyFieldValue | D |
| Order.keyFieldValue | O |
| Item.keyFieldValue | I |
| Delivery.parent | Root |
| Order.parent | Delivery |
| Item.parent | Order |
| Delivery.fieldSeparator | '0x09' |
| defaultFieldSeparator | , |
| Order.enclosureSignBegin | " |
| Order.enclosureSignBeginEscape | "" |
| Item.enclosureConversion | N |
| Item.enclosureSignBegin | " |
| rowOffset | 1 |
| indentXML | Y |
Result
| Input |  |
| Output |  |
Scenario 2
4 substructures with 3 level deep structure
Fixed length based conversion
Leading and trailing whitespaces are not trimmed
Missing fields are automatically populated with blank
Output XML indented
Module parameters
| Parameter Name | Parameter Value |
|---|---|
| conversionType | DeepPlain2XML |
| documentName | MT_DeepPlain2XML |
| documentNamespace | urn:equalize:com |
| recordsetStructure | Header,Delivery,Order,Item |
| keyFieldName | Type |
| Header.fieldNames | Type,FileName |
| Delivery.fieldNames | Type,DeliveryNo |
| Order.fieldNames | Type,DeliveryNo,OrderNo |
| Item.fieldNames | Type,OrderNo,ItemNo,Quantity |
| Header.keyFieldValue | H |
| Delivery.keyFieldValue | D |
| Order.keyFieldValue | O |
| Item.keyFieldValue | I |
| Header.parent | Root |
| Delivery.parent | Root |
| Order.parent | Delivery |
| Item.parent | Order |
| Header.fieldFixedLengths | 5,20 |
| Delivery.fieldFixedLengths | 5,10 |
| Order.fieldFixedLengths | 5,10,10 |
| Item.fieldFixedLengths | 5,10,10,10 |
| trimContents | N |
| indentXML | Y |
Result
| Input |  |
| Output |  |
Scenario 3
Finally for the third scenario, we will use the same input file that was used in the XSLT approach. Actual configuration and testing screenshots are shown.
Default field separator for all substructures is used.
A log version of the message payload before conversion is saved.
Module configuration on an SFTP sender channel.
The 'plain' log version shows the flat file before conversion.
The converted payload is sent to a receiver proxy without further mapping. As shown below, the target payload has been converted to a deeply nested XML structure.
The audit log shows the trace of steps being executed by the module.
Conclusion
With the completion of this second part for DeepFCCBean, we now have a generic, reusable and highly configurable custom adapter module that is able to handle conversion of flat file to deep XML and vice versa! 