Introduction

The purpose of this document is to show all the steps needed to configure an IDoc_AAE to file scenario

This document will help SAP PI consultants to understand the ALE (Application Link Enabling) and NWA settings needed to complete this scenario.

Systems involved in developing the scenario

·         Sender system : SAP ECC (SAPDEV_100) Client No -100

·         PI system : SAP NETWEAVER PI 7.3 (PIDEV_100) ) Client No-100

·         Receiver : Local File Directory (NFS)

Brief Overview of Configurations

a) Configurations in NWA

1)      Settings in Application Resources

2)      Create JCO RFC Provider destination

b) Configurations at the Sender System (SAP ECC)

1)      Create RFC Destination

2)      Create Port

3)      Create Logical System

4)      Create Partner Profile

c) Design the Scenario in the Enterprise Service Repository  

1)      Import the Required IDoc from Sender SAP ECC System

2)      Create Data Type for File structure

3)      Create Message Type for DataType

4)      Create Service Interface

5)      Create Message Mapping doing simple one to one mapping

6)      Create Operation Mapping

d) Configure the Scenario in Integration Directory

1)      Assign Business System for sender ECC

2)      Create Configuration Scenario

3)      Create Receiver Business component

4)      Create Sender Communication Channel

5)      Create Receiver Communication Channel

6)      Create Integrated Configuration

e) Testing the scenario

1)      Trigger the IDoc from ECC

2)      Check the triggered idoc message in PI Adapter engine in RWB

3)      Check whether the Message is received in your file system.

Detailed description of the steps

a) Configurations in NWA

Open NWA page

1) In NWA navigate  to the below path:

     Configuration --> Infrastructure ----> Application Resources

Enter "inboundRA"  in the Resource Name and click on filter icon

Select Resource Adapter, inboundRA

Click on "Properties"

Enter a value for "MaxReaderThreadCount"  between 5 to 10

Enter "true" for Local

XI_IDOC_DEFAULT_PIDEV,PIDEV refers to the PI system ID.

The ProgramID visible here, XI_IDOC_DEFAULT_ PIDEV, must be used when creating the RFC destination of type T on the ECC system

2) Create JCO RFC Provider destination

In NWA navigate  to the below path:

Configuration -->Infrastructure -->JCO RFC Provider

Create JCO RFC Provider destination with name XI_IDOC_DEFAULT_ PIDEV  in NWA where PIDEV refers to the PI System ID.

Enter Gateway host and gateway service of your PI system.

b)Configurations at the Sender System (SAP ECC)

This section describes all the configurations needed in the Sender SAP System (ECC) for sending an IDoc to PI

1. 1. Create RFC Destination

On your SAP ECC system in order to send the IDOCs you need to create an RFC destination of type T (TCP/IP)

Go to Transaction SM59 create a new RFC destination of Type T

1. 1.         1.    Select the radio button Registered Server Program
2. 2.      In the program ID enter the program ID from inboundRA of NWA(explained in the later part of the document)
3. 3.      Enter  the gateway host and gateway service of your PI  server

In PIDEVCLNT100_AAE  and  XI_IDOC_DEFAULT_PIDEV , PIDEV refers to the PI system ID.

Scroll down and fill the following details

Enter Gateway host and gateway service of your PI system.

After you configure that you should be able to check the configuration using the test connection button on the RFC destination.

1. 2. Create Port

Goto Transaction WE21

1. 1.      Click on the Transactional RFC from Ports (left tree panel)
2. 2.      Click on Create Button
3. 3.      Give Port Name(SAP<SID>)here or select generate port name and click on continue. In my example I have given as SAPPIDEV _AAE(PIDEV is PI System Id and _AAE for better understanding)
4. 4.      Give the description of the port and select RFC destination of type T created above. Now click on save.

1. 3. Create Logical System

Goto Transaction BD54

1. 1.      Click on create Button
2. 2.      Give name for your Logical System PIDEVCLNT100( <SID>CLNT<ClintNo.>)where PIDEV  is PI System ID and  100 is client  Number .
3. 3.      Now click on save

1. 4. Create Partner Profile

Create Partner Profile with outbound parameter (WE20) to be send to the PI receiver system

1. 1.      Go to Transaction WE20
2. 2.      Select Partner Type LS
3. 3.      Click on Create Button
4. 4.      Give Logical System Name which we have just created in step 3 as Partner No, Partner Type should be LS , Agent (some valid data) ,Language (EN). Now click on Save
5. 5.      Now create an Outbound Parameter.

o    Select the required Message Type.

o    Select the Receiver Port (which we have created in step 2 SAPPIDEV _AAE in this example).

o    Select Transfer IDoc Immediately option in Output mode for Immediate Testing. Select Basic Type Save.

o    Enter all credentials in the post processing permitted agent tab.

We can see Message Type is added in the outbound parameter.

c) Design the Scenario in the Enterprise Service Repository  

1) Import the Required IDoc from Sender SAP ECC System

2) Create Data Type for File structure

3) Create Message Type for DataType

4) Create Inbound Service Interface for File

5) Create Message Mapping doing simple one to one mapping

6) Create Operation Mapping

d)Configure the Scenario in Integration Directory

Prerequisites

1)Assign Business System for sender ECC

2)Create Configuration Scenario

3)Create Receiver Business component

4)Create Sender Communication Channel using IDOC_AAE adapter

Select Default(recommmended) in RFC Server Parameters

The Ack Destination can be left blank or the value given as above

5)Create Receiver Communication Channel

6)Create Integrated Configuration

Select appropriate sender and receiver components, channels in different tabs of integrated configuration.

e)Testing the scenario

1)Trigger the Idoc from ECC system.

·         Enter the tcode we19 to trigger the idoc

·         Search with message type  of idoc(ORDRSP) that you want to trigger, and then select the idoc type (ORDERS05)

·           We will get Idoc with all its segments

·         Click on the control segment that is EDIDC in this idoc and fill all the sender and receiver details like Port,Partner No and partner type.

·         Then select any other data segment and populate values in the idoc

·         We will get a message as IDoc sent to SAP system or external program

·         After entering values click on the tab standard outbound processing on the top to trigger the idoc.

2) Check the triggered idoc message in PI Adapter engine

     Navigate to Runtime work bench

     Go to Message Monitoring Link, Select Adapter Engine from the combo box Messages from component and click on display button

    You can see the message here with success status

3) Check whether the Message is received in your file system.

     Since we have selected NFS in our channel, we check it in al11 tcode in PI system.

     Go to the target directory that you specified in your channel. You can find your file being placed there.

• SAP’s Cloud for Customer application to on-premise SAP CRM / SAP ERP
• SuccessFactors BizX suite of applications to SAP HCM

These applications provide pre-packaged integration content for HCI, presented in an Integration Content Catalog and accessible over a web-based application. It eases the daily work of configurators, administrators and business analysts for exploring ready to run integration content as well as introductory information and demos. The integration content covers templates with prebuilt process integration, data integration flows and other integration artifacts that significantly reduce the implementation time, cost, and risk. These templates provide the bases for the easy adoption to specific business needs.
The design time is Eclipse based offering an Integration Designer perspective for integration developers to configure, deploy, administer, and monitor integration flows on detailed level.

Have a look at the SAP HANA Cloud Integration Landing Page - Public Integration Content Catalog to see how easy it is for customers and partners to find and understand what it is all about.

• SAP HANA Cloud Integration on SAP.com
• SAP HANA Cloud Integration documentation on SAP Help Portal
• SAP HANA Cloud Integration Product Availability Matrix
• SAP HANA Cloud Integration Knowledge Transfer on SAP Service Marketplace (S-user required)
• SAP Financial Services Network documentation on SAP Help Portal
• SAP Financial Services Network Connector documentation on SAP Help Portal
• SAP Financial Services Network Knowledge Transfer on SAP Service Marketplace (S-user required)

• SAP HANA Cloud Integration - Tools
• SAP HANA Cloud Integration - Public Integration Content Catalog

• SAP Financial Services Network (SAP FSN)
• SAP HANA Cloud Developer Center
• SAP Data Services
• SAP Process Orchestration

• NEW! SAP HCI: Standard and Professional Editions
• NEW! SAP HCI for Application-Development Partners
• NEW!SAP HCI Overview Presentation
• SAP HCI Roadmap (S-User required)
• SAP HCI – Early Customer and Partner Project - sign up to try HCI. It's easy!
• SAP HCI - Getting Started
• SAP HCI - An Intro
• NEW! SAP HANA Cloud Integration - a complementary offering to SAP Process Orchestration
• NEW!  SAP HCI Architectural Overview
• NEW!  10 Developer Blogs on SAP HCI-PI
• NEW!  Authentication and Connectivity Tests in SAP HCI
• NEW! Mapping Overview in SAP HCI
• NEW! Message Monitoring and Administration in SAP HCI
• SAP HCI for Data Services Tutorials

• E-Learning for Integration of SAP Cloud for Customer with SAP ERP and SAP CRM
• A recent information drop reveals many new details for HANA Cloud Integration
• A first attempt at using SAP HCI to integrate Workday and Salesforce

• HANA Cloud Integration - Info Days - WW Tour
• HCI Webinar Series
• SAPInsider hosted webinar on HCI: core capabilities and SalesForce adapter development

Events

• Replays of SAP TechEd && d-code Las Vegas 2014

Update 12 Mar 2015: Update links to second part of the series. Parameter for conversion type has been renamed to XML2DeepPlain. Reformat parameter reference section to split parameters applicable for field separator based and fixed length based conversion.

Introduction

In my recent two-part series on using XSLT to handle FCC of deep structures, I mentioned that even after 10 years, the standard FCC functionality provided has not changed significantly. This is unlikely to change as there are other areas of interest which SAP is focusing on these days. Borne out of my preference for reusable and configurable solutions, I decided to challenge myself to come up with an adapter module solution to hopefully fill this gap, if possible maybe once and for all!

In this two-part series, I will share about DeepFCCBean, the custom adapter module for handling conversion of flat files to deep XML structures and vice versa. As much as possible, the module's behaviors and functionalities are similar to standard SAP's FCC.

This first part covers deep XML to flat file conversion, while the second part covers flat file to deep XML conversion.

Source Code

The full source code can be found in the following public repositories on GitHub.

GitHub repository for DeepFCCBean

GitHub repository for required com.equalize.xpi.util library

EAR deployment file is also available for download from the following GitHub repository release. This EAR can be used for direct deployment in order to use the Adapter Module Bean. Note that this EAR is created based on EJB3.0 version on NWDS 7.31 SP13 Patch 0. For earlier versions of PI, the EJB and EAR file will have to be manually built and compiled using the source code provided.

Module Parameter Reference

Below is a list of the parameters for configuration of the module for deep XML to flat file conversion (conversionType = 'XML2DeepPlain'). Certain parameters will automatically inherit the default values if it is not configured.

The parameters are separated into parameters that are common for this conversion type, and also parameters that are specific for field separator based or fixed length based conversion.

1) Common Parameters

2) Field Separator Conversion Parameters

3) Fixed Length Conversion Parameters

Similar to standard FCC functionality, fieldSeparator and endSeparator parameters support non-printable ASCII characters as follows:

• characters encoded as hexadecimal values in the form of '0xHH' (including quotation marks)
• line break specified using 'nl' (including quotation marks)

Example Scenarios

Here are some example scenarios of the behavior of the conversion based on different configuration options.

Scenario 1

3 level deep input XML with delimited output

Field separator: Delivery and Order using comma. Item using tab (specified in hexadecimal)

End separator: Delivery has additional ZZZZ followed by line break. Order and Item not specified therefore using default line break

Module parameters

Result

Input

<?xml version="1.0" encoding="UTF-8"?> <ns0:MT_Deep xmlns:ns0="urn:equalize/DeepFCC"> <Delivery>   <Type>D</Type>   <DeliveryNo>Delivery2</DeliveryNo>   <Order>   <Type>O</Type>   <DeliveryNo>Delivery2</DeliveryNo>   <OrderNo>Order2_A</OrderNo>   <Item>     <Type>I</Type>     <OrderNo>Order2_A</OrderNo>     <ItemNo>10</ItemNo>     <Quantity>90</Quantity>   </Item>   <Item>     <Type>I</Type>     <OrderNo>Order2_A</OrderNo>     <ItemNo>20</ItemNo>     <Quantity>80</Quantity>   </Item>   </Order> </Delivery> </ns0:MT_Deep>

Output

Scenario 2

3 level deep input XML with fixed length output

Length too short handling: Order set to ignore so long fields can potentially push subsequent fields. Item set to cut to maximum configured length

Module parameters

Result

Input

<?xml version="1.0" encoding="UTF-8"?> <ns0:MT_Deep xmlns:ns0="urn:equalize/DeepFCC"> <Delivery>   <Type>D</Type>   <DeliveryNo>Delivery2</DeliveryNo>   <Order>   <Type>O</Type>   <DeliveryNo>Delivery2</DeliveryNo>   <OrderNo>Order2_A</OrderNo>   <Item>     <Type>I</Type>     <OrderNo>Order2_A</OrderNo>     <ItemNo>10</ItemNo>     <Quantity>90</Quantity>   </Item>   <Item>     <Type>I</Type>     <OrderNo>Order2_A</OrderNo>     <ItemNo>20</ItemNo>     <Quantity>80</Quantity>   </Item>   </Order> </Delivery> </ns0:MT_Deep>

Output

Scenario 3

Finally for the third scenario, we will use the same input file that was used in the XSLT approach. Actual configuration and testing screenshots are shown.

Default field separator for all substructures is used.

A log version of the message payload after conversion is saved.

Module configuration on an SFTP receiver channel.

The 'plain' log version shows the payload after conversion of deep XML to flat file.

The audit log shows the trace of steps being executed by the module.

Conclusion

With DeepFCCBean, content conversion of deeply nested flat files no longer require new custom development.

Do note that it is technically possible for DeepFCCBean to handle the flat files that can be converted by standard FCC. However, it is not the intention of DeepFCCBean to replace SAP's standard functionality, but to complement it in handling deep structures. As such, for flat files that can be converted by standard FCC, it is recommended that standard FCC is used.

Upcoming will be the second part of this series covering flat file to deep XML conversion with DeepFCCBean. Watch this space!

Second part is now out at DeepFCCBean - The better FCC at meeting your deep (structure) needs! (Part 2 - Flat File to Deep XML)

We use PI heavily in our landscape as middleware for connecting our SAP systems to external parties, one of our third party required us to make a secured FTP connection to transfer files from SAP Application server to their publicly listed (exposed on internet) FTP servers. As you can imagine, setting up interfaces through corporate firewalls can be a pain, we managed to setup this connection, get all the firewall rules sorted, tested with a secure FTP program and were able to connect from our network through the firewalls and transfer files securely over the internet.

The next step would be to setup our PI FTP adapter to transfer files through the same secure tunnel. We set it up using the following link: Configuring the Receiver FTP Adapter

The Adapter was configured to use FTPS connection, hence certificates were exchanged accordingly with the third-party, root and intermediate public root certificates were added to TrustedCAs keystore view, however transfer tests failed with the error: ftp 425 failed to establish connection a search on SCN results in nothing that particularly describes the issue and its potential resolution, besides a single thread where a user explains that you need to ask the firewall team to resolve this issue.

However, I was determined to understand why this issue occurs and how it can be prevented/resolved. A bit of research on internet and reading the FTP Specification RFC4217 explains the problem and provides the options to prevent/resolve it.

A typical FTPS works in the following way:

The protocol mandates that FTP client initiates two connections with the FTP Server.

1. Control Connection - For exchange of authentication

2. Data Connection - For transfer of files, this connection can be done in two modes, Active mode or passive mode

     A simplified view of Active Mode is depicted in the following diagram

     a) FTP Client initiates Control Connection

     b) On acceptance, FTP Client initiates Port command with port number

     c) FTP Server Acknowledges the port number and initiates a connection on the same.

     d) FTP Client acknowledges and initiates Data Transfer as necessary

     A simplified view of Passive Mode is depicted in the following diagram    

     a) FTP Client initiates Control Connection

     b) On acceptance, FTP Client initiates PASV command

     c) On acceptance of PASV mode, the Server responds back with IP Address/Port number combination that it will also listen to

     d) FTP Client has a choice, it can initiate a socket on original IP address used for Control Connection or the new IP sent by FTP server, the port number would be the           one returned by FTP server in step c, (some FTP clients like FILEZILLA and LFTP have flags/settings to override the new IP and use the original IP, however SAP PI           Adapter seems to be using the new IP)

     e) FTP Client initiates Data Transfer

     (A very detailed explanation is provided about Active & Passive modes is provided here )

3. The FTP 425 issue occurs when PI Adapter attempts to initiate a Data connection in passive mode, as the PI Adapter attempts to initiate Data connection on the IP address port numbers provided by FTP server in response to PASV command.

Resolution - There are couple of possibilities

Solution #1 - Fix the FTP server - there is a setting on FTP server that ensures that FTP server returns the correct IP address as a response to PASV command from FTP client. We asked our third party supplier to fix this setting and they did the needful, so we din't have to go implement the alternative.

Solution #2 - Allow PI Adapter to Use the new IP provided by FTP Server - This potential solution was researched by a member in the SAP Netweaver Application Server space, he recommends applying a SAP Note 1591971 ,  after implementing the note you get a new parameter (in older versions of PI ) strictHostnameChecking  which allows you to force PI to skip validation check and allow the data connection to be initiated, this solution is documented in the forum post  here.

Note: Solution #2 does not work if the FTP Server sends a garbage IP address which some implementation of FTP Servers do, as that IP won't exist and PI adapter will attempt to connect on this garbage IP and fail, this was our issue, our data connection failed all the time since our third party FTP Servers were dishing out garbage IP addresses to PASV responses.

Introduction

This is the second part of the two-part series on achieving file content conversion (FCC) for deep structure with a reusable custom adapter module. The first part covered the conversion of deep XML to flat file at a receiver channel.

This second part will focus on conversion of flat file to deep XML at a sender channel.

Source Code

The full source code can be found in the GitHub repository listed in the first part of this series.

Prerequisite

• Each substructure type must contain a field containing the key value to enable the parser to identify lines of this substructure type
• Each substructure type must belong to one and only one parent substructure type. The parent can be either the root element or other substructure types defined

Module Parameter Reference

Below are the parameters for configuration of the module for flat file to deep XML conversion (conversionType = 'DeepPlain2XML'). Certain parameters will automatically inherit the default values if it is not configured.

The parameters are separated into parameters that are common for this conversion type, and also parameters that are specific for field separator based or fixed length based conversion.

1) Common Parameters

2) Field Separator Conversion Parameters

3) Fixed Length Conversion Parameters

Similar to standard FCC functionality, fieldSeparator parameter supports non-printable ASCII characters as follows:

• characters encoded as hexadecimal values in the form of '0xHH' (including quotation marks)

Example Scenarios

Here are some example scenarios of the behavior of the conversion based on different configuration options.

Scenario 1

3 substructures with 3 level deep structure

Field separator based conversion

Field separator - Order and Item using default comma. Delivery using tab (specified in hexadecimal)

Order uses " as enclosure sign, and "" as enclosure escape

Item uses " as enclosure sign but enclosure sign is not removed when transferred

Row offset provided to skip first line

Output XML indented

Module parameters

Result

Input
Output

Scenario 2

4 substructures with 3 level deep structure

Fixed length based conversion

Leading and trailing whitespaces are not trimmed

Missing fields are automatically populated with blank

Output XML indented

Module parameters

Result

Input
Output

Scenario 3

Finally for the third scenario, we will use the same input file that was used in the XSLT approach. Actual configuration and testing screenshots are shown.

Default field separator for all substructures is used.

A log version of the message payload before conversion is saved.

Module configuration on an SFTP sender channel.

The 'plain' log version shows the flat file before conversion.

The converted payload is sent to a receiver proxy without further mapping. As shown below, the target payload has been converted to a deeply nested XML structure.

The audit log shows the trace of steps being executed by the module.

Conclusion

With the completion of this second part for DeepFCCBean, we now have a generic, reusable and highly configurable custom adapter module that is able to handle conversion of flat file to deep XML and vice versa!

Introduction

With the increase of mobile and cloud solutions, JSON (JavaScript Object Notation) is becoming popular as a lightweight data-interchange format. The file size of data represented in JSON can be significantly smaller than its corresponding representation in XML. Therefore, more and more web services (especially RESTful services) are using JSON as their data interchange format.

Prior to PI/PO 7.4 SP09, there is no native support for conversion of JSON to XML and vice versa. Even with SP09, JSON conversion is limited as a functionality within the new REST adapter, instead of a generic adapter module that can be used with any adapter.

In this two-part series, I will share about JSONTransformBean, a custom adapter module for handling conversion of JSON to XML and back. This module can be used in PI/PO versions older than PI/PO 7.4 SP09 and/or across different adapter types.

This first part covers JSON to XML conversion, while the second part covers XML to JSON conversion.

Source Code

The full source code can be found in the following public repositories on GitHub.

GitHub repository for JSONTransformBean

GitHub repository for required com.equalize.xpi.util library

EAR deployment file is also available for download from the following GitHub repository release.

This EAR can be used for direct deployment in order to use the Adapter Module Bean. Note that this EAR is created based on EJB3.0 version on NWDS 7.31 SP13 Patch 0. For earlier versions of PI, the EJB and EAR file will have to be manually built and compiled using the source code provided.

This module is based upon the open source JSON in Java library. In order to successfully compile and build this module, the source code for the library needs to be included. Alternatively, the compiled JAR file for the library is also included in the com.equalize.xpi.util library above.

Module Parameter Reference

Below are the parameters for configuration of the module for JSON to XML conversion (conversionType = 'JSON2XML'). Certain parameters will automatically inherit the default values if it is not configured.

Example

Module parameters

Result

Input

{   "glossary": {     "title": "example glossary",     "GlossDiv": {       "title": "S",       "GlossList": {         "GlossEntry": {           "ID": "SGML",           "SortAs": "SGML",           "GlossTerm": "Standard Generalized Markup Language",           "Acronym": "SGML",           "Abbrev": "ISO 8879:1986",           "GlossDef": {             "para": "A meta-markup language, used to create markup languages such as DocBook.",             "GlossSeeAlso": ["GML", "XML"]           },           "GlossSee": "markup"         }       }     }   } }

Output

Screenshots of actual configuration and testing are also shown below.

Module configuration on an SFTP sender channel.

The 'oriJSON' log version shows the original JSON content before conversion.

After conversion, the payload changes to XML format.

The audit log shows the trace of steps being executed by the module.

Conclusion

With JSONTransformBean, we are able to handle now services that provide content in JSON format.

Upcoming will be the second part of this series covering XML to JSON conversion with JSONTransformBean. Watch this space!

Second part is now out at JSONTransformBean Part 2: Converting XML to JSON content

Introduction

This is the second part of the two-part series on conversion of JSON content with a custom adapter module. The first part covered the conversion of JSON to XML at a sender channel.

This second part will focus on conversion of XML to JSON at a receiver channel.

Source Code

The full source code can be found in the GitHub repository listed in the first part of this series.

Module Parameter Reference

Below are the parameters for configuration of the module for XML to JSON conversion (conversionType = 'XML2JSON'). Certain parameters will automatically inherit the default values if it is not configured.

Example

Module parameters

Result

Input

<?xml version="1.0" encoding="UTF-8" standalone="no"?> <ns:MT_XML2JSON xmlns:ns="urn:equalize:com"> <glossary>   <title>example glossary</title>   <GlossDiv>    <GlossList>     <GlossEntry>      <SortAs>SGML</SortAs>      <GlossDef>       <GlossSeeAlso>GML</GlossSeeAlso>       <GlossSeeAlso>XML</GlossSeeAlso>       <para>A meta-markup language, used to create markup languages such as DocBook.</para>      </GlossDef>      <GlossTerm>Standard Generalized Markup Language</GlossTerm>      <GlossSee>markup</GlossSee>      <ID>SGML</ID>      <Acronym>SGML</Acronym>      <Abbrev>ISO 8879:1986</Abbrev>     </GlossEntry>    </GlossList>    <title>S</title>   </GlossDiv> </glossary> </ns:MT_XML2JSON>

Output

Screenshots of actual configuration and testing are also shown below.

Module configuration on an SFTP receiver channel.

The XML input payload.

The 'jsonContent' log version shows the JSON content after conversion. The JSON content produced is compact since parameter indentFactor was not used.

The audit log shows the trace of steps being executed by the module.

Conclusion

With the completion of this second part for JSONTransformBean, we now have a generic and reusable custom adapter module that is able to handle conversion of JSON to XML and vice versa! Bring on the RESTful web services!

• SAP Cloud for Customer to on-premise SAP CRM / SAP ERP
• SuccessFactors to SAP HCM

These applications provide pre-packaged integration content for HCI, presented in an Integration Content Catalog and accessible over a web-based application. It eases the daily work of configurators, administrators and business analysts for exploring ready to run integration content as well as introductory information and demos. The integration content covers templates with prebuilt process integration, data integration flows and other integration artifacts that significantly reduce the implementation time, cost, and risk. These templates provide the bases for the easy adoption to specific business needs.
The design time is Eclipse based offering an Integration Designer perspective for integration developers to configure, deploy, administer, and monitor integration flows on detailed level.

Have a look at the SAP HANA Cloud Integration Landing Page - Public Integration Content Catalog to see how easy it is for customers and partners to find and understand what it is all about.

• SAP HANA Cloud Integration on SAP.com
• SAP HANA Cloud Integration documentation on SAP Help Portal
• SAP HANA Cloud Integration Product Availability Matrix
• SAP HANA Cloud Integration Knowledge Transfer on SAP Service Marketplace (S-user required)
• SAP Financial Services Network documentation on SAP Help Portal
• SAP Financial Services Network Connector documentation on SAP Help Portal
• SAP Financial Services Network Knowledge Transfer on SAP Service Marketplace (S-user required)

• SAP HANA Cloud Integration - Tools
• SAP HANA Cloud Integration - Public Integration Content Catalog

• SAP Financial Services Network (SAP FSN)
• SAP HANA Cloud Developer Center
• SAP Data Services
• SAP Process Orchestration

• SAP HCI: Standard and Professional Editions
• NEW! SAP HCI for Application-Development Partners
• SAP HCI Overview Presentation
• SAP HCI Roadmap (S-User required)
• SAP HCI – Early Customer and Partner Project - sign up to try HCI. It's easy!
• NEW! SAP HCI Partner Content Certification
• SAP HANA Cloud Integration - a complementary offering to SAP Process Orchestration
• SAP HCI Architectural Overview
• 10 Developer Blogs on SAP HCI-PI
• Authentication and Connectivity Tests in SAP HCI
• Mapping Overview in SAP HCI
• Message Monitoring and Administration in SAP HCI
• SAP HCI for Data Services Tutorials

• E-Learning for Integration of SAP Cloud for Customer with SAP ERP and SAP CRM
• A recent information drop reveals many new details for HANA Cloud Integration
• A first attempt at using SAP HCI to integrate Workday and Salesforce

• SAPInsider hosted webinar on HCI: Core capabilities and SalesForce adapter development

Events

• Onsite Event: HANA Cloud Integration - Info Days - WW Tour, next event in the Netherlands March 26th
• Replays of SAP TechEd && d-code Las Vegas 2014

To repeat a target field in message mapping, we can use one-to-one mapping, CopyValue, Graphical Variable and useOneAsMany.

Example 1: -

Let’s try one-to-one mapping and CopyValue in below example. Both one-to-one mapping and CopyValue fail to produce correct result.

Using Graphical Variable / useOneAsMany, produces correct result.

Example 2:-

Using Graphical Variable will not produce correct result, in below example. Using useOneAsMany, produces correct result.

So, how to choose between Graphical Variable and useOneAsMany?

If input hierarchy is maintained in output, choose Graphical Variable. If input hierarchy is becoming flat in output, choose useOneAsMany. FYI, Graphical Variable is simple and efficient at runtime (Link page 8).

Below diagram guides on, choosing between Graphical Variable and useOneAsMany.

FYI. How to use Graphical Variable (slide 9 in this demo). Variables in the Target Structure. useOneAsMany. copyValue.

Requirement of reading entire input text content (CSV, fixed-length) into a output field, can be achieved using 'File Content Conversion' OR 'MessageTransformBean' in sender channel and a UDF in message mapping.

Message mapping to concatenate all lines into a single line.

Create UDF with ‘Execution Type’ : ‘All Values of a Context’. If required change newline character (\r is carriage return, \n is line feed).

public void udf_Concat(String[] eachLine, ResultList outSingleLine, Container container) throws StreamTransformationException {    String outS = "";    for (String line : eachLine) {        outS = outS + line + "\r\n";    }    outSingleLine.addValue(outS);
}

FYI.

Above solution works for XML input also. But, better option is Copy XML Subtree

Reference for this doc: Re: FCC with multiple structure

Other solutions on SCN: Read Entire Input file data as-is in Single node and pass to Output - With Java Mapping in ESR, Whole Payload to a XML field - Process Integration - SCN Wiki

Update 25 Mar 2015: ExcelTransformBean has been refactored to be part of FormatConversionBean. Parameter conversionType replaced with converterClass. Parameter indentXML replaced by indentFactor

Update 11 Feb 2015: Updated source code links to shared module repository on GitHub. Previous GitHub repository will be deleted.

Update 27 Jan 2015: Updated Source code section to indicate EAR deployment file released for download at GitHub repository

Update 3 Nov 2014: Refactoring of source code to cater for SimpleExcel2XML and SimpleXML2Excel conversions.

Introduction

There are already a handful of blogs on SCN dealing with Excel to XML conversion - so why another one??

Here is my wish list for a comprehensive solution:-

• Able to read all kinds of Excel format (XLS and XLSX)
• Behaves in a similar way as MessageTransformBean
• Highly configurable - develop/deploy once, use multiple times
• Able to handle XML special characters

In the reference section below are some of the more popular approaches, however below are some of the (non-exhaustive) limitations/drawbacks

• XLSX files stores string contents in a separate sharedStrings.xml file in the zipped XLSX file
• JExcel API does not support Excel 2007 XLSX formats
• Limitations in handling formulas, formatting and special characters in cells

ExcelTransformBean (ETB) aims to provide a generic adapter module solution (a la MessageTransformBean) that is highly configurable and reusable. It is based on the Apache POI API. Utilizing the combined SS interface of the API, it uses a single logic to read all kinds of Excel files (XLS and XLSX).

This first part covers Excel to simple XML conversion, while the second part covers simple XML to Excel conversion.

Source Code

The full source code can be found in the following public repository on GitHub.

GitHub repository for ExcelTransformBean

Update: EAR deployment file is available for download from following GitHub repository release. This EAR can be used for direct deployment in order to use the Adapter Module Bean. Note that this EAR is created based on EJB3.0 version on NWDS 7.31 SP13 Patch 0. For earlier versions of PI, the EJB and EAR file will have to be manually built and compiled using the source code provided.

This module is based on Apache POI 3.9 library. In order for the Java project to compile and build successfully, the following JAR files need to be referenced/imported into the project.

• poi-3.9-20121203.jar
• poi-ooxml-3.9-20121203.jar
• poi-ooxml-schemas-3.9-20121203.jar
• xmlbeans-2.3.0.jar
• dom4j-1.6.1.jar

The library files can be downloaded from Apache's website, direct link to the ZIP file is provided below.

Apache POI 3.9 ZIP file

Module Parameter Reference

Below is a list of the parameters for configuration of the module for Excel to XML conversion (converterClass = 'com.equalize.xpi.af.modules.excel.Excel2XMLTransformer'). Certain parameters will automatically inherit the default values if it is not configured.

Example Scenarios

Here are some example scenarios of the behavior of the conversion based on different configuration options.

Scenario 1

Excel 2007 XSLX file format.

Extract Sheet1 with column names determined directly from header line of file.

Special character & automatically converted

Module parameters

Result

Input
Output

Scenario 2

Excel binary XLS file format.

Extract sheet at index 0. Column names are provided from configuration.

Row offset provided to skip first two lines.

No formatting of cells, so raw values displayed.

Module parameters

Result

Input
Output

Scenario 3

Excel 2007 XSLX file format.

Extract sheet at index 0.

Column names are not available. Number of columns = 5.

Row offset provided to skip first line.

Empty rows are included.

Cells with formula are displayed with formula instead of result.

Empty cells are displayed with default value "space".

Module parameters

Result

Input
Output

Reference

Part 2 - ExcelTransformBean Part 2: Convert simple XML to various Excel formats easily

This article does not cover the steps for creating a custom adapter module. This can be found easily via SCN search. It is also listed in the reference section of the article below regarding adapter module testing.

Standalone testing of Adapter Module in NWDS

Alternative methods for Excel conversion

PI/XI: Reading MS Excel's XLSX and XLSM files with standard PI modules - easily...

Excel Files - How to handle them in SAP XI/PI (The Alternatives)

A Simple approach in Reading Excel File

Update 25 Mar 2015: ExcelTransformBean has been refactored to be part of FormatConversionBean. Parameter conversionType replaced with converterClass.

Introduction

This is a continuation of the ExcelTransformBean adapter module. The first part ExcelTransformBean Part 1: Convert various Excel formats to simple XML easily covers Excel to simple XML conversion.

This second part focuses on simple XML to Excel conversion. Similar to the first part, this uses the combined SS interface of Apache POI's API, therefore it is able to write both XLS and XLSX files.

Source Code

The source code is shared with the first part. Refer to the corresponding section in the first article for details on accessing the source code from the GitHub repository.

Module Parameter Reference

Below is a list of the parameters for configuration of the module for XML to Excel conversion (converterClass = 'com.equalize.xpi.af.modules.excel.XML2ExcelTransformer'). Certain parameters will automatically inherit the default values if it is not configured.

Note: All fields in the simple XML structure must exists. If any field does not exist, this will cause the alignment of the columns to be incorrect. This is similar to the behavior of FCC or MTB's SimpleXML2Plain conversion. To ensure proper column alignments, use MapWithDefault when mapping the fields of the target structure.

Example Scenarios

Here are some example scenarios of the behavior of the conversion based on different configuration options.

Scenario 1

Default values for all optional parameters.

Excel 2007 XSLX file format with default Sheet1.

No header line.

Module parameters

Result

Input
Output

Scenario 2

Excel binary XLS file format.

Active sheet name provided.

Header line added with details from input XML fields.

Escape sequence &amp; automatically converted to special character &.

Module parameters

Result

Input
Output

Scenario 3

Excel 2007 XSLX file format with default Sheet1.

Header line added. Column names are provided from configuration.

Certain fields are populated with blank values.

Module parameters

Result

Input
Output

Previously featured
SAP HANA Cloud Integration for Application-Development Partners Partners have now the possibility to use SAP HCI for application development to commercialize innovative cloud solutions. Read the blog by Vladimir Dinev to get first insight to the new SAP HANA Cloud Integration Partner Edition. Jan. 2015
Architectural Overview of SAP HANA Cloud Integration SAP HANA Cloud Integration is SAP's strategic cloud-based integration as a service solution. Within his blogSujit Hemachandran presents an architectural overview and highlights also the advantages. Jan. 2015
Understanding Authentication and Testing Connectivity in SAP HANA Cloud Integration This blog series explores the different options to connect systems to SAP HANA Cloud Integration. In the first 2 blogs the supported authentication types are presented, in the later you gain the understanding of the connectivity steps and the connectivity testing. Jan. 2015
Basic Monitoring and Administration with SAP HANA Cloud Integration Integration Developers get now insight on message monitoring and administration with the Integration Operations plug-in. In this blogSujit Hemachandran explains the monitoring basics including also the status checks of the deployments and how to understand the logs in SAP HANA Cloud Integration. Jan. 2015
Ten Integration Developer Blogs on SAP HANA Cloud Integration Sujit Hemachandran explains in extensive blogs important features for Integration Developers. Watch  the technical knowledge sharing  series and get access e.g. to several patterns that may be used in integration flows. Jan. 2015
SAP PI Rest Adapter Watch  Ivo Kulm's collection of blog entries for the PI REST Adapter that shows architectural concepts and configuration for integrating your SAP PI landscape with other REST services. Dec 2014
SAP HANA Cloud Integration Web Tooling Get insight how pre-packaged integration content that is available in the public content catalogue may be directly consumed and if necessary also be extended. Read this blog written by Anette Asmus to understand how to use the Web User Interface for SAP HCI. Dec. 2014
OData Adapter in SAP HANA Cloud Integration Sabarish T S provides in his blog an excellent overview on the OData Adapter and OData support with the SuccessFactors Adapter that are offered with SAP HANA Cloud Integration. Dec. 2014
Watch the SAP TechEd && d-code Replays! If you've missed the integration sessions @ TechEd && d-code in Las Vegas, October 20-24 take now the opportunity to watch the replays and receive the newest highlights of our SAP Process Orchestration / SAP Process Integration and HANA Cloud Integration. Nov.2014
Results of the Global SAP PI survey now available The results of this year's global PI survey is now published. As every year the International Focus Group for Integration conducted a survey and 362 users from over 20 different user groups participated. Watch Holger Himmelmann's blog to see the first part of the results with the analysis of general questions. Nov. 2014
New adapter in SAP Process Integration for consumption and provisioning of REST based services With the latest SP of SAP Process Orchestration / SAP Process Integration our REST adapter is available. Read Alexander Bundschuh's blog to get more details. Nov. 2014
Impact of Integration changes - How to talk to non-SAP systems Read this third blog of a series by Owen Pettiford that gives an excellent view on the new world of SAP's solution set supporting process and data integration. He explains how to take the advantages of the long list of capabilities depending on existing integration reqirements and gives rules for futher decisions. Nov. 2014
New Adapters for Running Hybrid Scenarios with SAP Process Integration Get a comprehensive overview on recent enhancements in terms of connectivity for SAP Process Integration that facilitates support for hybrid scenarios. Alexander Bundschuh presents in his blog the new adapters for integrating on-premise applications with services provided in the cloud. Nov. 2014
SAP has released SP4 for B2B Add-on and Secure Connectivity Add-on After SAP's announcement that B2B is now included in the SAP Process Orchestration license the Support Package 4 is now released with important enhancements. Get an overview on the major enhancements with the blog by Piyush Gakhar. Oct. 2014
Reduce your integration efforts with the new solution "Integration Advisor" The Integration Advisor is a new cloud based solution running on SAP HANA Cloud Integration (HCI) starting with its beta program in January 2015. This tool will tremendously improve and simplify the entire B2B integration project by bringing the different integration phases of business domain experts and integration experts together and by using crowd sourcing based intelligence. Read Gunther Stuhec's article to understand the new solution and have a look on his blog describing his believe how tomorrow's simplified environment for B2B integration will look like. Oct 2014
RKT Workshop on SAP HANA Cloud Integration We are organizing a RKT workshop for SAP HANA Cloud Integration (SAP HCI) in Bangalore on Nov. 20+21, 2014. Visit us and get deep insight on SAP HCI with solution details, use cases and hands-on exercises. Find more information and the registration link in the blog by Piyush Gakhar.  Oct. 2014
Results of Global Process Integration Survey 2014 The blog by Holger Himmelmann reveals the results of the 2014 global SAP PI survey. Thanks to everyone who participated and shared their feedback! See also the central blog International Focus Group for SAP Process Integration (IFG for SAP PI) that gives you the background of the first initiative of SAP user groups around the globe in the area of SAP PI. Oct 2014
Build your SAP TechEd && d-code agenda! Have you already finalized your agenda for TechEd && d-code Las Vegas? The event in Las Vegas is now just one week ahead. Don’t miss the integration sessions @ TechEd && d-code in Las Vegas, October 20-24. Take the advantage to meet the integration experts in person and receive the newest highlights of our on-premise and also cloud-based integration platform. Learn in demo rich lectures and hands-on trainings how SAP integration technologies, like SAP HANA Cloud Integration and SAP Process Orchestration are being used for the integration of SAP's cloud pillars. Get insights into the current solutions and road maps and get a preview on prepackaged content for jump-starting your integration projects. Get on overview on relevant SAP Middleware-related sessions with the blog by Smadar Ludomirski and read the blog by Gunther Stuhec to know all important sessions to understand the new tool of the Integration Advisor and see how it will work in real. Oct 2014
Meet SAP at “I love API 2014” Conference Sep 8 – 10 in San Francisco I  Love APIs 2014 is the conference for business and technology leaders driving digital acceleration with big data and APIs hosted by Apigee. This year SAP has strong presence at the event following the recent strategic partnership announcement  made between SAP and Apigee at end of July. August 2014
OData Adapter and SFSF Adapter (extensions) for SAP Process Integration SP1 of the SAP Process Integration, connectivity add-on 1.0 has just been released. This release consists of an OData Adapter and extensions to the existing SFSF Adapter. Find out more in this blog by Finny Babu and reach out to him with any questions. July 2014
SAP HANA Cloud Integration Info Day Tour starting in Walldorf, Germany on July 30th SAP HANA Cloud Integration (HCI) is SAP’s strategic integration platform to integrate SAP Cloud applications with other systems, SAP and non-SAP, on premise or on the cloud. The new SAP HANA Cloud Integration editions: Standard and Professional allow organizations to leverage SAP HCI in arbitrary (i.e. any system to any system) integration scenarios. Join us in Walldorf on July 30th for a free info day to learn more and try HANA Cloud Integration hands-on! Follow Udo Paltzer to find out about other locations coming soon. July 2014
2014 Global SAP Process Integration Survey The yearly PI survey is open until August 19th and the 2014 special topics are BPM and Process Orchestration. The survey aims to collect information about the latest state of positioning, organization and use in companies using SAP PI or SAP Process Orchestration as an integration platform. You will find all details and survey access link in Holger Himmelmann’s blog. Participate and help beat the 2013 response record! July 2014
Get your free seat for the Intelligent Business Operations #IBO Tech Academy in the UK on 25/7 With this blog Tony Read invites you to join SAP and CompriseIT for a hands-on workshop on Intelligent Business Operations covering SAP Process Orchestration and SAP Operational Process Intelligence on HANA. The workshop is free for all SAP customers and prospects. Next opportunity in the UK is on 25/7. Seats are limited so mail Tony to book your seat today. June 2014
HANA Cloud Integration: Webinar Series In this blog Meghna Shishodiya announces a new HANA Cloud Integration (HCI) webinar series starting on May 29 with an overview, architecture and security aspects. Bookmark the blog for upcoming dates and join us to learn everything you wanted to know about hci and get your questions answered. May 2014
2014 Webcast Series: Tap into business agility with SAP Middleware solutions These informative public webcasts will showcase the advantages of SAP Middleware offerings that build on the trusted SAP NetWeaver technologies and leverage SAP HANA and the cloud. Grab your seat for the 1st webcast on April 30 for a talk on Intelligent Business Operations: Infuse Big Data insights into your processes in real-time. April 2014
SAP HANA Cloud Integration (HCI) Roadmap Webinar - 3 July How much do you know about SAP HCI? How it is leveraged for integration of SAP Cloud solutions, such as Cloud for Customers, SuccessFactors, Ariba etc. Join us for a roadmap session to learn the latest innovations and planning for HCI and get your questions answered. June 2014
SAP Process Orchestration Roadmap webinar - recording is now available! The webinar covers the latest innovations and planning for SAP Process Orchestration. You will also learn how Process Orchestration customers can benefit by enabling intelligent business operations ibo on the SAP HANA platform and how to take advantage of other Middleware offerings such as SAP Gateway and SAP HANA Cloud Integration hci. June 2014
12/6 - global webcast on Intelligent Business Operations: Infuse bigdata insights into your business processes in realtime This webcast will showcase real-life examples of how to work smarter by infusing #BigData insights into your processes, how to take corrective actions when or even before issues occur, anticipate what will happen using predictive analytics, gain real-time visibility into your end-to-end operations. Register for the live event on 12/6 and/ or to get the slides and replay. June 2014
Middleware Tech Academies at Saphila 2014 - 9&10 June If you plan to attend the conference do not miss the Intelligent Business Operations ibo hands-on workshops. On both 9 & 10 June you have the opportunity to learn more about the SAP IBO technology bundle and try the software. June 2014
SP11 of SAP Process Orchestration 7.31 is now available in Public Cloud! If you are using the SAP Process Orchestration trial offering on AWS, check this blog by Abhinag Palvadi. You can now access an instance with the latest available version - SP11 equivalent to SP6 of SAP Process Orchestration 7.4. May 2014
2014 Global SAP Process Integration Survey The yearly PI survey is now open and the 2014 special topics are BPM and Process Orchestration. The survey aims to collect information about the latest state of positioning, organization and use in companies using SAP PI or SAP Process Orchestration as an integration platform. You will find all details and survey access link in Holger Himmelmann’s blog. Participate and help beat the 2013 response record! May 2014
May 27th - join us for an SAP Process Orchestration Roadmap Webinar! The webinar will be delivered by SAP Product Management and will cover the latest innovations and planning for SAP Process Orchestration. You will also learn how Process Orchestration customers can benefit by enabling intelligent business operations ibo on the SAP HANA platform and how to take advantage of other Middleware offerings such as SAP Gateway and SAP HANA Cloud Integration hci. May 2014
SP3 of B2B Add-On and Secure Connectivity Add-On Released! In this blog Piyush Gakhar introduces all new features and improvements with the SP3 release, like secure PGP key storage, multiple directory support for SFTP adapter, SLA alerts, enhanced AS2 support, EDI content manager and more. April 2014
New roadmap for SAP Process Orchestration is now available In this blog Christian Loos announces that the former individual PI and BPM product roadmaps have been combined into one single Process Orchestration roadmap. Review all planned innovations and stay tuned for the upcoming roadmap webcast. April 2014
SuccessFactors (SFSF) Adapter for SAP NetWeaver Process Integration In this blog Finny Babu introduces the new “SFSF Adapter” released by SAP in February 2014. This adapter is now available as part of the release independent SAP NetWeaver Process Integration, connectivity add-on 1.0. Mar 2014
SAP Middleware newsletter – the newest addition to the SCN newsletters family! Free monthly insight into all innovations in ALM, Software Logistics, Software Defined Data Center solutions, Virtualization & Cloud Management, Architecture, Process Orchestration, Decision Service Management,Operational Process Intelligence, SAP HANA Cloud Integration, Big Process and Big Data and powerful technology bundles like Intelligent Business Operations powered by SAP HANA. Mar 2014
Survey: usage of cloud based business scenarios and their integration Your opinion counts! With this blog Udo Paltzer invites you to invest 5-10 mins of your time to support a scientific study carried out by Itelligence AG, SAP and the University of Paderborn. Take the short survey to help us understand your requirements, knowledge, acceptance, complexity and safety concerns around using "services from the cloud". Mar 2014
Explained: the new B2B Trading Partner Management functionality In this blog Shilpa Nair and Sarath Sasi provide a summary and links to all recent content on the new Trading Partner Management functionality shipped with SP2 of the SAP Process Orchestration B2B Add-on. Feb 2014
Improved monitoring of B2B scenarios running on SAP NetWeaver Process Orchestration - Part 1, Part 2 and Part 3 In this blog series, Alexander Bundschuh focuses on the new improvements which help enhance monitoring and tracking of your B2B business transactions and how SAP customers using the B2B add-on with SAP NetWeaver Process Integration & Orchestration may benefit from the latest enhancements shipped with SP02 of the B2B add-on. Also read about the new central monitor with SAP Solution Manager 1.0 SP10, the so called Message Flow Monitoring, which provides you with an end-to-end insight into the correct closure of A2A and B2B conversations. Feb 2014
Your SAP PI 7.31 Demo System on Amazon Web Services In this blog Arman Eker shares his “awesome experience” with the free Process Orchestration developer trial system which he used to learn how to build PI 7.31 on a single stack and to prepare a PI video course with over 3 hours of PI 7.31 content. Here is his Post configuration steps blog. Check it out and set up your own system - it’s easy and it’s fun! Feb 2014
IFG Survey: Central PI Monitoring with SAP Solution Manager Monitoring is a key challenge according to the global 2013 PI survey. To gain a better understanding of the situation and requirements, the IFG for PI and SAP have launched a follow-up survey with focus on central PI monitoring with SAP Solution Manager. Read all details in Holger Himmelmann’s blog and participate. Jan 2014
SAP Orchestration and Integration Solutions: TechEd 2013 Replays Missed TechEd? In this blog Gabriela Gahse highlights the available replays from TechEd Las Vegas 2013 and invites you to check the latest and greatest on B2B with Process Orchestration and HANA Cloud Integration hci , SAP Operational Process Intelligence opint, as well as Business Rules and Decision Service Management dsm with SAP. Jan 2014
Released: SAP NetWeaver Process Orchestration B2B Add-On SP2 In this blog Piyush Gakhar highlights the main enhancements with the latest SP like Trading Partner Management, new Messages Support for Tradacoms and EANCOM. Read this blog for a full overview of all new EDI and B2B features and enhancements. Dec 2013
Roadmap for SAP HANA Cloud Integration In this presentation Udo Paltzer provides an overview of the current capabilities and uses cases, as well as the roadmap for SAP HANA Cloud Integration (HCI). Do not miss the HCI library on SCN where you can find everything about hci in one place. Dec 2013
What's new in Process Orchestration 7.31 SP09 | 7.4 SP04 Key enhancements with SP09 are SAP NetWeaver Process Orchestration on HANA, co-deployment of SAP NetWeaver Process Orchestration and SAP OPInt opint  on the same HANA system and ValueHelp API. This blog by Mariana Mihaylova highlights all enhancements for customers running Business Process Management, Process Integration and Orchestration scenarios. Dec 2013
SAP NetWeaver Process Orchestration @ SAP TechEd Bangalore 2013 If you are lucky to be in Bangalore this week, this blog by Alexander Bundschuh may help you to build your Integration and Orchestration agenda. Join our experts to find the latest and greatest about BPM | PI | Process Orchestration, SAP HANA Cloud Integration, Business Rules, B2B and SAP Operational Process Intelligence powered by SAP HANA. Dec 2013
Results: Global SAP NetWeaver PI Survey 2013 - Part 1 of 2 and Part 2 of 2 These two blogs by Holger Himmelmann reveal the results of the 2013 global SAP NetWeaver PI survey. Thanks to everyone who participated and shared their feedback! Nov 2013
Integration: Extracts from TechEd LV 2013 Process Orchestration, HANA Cloud Integration, OData and more. Read the summary and key takeaways of SAP Mentor Shabarish Vijayakumar from the integration arena at this year’s teched_las_vegas. Nov 2013
SAP HANA Cloud Integration – Early Customer and Partner Project In this blog Udo Paltzer shares details about the opportunity to get hands-on experience with SAP HANA Cloud Integration. Join the program and become one of the early adopters! Nov 2013
SAP HANA Cloud Integration (HCI): Getting Started The first set of projects are underway in the HCI space, and now is a good time to get a closer look at the HCI world. Review this blog by Meghna Shishodiya and also the SAP HANA Cloud Integration: An Intro by Sujit Hemachandran. Nov 2013
SAP NetWeaver Process Orchestration @ SAP TechEd 2013 teched_amsterdam is in full swing now! Find out from Alexander Bundschuh which sessions are a must see in the areas of SAP NetWeaver BPM | PI | Process Orchestration, SAP HANA Cloud Integration, Business Rules, B2B and SAP Operational Process Intelligence powered by SAP HANA. Nov 2013
Protocol Decision Tree Between SAP ECC and SAP Process Orchestration In this blog, Agasthuri Doss introduces a decision tree to select the appropriate integration method between ECC and SAP Process Orchestration environments and between SAP Process Integration and client business systems.
What's new in SAP Process Orchestration 7.31 SP8/7.4 SP3 + Video In this blog Christian Loos shares all new enhancements for customers running Business Process Management, Process Integration and Orchestration scenarios. He has highlighted the most important new features in a short video.
Webcast: SP1 of B2B Add-on with SAP NetWeaver Process Orchestration> Overview and Roadmap This session is part of the ramp-up knowledge transfer program and will be presented by Piyush Gakhar from SAP Product Management. Read this blog for details and join us on Sep 30 or Oct 1!
Step by Step guide to update Advanced Adapter Engine System to Process Orchestration System Upgrade of AEX to Process Orchestration System is supported from 7.31 SP0 onwards. In this article Abhinag Palvadi explains in detail the steps of the upgrade.
SAP NetWeaver Process Orchestration – the best is yet to come! In this blog Volker Stiehl explains why you should opt in for Process Orchestration as your single Middleware platform from SAP. Also check in detail What is new in SP7 of SAP NetWeaver Process Orchestration 7.31 and see all new and continuous investments that make SAP’s Middleware platform best in class.
Global SAPNetWeaver PI Survey 2013: new record and 4 weeks to go! The survey will be closed on August 24th. Read more in Holger Himmelmann’s latest blog and do not miss to share your feedback. Thanks to everyone who already participated.
How AmerisourceBergen uses SAP’s Process Orchestration technologies In this blog Eduardo Chiocconi is sharing key takeaways from SAPPHIRE NOW 2013 and highlighting a great customer story. Watch the full session replay to learn how AmerisourceBergen,one of the world's largest pharmaceutical services companies is transforming their business with SAP NetWeaver Process Orchestration technologies.
SAP NetWeaver Process Orchestration Webcast Series 2013 Join us for a 5 webcast series to hear latest news about Process Management software from SAP: SAP NetWeaver Process Orchestration including B2B, SAP Operational Process Intelligence and SAP NetWeaver Decision Service Management. Please share with anyone who may be interested. We look forward to meeting you there.
Global Survey for SAP NetWeaver Process Integration 2013 The 2013 PI survey is now on and the focus this year is, not surprisingly, B2B and EDI. More details and survey access link in Holger Himmelmann’s blog. We look forward to your participation.
Try SAP NetWeaver Process Orchestration in Public Cloud! SAP is now giving you a free license to try SAP NetWeaver Process Orchestration in the cloud. Read this blog to see how you can get started.
Simple use-cases with SAP NetWeaver Process Orchestration B2B Add-on Check these new articles by Vikas Singh Rajpurohit providing use-cases and configuration options of OFTP; SFTP and PGP; AS2 and EDI Separator; Modules and X400 adapter available with the SAP B2B Add-on.
* What is new in SP6 of SAP NetWeaver Process Orchestration 7.31 Find out about the multiple new features and enhancements for Business Process Management, Process Integration and Orchestration scenarios. See how integration between SAP NetWeaver PI and BPM has been tightened further.
* Generate a Migration Report to estimate the migration effort from PI dual-stack to AEX Migration from PI dual-stack to single-stack (Java-only) can be a daunting task. In this article William Li presents a java client program (read only) to be used to browse through all the configurations in the Integration Directory and produce a report helpful for the preliminary assessment of the migration task in your landscape.

* cbs PI MeMo App for MobiIe Message Monitoring

* Released: SP1 of B2B Add-on and SFTP PGP with SAP NetWeaver Process Orchestration

* Upgrade options to the latest Process Integration or Process Orchestration

* IFG for PI: Subscription process for new PI Features in SAP Customer Connection Program

* Client Certificate based authentication while using ABAP Web Service for communication between ERP and SAP NET Weaver PI and The Myth of a Load Balancer - PI/Web Service Scenario

* Try SAP NetWeaver Process Orchestration in Public Cloud!

* Near Zero Downtime Management for SAP Netweaver Process Integration (nZDM/PI) available on Service Marketplace

* How to Load keys and certificates in SAP PI 7.3, SAP PO 7.3 EHP1 NWA's Key Storage

* Dynamic filename in mail receiver adapter made easier

* TechEd 2012: Process Orchestration session replays!

* SAP NetWeaver Process Orchestration technology in Healthcare

* Consolidated view on release notes for Process Integration and Orchestration

*SAP NetWeaver Process Orchestration PI | B2B | BPM | BRM on SCN - October

*SAP TechEd 2012 Sessions covering Process Orchestration with focus on Process Integration

* Results: 1) 2012 Global PI Survey  2) PI Requirements Prioritization

* SAP’s B2B Integration Strategy

* New SAP NetWeaver Process Orchestration RDS for EDI available now!

* Getting Started with SAP NetWeaver Process Orchestration

* IWAY adapters bundled within Process Integration / Orchestration

2012 Global SAP NetWeaver PI Survey

'File Content Conversion' OR 'MessageTransformBean' in sender channel can produce a 4 level deep XML, maximum.

If expected XML is more than 4 level deep.

Solution: - Develop FCC or MTB to read each line of input.

Map using an UDF, to split each line to corresponding field.

UDF code: - ‘Execution Type’ : ‘All Values of a Context’.

public void udf_Shopping(String[] eachLine, ResultList Customer, ResultList Name, ResultList ID, ResultList Order, ResultList OrderNumber, ResultList OrderNote, ResultList LineItem, ResultList LineNumber, ResultList Material, ResultList TaxLine, ResultList Type, ResultList Amount, Container container) throws StreamTransformationException{
int countCust = 0, countOrdr = 0, countLine = 0;
String values[];
for (String line : eachLine) {    values = line.split(",");    if (line.startsWith("Cust")) {        Customer.addValue("");        Name.addValue(values[1]);  Name.addContextChange();        ID.addValue(values[2]);    ID.addContextChange();        //Add context change form second Customer.        if (countCust > 0) {       Order.addContextChange();  }        countCust++;    } else if (line.startsWith("Ordr")) {        Order.addValue("");        OrderNumber.addValue(values[1]);  OrderNumber.addContextChange();        OrderNote.addValue(values[2]);    OrderNote.addContextChange();        //Add context change form second Order.        if (countOrdr > 0) {              LineItem.addContextChange();  }        countOrdr++;    } else if (line.startsWith("Line")) {        LineItem.addValue("");        LineNumber.addValue(values[1]);  LineNumber.addContextChange();        Material.addValue(values[2]);    Material.addContextChange();        //Add context change form second LineItem.        if (countLine > 0) {             TaxLine.addContextChange();        }        countLine++;    } else if (line.startsWith("Taxl")) {        TaxLine.addValue("");        Type.addValue(values[1]);    Type.addContextChange();        Amount.addValue(values[2]);  Amount.addContextChange();    }
}
}

Note:-

Comma(delimiter), 'Cust', 'Ordr', 'Line' and 'Taxl' arbitrary values are used to demonstrate the concept.

If some fields are optional in input, to avoid ArrayIndexOutOfBoundsException append delimiters.

for (String line : eachLine) {

    line = line + ",,"; //Append commas for optional fields.

    values = line.split(",");

  ....

}

If input is fixed length file (instead of delimited), use java String method substring(int beginIndex, int endIndex).

FYI. Other solution available on SCN.

DeepFCCBean - The better FCC at meeting your deep (structure) needs! (Part 2 - Flat File to Deep XML)

XI/PI: Convert Flat File to Deeply Nested XML Structures Using Only Graphical Mapping

They say "you can't have your cake and eat it."

They also say you cannot use dynamic configuration with multi-mapping.

But what if you could...

Introduction

In this article, I will demonstrate an approach to overcome one of the most common limitation of multi-mapping, which is the usage of dynamic configuration.

As each child message of a multi-mapping split shares the same dynamic configuration header during the mapping step, it is not possible to have different values (for a particular namespace attribute) assigned during mapping for each child message. One common scenario that requires such functionality is when a source message needs to be split into multiple files, and each target file needs to be dynamically named.

This approach is named the SOAP loopback approach, and has the following benefits:-

• Minimal coding required, i.e. no custom adapter module development
• Utilize standard features of PI for design and configuration
• No additional file system space required for placing temporary intermediate files

Approach

The key concepts to achieve this approach are:

• Intermediate structure with additional placeholder field to store value of dynamic configuration
• Loopback from receiver of iFlow1 to sender of iFlow 2 via SOAP adapter

Below are the flow diagrams of the two iFlow scenarios.

Following are the steps for the design and configuration of this approach.

Design

Data Type

Here is an example of the data type. The source contains a segment with multiple occurrences, while the target has only single occurrence (for each order.) An additional intermediate structure is defined that is exactly the same as the original target structure but with an additional placeholder field for dynamic configuration.

Structure	Definition
Source
Target
Intermediate Target

Service Interface

The sender and receiver interfaces are defined. Additionally, an abstract interface based on the intermediate structure is defined. This abstract interface will be used as the receiver of the first iFlow and also the sender of the second iFlow. The associated Message Types are also created.

Message Mapping

Message mappings are created for the two iFlow scenarios. Important thing to note here is that the first iFlow mapping is a 1-n multi-mapping, while the second iFlow mapping is a normal 1-1 mapping.

For the first mapping, the source is mapped to the intermediate structure. Additionally, the filename is dynamically constructed based on source fields, and mapped to the placeholder field in the intermediate structure.

For the second mapping, all the fields are mapped 1-1 from the intermediate structure to the final target structure. Additionally, the value in the placeholder field of the intermediate structure is used in a UDF mapping to set the dynamic configuration value. This can be mapped to a variable or to any arbitrary field with occurrence = 1.

Operation Mapping

The operation mappings are similar to the message mapping for the two iFlow scenarios.

Configuration

Systems and Communication Channel

For the configuration steps, we will need to configure 2 iFlows. Therefore there will be 2 sender systems and 2 receiver systems. For simplicity sake, we will reuse the same sender and receiver system, so in this example, it will only be BC1 and BC2. Alternatively, the receiver in the first iFlow and the sender of the second iFlow can be any arbitrary system or virtual component.

iFlow

We will configure the iFlow for the second scenario first, as the SOAP endpoint URL generated from the second scenario will be needed in the first scenario.

iFlow 2

The second iFlow will be used for processing the individual child messages to the final target. The main things to take note during configuration here are:

• Sender SOAP channel used for loopback is set to QOS Exactly Once to ensure async processing
• ASMA needs to be set for the receiver SFTP channel

Once iFlow 2 is completed, we need to get the SOAP endpoint for the iFlow. Follow the steps in Generating WSDL in IFLOW in SAP PO 7.4 to switch the perspective and Copy the endpoint.

iFlow 1

Now we are ready to configure the first iFlow that performs the multi-mapping to the intermediate structure. The main thing to note here is the configuration of the receiver SOAP channel for loopback.

Using the value of the endpoint from iFlow 2, enter this into the SOAP receiver channel's target URL - the hostname can be changed to localhost. Enter a messaging service user with sufficient credentials for the SOAP channel, typically it will be PIAPPL<SID>.

Testing Results

Here are the results from testing of the design.

A test message with two orders in the source XML is sent via the testing functionality

iFlow 1

In the payload after mapping of iFlow 1, we can see two child messages with the dynamically constructed filenames in the placeholder field.

The log also shows the child messages created from the split.

The log for the child message shows that the message is sent to the endpoint URL of the second iFlow.

iFlow 2

Next we can view the payload and logs of the child messages in the second iFlow. From the logs we can see that the child message has been transformed to the final target structure, and the filename is stored in the dynamic configuration attribute, which is finally used during processing by the SFTP receiver channel.

First child message

Second child message

Finally, we see in the target folder, the two files created for two child messages with dynamically constructed filenames.

Conclusion

Voila! There we go, "having our cake and eating it"

With this no-hassle approach, another common hurdle in the PI world can now be easily overcome.

'File Content Conversion' OR 'MessageTransformBean' in receiver channel can process 3 level deep XML, as input.

If input XML to receiver channel is more than 3 level deep.

Solution:- Map using an UDF.

Create an UDF with ‘Execution Type’ : ‘All Values of a Context’.

public void udf_DeepToFlat(String[] in, ResultList eachLine, ResultList element, Container container) throws StreamTransformationException{  //Identifiers are added. E.g: Cust, Ordr, Line, Taxl.  in[0] = in[0].replace("<Customer>", "<Tag>Cust</Tag>").replace("<Order>", "<Tag>Ordr</Tag>").replace("<LineItem>", "<Tag>Line</Tag>").replace("<TaxLine>", "<Tag>Taxl</Tag>");  //Match each element. E.g: <Name>AAA</Name>, <ID>123</ID>.  java.util.regex.Matcher m = java.util.regex.Pattern.compile("<(.*?)>[^<]*</\\1>").matcher(in[0]);  int count = 0;  eachLine.addValue("");  String elem;  while (m.find()) { //Loop over elements.  elem = m.group();  if (elem.startsWith("<Tag>") && count > 0) { //From second Tag  eachLine.addValue("");  element.addContextChange();  }  //Get text content of element. E.g: <Name>AAA</Name> = AAA.  element.addValue(elem.substring(elem.indexOf(">") + 1, elem.indexOf("</")));  count++;  }
}

Create receiver FCC or MTB.

Note:-

'Cust', 'Ordr', 'Line' and 'Taxl' arbitrary values are used to demonstrate the concept.

If some elements are represented in short form i.e., <Name></Name> is represented <Name/>. Add below code before element match, to replace short form with long form representation.

java.util.regex.Matcher m1 = java.util.regex.Pattern.compile("<[^<]*?/>").matcher(in[0]);

while (m1.find()) {

    String elementName = m1.group().substring(1, m1.group().length() - 2);

    in[0] = in[0].replace(m1.group(), "<" + elementName + "></" + elementName + ">");

}

FYI.

Flat file to deep XML - using an UDF

Copy XML Subtree

Choosing XML parser, 'Choosing Your Model' When to Use DOM .

Other solutions available on SCN.

File Conversion using 'Nodeception',

XSLT approach - Deep XML to Flat File, DeepFCCBean - Deep XML to Flat File

• SAP Cloud for Customer to on-premise SAP CRM / SAP ERP
• SuccessFactors to SAP HCM or to 3rd party recruiting systems as SHL or PeopleAnswers
• SAP eDocuments to government systems (Peru, Chile) for exchanging B2G e-invoices

These applications provide pre-packaged integration content for HCI, presented in an Integration Content Catalog and accessible over a web-based application. It eases the daily work of configurators, administrators and business analysts for exploring ready to run integration content as well as introductory information and demos. The integration content covers templates with prebuilt process integration, data integration flows and other integration artifacts that significantly reduce the implementation time, cost, and risk. These templates provide the bases for the easy adoption to specific business needs.
The design time is Eclipse based offering an Integration Designer perspective for integration developers to configure, deploy, administer, and monitor integration flows on detailed level.

Have a look at the SAP HANA Cloud Integration Landing Page - Public Integration Content Catalog to see how easy it is for customers and partners to find and understand what it is all about.

• SAP HANA Cloud Integration on SAP.com
• SAP HANA Cloud Integration documentation on SAP Help Portal
• SAP HANA Cloud Integration Product Availability Matrix
• SAP HANA Cloud Integration Knowledge Transfer on SAP Service Marketplace (S-user required)
• SAP Financial Services Network documentation on SAP Help Portal
• SAP Financial Services Network Connector documentation on SAP Help Portal
• SAP Financial Services Network Knowledge Transfer on SAP Service Marketplace (S-user required)

• SAP HANA Cloud Integration - Tools
• SAP HANA Cloud Integration - Public Integration Content Catalog

• SAP Financial Services Network (SAP FSN)
• SAP HANA Cloud Developer Center
• SAP Data Services
• SAP Process Orchestration

• SAP HCI: Standard and Professional Editions
• NEW! SAP HCI for Application-Development Partners
• SAP HCI Overview Presentation
• NEW! SAP HCI Customer Use Cases
• SAP HCI Overview Presentation
• SAP HCI Roadmap (S-User required)
• SAP HCI – Early Customer and Partner Project - sign up to try HCI. It's easy!
• NEW! SAP HCI Partner Content Certification
• SAP HANA Cloud Integration - a complementary offering to SAP Process Orchestration
• SAP HCI Architectural Overview
• 10 Developer Blogs on SAP HCI-PI
• Authentication and Connectivity Tests in SAP HCI
• Mapping Overview in SAP HCI
• Message Monitoring and Administration in SAP HCI
• SAP HCI for Data Services Tutorials

• E-Learning for Integration of SAP Cloud for Customer with SAP ERP and SAP CRM
• A recent information drop reveals many new details for HANA Cloud Integration
• A first attempt at using SAP HCI to integrate Workday and Salesforce

• SAPInsider hosted webinar on HCI: Core capabilities and SalesForce adapter development

Events

• Onsite Event: HANA Cloud Integration - Info Days - WW Tour, next event in the Netherlands March 26th
• Replays of SAP TechEd && d-code Las Vegas 2014

We use PI heavily in our landscape as middleware for connecting our SAP systems to external parties, one of our third party required us to make a secured FTP connection to transfer files from SAP Application server to their publicly listed (exposed on internet) FTP servers. As you can imagine, setting up interfaces through corporate firewalls can be a pain, we managed to setup this connection, get all the firewall rules sorted, tested with a secure FTP program and were able to connect from our network through the firewalls and transfer files securely over the internet.

The next step would be to setup our PI FTP adapter to transfer files through the same secure tunnel. We set it up using the following link: Configuring the Receiver FTP Adapter

The Adapter was configured to use FTPS connection, hence certificates were exchanged accordingly with the third-party, root and intermediate public root certificates were added to TrustedCAs keystore view, however transfer tests failed with the error: ftp 425 failed to establish connection a search on SCN results in nothing that particularly describes the issue and its potential resolution, besides a single thread where a user explains that you need to ask the firewall team to resolve this issue.

However, I was determined to understand why this issue occurs and how it can be prevented/resolved. A bit of research on internet and reading the FTP Specification RFC4217 explains the problem and provides the options to prevent/resolve it.

A typical FTPS works in the following way:

The protocol mandates that FTP client initiates two connections with the FTP Server.

1. Control Connection - For exchange of authentication

2. Data Connection - For transfer of files, this connection can be done in two modes, Active mode or passive mode

     A simplified view of Active Mode is depicted in the following diagram

     a) FTP Client initiates Control Connection

     b) On acceptance, FTP Client initiates Port command with port number

     c) FTP Server Acknowledges the port number and initiates a connection on the same.

     d) FTP Client acknowledges and initiates Data Transfer as necessary

     A simplified view of Passive Mode is depicted in the following diagram    

     a) FTP Client initiates Control Connection

     b) On acceptance, FTP Client initiates PASV command

     c) On acceptance of PASV mode, the Server responds back with IP Address/Port number combination that it will also listen to

     d) FTP Client has a choice, it can initiate a socket on original IP address used for Control Connection or the new IP sent by FTP server, the port number would be the           one returned by FTP server in step c, (some FTP clients like FILEZILLA and LFTP have flags/settings to override the new IP and use the original IP, however SAP PI           Adapter seems to be using the new IP)

     e) FTP Client initiates Data Transfer

     (A very detailed explanation is provided about Active & Passive modes is provided here )

3. The FTP 425 issue occurs when PI Adapter attempts to initiate a Data connection in passive mode, as the PI Adapter attempts to initiate Data connection on the IP address port numbers provided by FTP server in response to PASV command.

Resolution - There are couple of possibilities

Solution #1 - Fix the FTP server - there is a setting on FTP server that ensures that FTP server returns the correct IP address as a response to PASV command from FTP client. We asked our third party supplier to fix this setting and they did the needful, so we din't have to go implement the alternative.

Solution #2 - Allow PI Adapter to Use the new IP provided by FTP Server - This potential solution was researched by a member in the SAP Netweaver Application Server space, he recommends applying a SAP Note 1591971 ,  after implementing the note you get a new parameter (in older versions of PI ) strictHostnameChecking  which allows you to force PI to skip validation check and allow the data connection to be initiated, this solution is documented in the forum post  here.

Note: Solution #2 does not work if the FTP Server sends a garbage IP address which some implementation of FTP Servers do, as that IP won't exist and PI adapter will attempt to connect on this garbage IP and fail, this was our issue, our data connection failed all the time since our third party FTP Servers were dishing out garbage IP addresses to PASV responses.