Oracle Managed File Transfer (MFT) Handson - Part 4: Integration with SOA Suite

[Part 1] [Part 2] [Part 3] [Part 4] 

For users of both, MFT and SOA-Suite, both can be integrated. The SOA Suite can push data to MFT, which takes care of distributing those in the file system. And vice versa, the SOA Suite can be a target for a MFT transaction.
For this example, both MFT and SOA Suite should be installed. Basic knowledge of both products is required.

Starting with the Design tab of the MFT Console. A new source of type SOA is created here. Important: don't chose the name soa as depicted above, as this will lead to problems later. So chose a different name like service-source.

Next, a Transfer needs to be created, using the newly created service-source as Source. Create any File Target as target for this transformation, eg. the file-target from chapter 3.
Then click Save and Deploy, so both artifacts are deployed.

Next, open JDeveloper and create a new SOA-Project.

In the composite, drop a MFT adapter to the right side.

Accept the defaults

Also keep the defaults here

In this step, an existing application server connection can be chosen or a new one be created. The MFT server in that domain will be found automatically, click on Test MFT to check the connection.

The serivce-source will be found automaticall, end this wizard with Finish.

To have something to communicate with the MFT adapter, drop a BPEL process into the composite. In the dialog shown above, switch it to Synchronous BPEL Process and leave the rest.

Connect the MFT adapter with the BPEL process, then open the process with a double click.

Place an Invoke activity in the middle and connect it with the MFT partner link.

Create an input- and output-variable by clicking on each green plus sign (+). Then close the dialog with OK.

An assign activity is needed, place it before the Invoke1 and open it via double click.

Right click on ns1:InlinePayload to assign an expression.

Create a greeting, eg. one matching the region you live in. In my case:
concat("Moin, Moin ", $inputVariable.payload/client:input)

Do the same for ns1:TargetFilename. As this is the name for the target file, better avoid special characters which could cause troubles on some file systems. Eg.:
concat("Hello-",$inputVariable.payload/client:input)

The composite is ready now and need to be deployed (right click on [project name]|Deploy).
Next, open the Enterprise Manager and navigate to that project. Go to the Test Web Service page, enter a test string and start the test.

If everything was OK, the Flow Trace can be launched.

The flow trace will contain a MFT step, click on it.

This will lead to the MFT Console where the Transfer will be displayed.

In the file system, the resulting will be created. Open it and check the contents.

That's the end of the MFT tutorial

[Part 1] [Part 2] [Part 3] [Teil 4]

Oracle Managed Files handson tutorial part 3: Custom Callouts

[Part 1] [Part 2] [Part 3] [Part 4]

The built-in functionality from MFT can be extended with custom code via Custom Callouts in Java.
Three things are necessary: the Java code itself, an xml file which describes what MFT should do with this code and a WST call to register that code to MFT.
The complete documentation can be found on OTN, this tutorial should demonstrate the creation and integration of Custom Callouts with a very simple example.
In this example, a Custom Callout will be developed, which replaces the file name with the actual system time.
For stepping through this tutorial, it should be known, how to create and compile a Java class and Part 1 of this tutorial should be completed.
Here is the Java code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

package com.oracle.callout.sample; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.util.Map; import java.util.Date; import oracle.tip.mft.engine.processsor.plugin.PluginContext; import oracle.tip.mft.engine.processsor.plugin.PluginOutput; import oracle.tip.mft.engine.processsor.plugin.PreCalloutPlugin; public class FilenameCallout implements PreCalloutPlugin { @Override public boolean isPayloadChangeRequired(PluginContext context, Map<String, String> calloutParams) { return false; } @Override public PluginOutput process(PluginContext context, InputStream input, OutputStream out, Map<String, String> calloutParams) { String type = calloutParams.get("Type"); return null; } @Override public PluginOutput process(PluginContext context, InputStream input, Map<String, String> calloutParams) { PluginOutput pOutput = new PluginOutput(); pOutput.setNewFileName("MFT " + new Date() ); return pOutput; } }

All imports, except java.util.Date are required by every Custom Callout.
The method isPayloadChangeRequired() gives MFT a hint, whether the payload will be changed or not - as the name implies. Depending on that, either of the following methods will be called. As in this example only the file name, but not the content, will be changed, isPayloadChangeRequired() returns false. The first process method is not needed and just returns null. Only the second process method will be called during runtime and does something. It changes the file name.

[oracle@oel6ab src]$ javac -classpath "/home/oracle/Oracle/Middleware/soa12103/mft/modules/oracle.mft_12.1.3.0/core-12.1.1.0.jar" com/oracle/callout/sample/FilenameCallout.java 
[oracle@oel6ab src]$ jar cf FilenameCallout.jar com/oracle/callout/sample/FilenameCallout.class 
[oracle@oel6ab src]$ ll
total 12
drwxr-xr-x. 3 oracle oinstall 4096 Feb  6 15:23 com
-rw-r--r--. 1 oracle oinstall 1187 Feb  9 16:42 FilenameCallout.jar

The class has to be compiles as shown above, the path names need to be adapted to your environment. As a result, we get a FilenameCallout.jar file and we are done with the Java part.
In the next step, an XML file will be created, so MFT knows what to do with this JAR file.

1 2 3 4 5 6 7 8 9 10 11 12 13

<?xml version="1.0" encoding="UTF-8"?> <mft:Callouts xmlns:mft="http://xmlns.oracle.com/mft" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/mft callout.xsd "> <mft:Callout description="Filename conversion" helpText="File name conversion" groupName="Source-pre,Target-pre,Target-post" timeout="300" implementationClass="com.oracle.callout.sample.FilenameCallout" libraryName="FilenameCallout.jar" name="Filename conversion"> </mft:Callout> </mft:Callouts>

The XML file is very straight forward. Important is the name, which has to be unique in a MFT instance. The attributes libraryName and implementationClass are self explanatory (hopefully ;-).

Both files needed to be copied into the mft directory of the WLS domain. If the subfolder callouts doesn't already exist, it needs to be created manually. Then FilenameCallout.jar and FilenameCallout.xml have to be copied into the callouts-folder.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

CLASSPATH=:/home/oracle/Oracle/Middleware/soa12103/mft/modules/oracle.mft_12.1.3.0/core-12.1.1.0.jar Initializing WebLogic Scripting Tool (WLST) ... Welcome to WebLogic Server Administration Scripting Shell Type help() for help on available commands wls:/offline> connect("weblogic","welcome1","t3://localhost:7003") Connecting to t3://localhost:7003 with userid weblogic ... Successfully connected to managed Server "mft_server1" that belongs to domain "soa_domain". Warning: An insecure protocol was used to connect to the server. To ensure on-the-wire security, the SSL port or Admin port should be used instead. wls:/soa_domain/serverConfig> crtCalls("/home/oracle/Oracle/Middleware/soa12103/user_projects/domains/soa_domain/mft/callouts/FilenameCallout.xml") Callout Filename conversion created. wls:/soa_domain/serverConfig> listCallouts() Callouts ----------- [Name:Filename conversion, Library:FilenameCallout.jar, Impl Class:com.oracle.callout.sample.FilenameCallout, Description:Filename conversion, Group:Source-pre,Target-pre,Target-post] wls:/soa_domain/serverConfig>

To inform MFT about the new callout, WLST will be used. For using the MFT specific commandos, the wlst.sh has to be started from the MFT directory under $MW_HOME/mft/common/bin. The first line of the output with the CLASSPATH shows, if the correct script has been used.
As always with WLST, first connect to the server. Then the callout will be registered by a crtCalls(...) call with the XML file as a parameter.
All registered callouts can be shown via listCallouts() and they can be deleted by deleteCallout(...).

Again, two directories for source and target are needed. This could be the ones from the first part of this tutorial, or just create new ones.

In the MFT Console, both directories are configured as source and target, as shown in part 1.
Then, a Transfer will be configured using both directories.

After clicking at <add pre-processing actions>, the new choice 'Filename conversion' can be added via the Add to List Button.

The result should look as shown above. Click on Save, then finish with Deploy.

After a short period of time, the successfull Transfer should be shown on the dashboard.

Also, the transferred file with the changed name should appear in the file system. That's it for the MFT Custom Callouts handson tutorial. The next part will explain the integration with the Oracle SOA Suite.

--> Next: Integration with SOA Suite

[Part 1] [Part 2] [Teil 3] [Part 4]

Oracle Managed Files handson part 2: ftp transfer

[Part 1] [Part 2] [Part 3] [Part 4] 

In the first part, only local files have been transferred. In this second part it will be shown how to do the same with files via ftp.
As Oracle Managed File Transfer (MFT) is an application running on Weblogic Server (WLS), MFT is also utilizing the security mechanisms of WLS. So the access rights first have to be configured in the Weblogic-Console (http://HOSTNAME:PORT/console, e.g. http://localhost:7001/console).

Starting in the Weblogic Console, click on Security Realms in the left pane, then on myrealm in the right pane.

Under Users and Groups, create a new user by clicking on New.

Give him a name and a password, finish with OK.

Now the user is known to WLS, but has no rights to use MFT. Switch to the MFT Console. On the Administration tab, under Embedded Server in the tree view on the left, click on User Access.
For searching the user, enter the first three letters 'ftp' into the search box and choose the complete name from the list box. Then click on the right arrow button to select that user.
The /ftptransfer folder exists by default. Check the boxes for write- and list-permissions and click on Save.

By choosing Ports in the left tree panel, the ports of the embedded ftp server can be shown.

With these settings, a first connection test to the internal ftp-server could be done. Any ftp client would be fine, eg. via the command line: ftp localhost 7021. For better convenience, start ftp from the directory where the zipped test file (eg. hjkl.zip from part 1) resides.
Leave the ftp session open, as it will be needed again in a few steps.

For transfering the files from the internal ftp server, another source is needed. This one will also be created under Design/Sources. This time, choose the type FTP Embedded and Browse to the /ftptransfer directory. This is already available, as MFT creates one for every ftp user automatically.

For the newly created source, a new Transport needs to be created.

For the source, choose the new ftp-sorce. The targets will be the same as in part 1. The only-txt target gets an additional decompress as post-processing action. Save and deploy to finish this step.

Use the formerly opened ftp session to test the ftp-transfer. use the commands bin (for binary, because of the zip format), then put hjkl.zip and ls to check if the file arrived correctly.

After a moment, both files should be listed at the Monitoring Dashboard in the bottom right area.

The result can also be checked at the file sytem. When the files arrived correctly, this part of the tutorial for the built-in ftp-server is done.

--> Oracle Managed File Transfer (MFT) handson - Teil 3: Custom Callouts in Java

[Part 1] [Part 2] [Part 3] [Part 4] 

Oracle Managed Files handson part 1: Basics

[Part 1] [Part 2] [Part 3] [Part 4]

This tutorial demonstrates the first steps with Oracle Managed File Transfer (MFT). Prerequisites are an installed and running system, the installation guide can be found under http://www.oracle.com/technetwork/middleware/mft/documentation/index.html.
This tutorial has been done using Oracle Linux 6. For other systems, the OS commands have to be adapted accordingly.

Starting with the MFT-Console. It is located at [servername]:[MFT-Server Port]/mftconsole, in this case http://localhost:7003/mftconsole. The UI is grouped into three parts: Design, Monitoring and Administration. We start at the design tab where the needed artifacts can be defined.

We need three folders in the file system first: File-Source, only-txt and only-zip. To get some payload, we need two text files with meaningful names (e.g. touch asdf.txt). One text file will be zipped (zip hjkl.txt). You should end up with files and folders like the ones in the screenshot above.

These directories need to be configured in MFT. Clicking on Sources in the left area opens the Create Source Dialog. It needs a name, the type File and the path to the folder created above. To finish click on Create.

Do the same by clicking on Targets and configure both targets.

In the left tree, the source and the two targets should be shown. Clicking on one of them diplays detail information on the right side.

Now the first Transfer can be defined by clicking on Transfers. There are no additional parameters at this stage, click on Create to finish.

In the transfers detail view, the source can be added by clicking on the <add source> link or by drag & drop from the tree view.

Then the targets can be assigned. The <add target> link opens the Select targets dialog. Multiple targets can be chosen here, so move all targets to the right side and click OK.

To make sure, that only .txt files will be transfered, enter an according (*.txt) wildcard below ‚Content Filters'. For more complex filters, regular expressions also could be defined.

The only-zip target should only receive zipped files. So a compress actions needs to be added via <add pre-processing actions>.

The result should look as shown above. One file source with a filter on a *.txt wildcard. Both targets,
only-txt and only-zip, are registered. The only-zip target has an additional Compress pre-processing action.
In this state, the tranfer is defined, but not running on the server. To achieve this, click on the links ‚Save' and then‚Deploy' in the top right corner. Close the dialog with Deploy and OK. Now the link should read Deployed as shown above.

On the monitoring page, click on Deployments in the left area. Not only the file-txt-transfer has been deployed, but also the source and both targets. As all three are needed to run the transfer, MFT automatically deployed all artifacts that the transfer relies on.

For testing, copy one file (e.g. asdf.txt) into the file-source folder.

The results can be seen on the Dashboard in the Monitoring section. The Active Deliveries area on the bottom left can be switched to auto refresh. After a short period of time, both successfull transfers should be shown here.

In the file system, it can be seen that the file has been copied to the only-txt folder whereas it has additionally been compressed for the only-zip folder.

For practicing, a second transfer could be set up, doing just the opposite of the file-txt-transfer. Zip-files should be copied into only-zip and txt-files should be compressed first, then transfered to only-txt.

--> Part 2: ftp-Transfer

[Part 1] [Part 2] [Part 3] [Part 4]