Streaming VFS File Transfer using WSO2 ESB

Brief idea for VFS

VFS means Virtual File System. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference.

Brief Intro to WSO2 ESB VFS transport with streaming

Through this we can pick up a file from a directory and process it within the ESB. You can get more about WSO2 ESB VFS transport from here. But when we using this VFS configurations for transferring large files (Greater that 500 MB), we get an Out Of Memory exceptions. In this time we can use streaming facility. To enable streaming, we have to add message builder "org.apache.axis2.format.BinaryBuilder" for this. Apart from that  we need to include the property "ClientApiNonBlocking" in the proxy configuration. To use vfs transport facility from WSO2 ESB, there are some steps you have to follow.

1. Enable the VFS transport by editing the <ESB_HOME>/repository/conf/axis2/axis2.xml file and uncomment the VFS listener and the VFS sender as follows:
   

   <transportreceiver name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportListener"/>
   ...
   <transportSender name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportSender"/>

2.  Add message builder and formatter in the <ESB_HOME>/repository/conf/axis2/axis2.xml file.
   
   • Add <messageFormatter contentType="application/file"
             class="org.wso2.carbon.relay.ExpandingMessageFormatter"/> under the message formatters.
   • Add  <messageBuilder contentType="application/file"
             class="org.apache.axis2.format.BinaryBuilder"/> under the message builders.
      
3. You have to add a proxy like below.
   
   <?xml version="1.0" encoding="UTF-8"?>
   <proxy xmlns="http://ws.apache.org/ns/synapse"
          name="FileProxy"
          transports="vfs"
          startOnLoad="true"
          trace="disable">
      <description/>
      <target>
         <inSequence>
            <log level="custom">
               <property name="FileProxy" value="Processing file"/>
            </log>
            <property name="OUT_ONLY" value="true"/>
            <property name="ClientApiNonBlocking"
                      value="true"
                      scope="axis2"
                      action="remove"/>
            <send>
               <endpoint name="FileEpr">
                  <address uri="vfs:file:///home/Desktop/testESB/out"/>
               </endpoint>
            </send>
         </inSequence>
      </target>
      <parameter name="transport.vfs.Streaming">true</parameter>
      <parameter name="transport.PollInterval">20</parameter>   <parametername="transport.vfs.ActionAfterProcess">MOVE</parameter>
      <parameter name="transport.vfs.FileURI">file:///home/Desktop/testESB/in</parameter>
      <parameter name="transport.vfs.MoveAfterProcess">file:///home/Desktop/testESB/original</parameter>
      <parameter name="transport.vfs.MoveAfterFailure">file:////home/Desktop/testESB/failure</parameter>
      <parameter name="transport.vfs.Append">true</parameter>
      <parameter name="transport.vfs.Locking">enable</parameter>
      <parameter name="transport.vfs.FileNamePattern">.*.zip|.*.txt</parameter>
      <parameter name="transport.vfs.ContentType">application/file</parameter>
      <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
   </proxy>
   
   Here we specify ActionAfterProcess for doing some process after that file send to the specific location. We can specify MOVE/DELETE for this.
   
   We can specify source file to send from transport.vfs.FileURI. Destination of the file where we want to send that file can be specified in <address uri="vfs:file:///home/Desktop/testESB/out"/>
   
   This can be done through file inbound/ file connector. Next blog will cover about it. Until that bye. :)