WSO2 Developers' Guide

Chapter 1. Getting Started with SOA and WSO2

We will try to introduce this book with a simple, brief, and concise discussion of SOA, talking about its origin and what it means. We will discuss the facts or problems that large companies with a huge IT system had to face, and that finally gave rise to the SOA approach.

Once we know what we are talking about, we will introduce the WSO2 technology and describe the role it plays in SOA, which will be followed by the installation and configuration of the WSO2 products we will use.

So, in this chapter, we will deal with the following topics:

A basic knowledge of SOA
Download WSO2 Enterprise Integrator
WSO2 Update Manager (WUM)
Update with the official Patches
Set up WSO2 Enterprise Integrator
Start Enterprise Integrator

Service-oriented architecture (SOA) is a style, an approach to design software in a different way from the standard. SOA is not a technology; it is a paradigm, a design style.

There comes a time when a company grows and grows, which means that its IT system also becomes bigger and bigger, fetching a huge amount of data that it has to share with other companies. This typical data may be, for example, any of the following:

Sales data
Employees data
Customer data
Business information

In this environment, each information need of the company's applications is satisfied by a direct link to the system that owns the required information. So, when a company becomes a large corporation, with many departments and complex business logic, the IT system becomes a spaghetti dish:

Spaghetti dish

The spaghetti dish is a comparison widely used to describe how complex the integration links between applications may become in this large corporation. In this comparison, each spaghetti represents the link between two applications in order to share any kind of information.

Thus, when the number of applications needed for our business rises, the amount of information shared is larger as well. So, if we draw the map that represents all the links between the whole set of applications, the image will be quite similar to a spaghetti dish. Take a look at the following diagram:

Spaghetti integrations by Oracle (https://image.slidesharecdn.com/2012-09-20-aspire-oraclesoawebinar-finalversion-160109031240/95/maximizing-oracle-apps-capabilities-using-oracle-soa-7- 638.jpg?cb=1452309418)

The preceding diagram represents an environment that is closed, monolithic, and inefficient, with the following features:

The architecture is split into blocks divided by business areas.
Each area is close to the rest of the areas, so interaction between them is quite difficult.
These isolated blocks are hard to maintain.
Each block was managed by just one provider, which knew that business area deeply.
It is difficult for the company to change the provider that manages each business area due to the risk involved.
The company cannot protect itself against the abuses of the provider. The provider may commit many abuses, such as raising the provided service fare, violating service level agreement (SLA), breaching the schedule, and many others we can imagine. In these situations, the company lacks instruments to fight them because if the business area managed by the provider stops working, the impact on the company profits is much larger than when assuming that the provider abuses.
The provider has a deeper knowledge of the customer business than the customer itself.
The maintenance cost is high due to the complexity of the network for many reasons; consider the following example:
- It is difficult to perform impact analysis when a new functionality is needed, which means high costs and a long time to evaluate any fix, and higher costs of each fix in turn.
The complex interconnection network is difficult to know in depth.
Finding the cause of a failure or malfunction may become quite a task.
When a system is down, most of the others may be down as well.
A business process is used to involve different databases and applications. Thus, when a user has to run a business process in the company, he needs to use different applications, access different networks, and log in with different credentials in each one; this makes the business quite inefficient, making simple tasks take too much time.
When a system in your puzzle uses an obsolete technology, which is quite common with legacy systems, you will always be tied to it and to the incompatibility issues with brand new technologies, for instance.
Managing a fine-grained security policy that manages who has access to each piece of data is simply an uthopy.

Something must to be done to face all these problems and SOA is the one to put this in order. SOA is the final approach after the previous attempt to try to tidy up this chaos.

We can take a look at the SOA origin in the white paper, The 25-year history of SOA, by Erik Townsend (http://www.eriktownsend.com/white-papers/technology). It is quite an interesting read, where Erik establishes the origin of the manufacturing industry. I agree to that idea, and it is easy to see how the improvements in the manufacturing industry, or other industries, are applied to the IT world; take these examples:

The hardware bus in motherboards has been used for decades, and now we can also find the software bus, Enterprise Service Bus (ESB) in a company. The hardware bus connects hardware devices such as microprocessors, memory, or hard drives; the software bus connects applications.
A hardware router in a network routes small fragments of data between different nets to lead these packets to the destination net. The message router software, which implements the message router enterprise integration pattern, routes data objects between applications.
We create software factories to develop software using the same paradigm as a manufacturing industry.
Lean IT is a trending topic nowadays. It tries, roughly speaking, to optimize the IT processes by removing the muda (Japanese word meaning wastefulness, uselessness). It is based on the benefits of the lean manufacturing applied by Toyota in the '70s, after the oil crisis, which led it to the top position in the car manufacturing industry.
We find an analogy between what object-oriented language means to programming and what SOA represents to system integrations as well.
We can also find analogies between ITIL V3 and SOA. The way ITIL V3 manages the company services can be applied to managing the SOA services at many points. ITIL V3 deals with the services that a company offers and how to manage them, and SOA deals with the service that a company offers to expose data from one system to the rest of them. Both the conceptions are quite similar if we think of the ITIL V3 company as the IT department and of the company's service as the SOA service.

There is another quite interesting read--Note on Distributed Computing from Sun Microsystem Laboratories published in 1994. In this reading, four members of Sun Microsystems discuss the problems that a company faces when it expands, and the system that made up the IT core of the company and its need to share information. You can find this reading at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.48.7969&rep=rep1&type=pdf.

In the early '90s, when companies were starting to computerize, they needed to share information from one system to another, which was not an easy task at all. There was a discussion on how to handle the local and remote information as well as which technology to use to share that information.

The Network File System (NFS), by IBM was a good attempt to share that information, but there was still a lot of work left to do. After NFS, other approaches came, such as CORBA and Microsoft DCOM, but they still keep the dependencies between the whole set of applications connected. Refer to the following diagram:

The SOA approach versus CORBA and DCOM

Finally, with the SOA approach, by the end of the '90s, independent applications where able to share their data while avoiding dependencies. This data interchange is done using services. An SOA service is a data interchange need between different systems that accomplishes some rules. These rules are the so-called SOA principles that we will explain as we move on.

SOA principles

The SOA principles are the rules that we always have to keep in mind when taking any kinds of decisions in an SOA organization, such as the following:

Analyzing proposals for services
Deciding whether to add a new functionality to a service or to split it into two services
Solving performance issues
Designing new services

There is no industry agreement about the SOA principles, and some of them publish their own principles. Now, we will go through the principles that will help us in understanding its importance:

Service Standardization: Services must comply with communication and design agreements defined for the catalog they belong to. These include both high-level specifications and low-level details, such as those mentioned here:
- Service name
- Functional details
- Input data
- Output data
- Protocols
- Security
Service loose coupling: Services in the catalog must be independent of each other. The only thing a service should know about the rest of the services in the catalog is that they exist. The way to achieve this is by defining service contracts so that when a service needs to use another one, it has to just use that service contract.
Service abstraction: The service should be a black box just defined by its contracts. The contract specifies the input and output parameters with no information about how the process is performed at all. This reduces the coupling with other services to a minimum.
Service reusability: This is the most important principle and means that services must be conceived to be reused by the maximum number of consumers. The service must be reused in any context and by any consumer, not only by the application that originated the need for the service. Other applications in the company must be able to consume that service and even other systems outside the company in case the service is published, for example, for the citizenship. To achieve this, obviously, the service must be independent of any technology and must not be coupled to a specific business process. If we have a service working in a context, and it is needed to serve in a wider context, the right choice is to modify the service for it to be able to be consumed in both the contexts.
Service autonomy: A service must have a high degree of control over the runtime environment and over the logic it represents. The more control a service has over the underlying resources, the less dependencies it has and the more predictable and reliable it is. Resources may be hardware resources or software resources, for example, the network is a hardware resource, and a database table is a software resource. It would be ideal to have a service with exclusive ownership over the resources, but with an equilibrated amount of control that allows it to minimize the dependencies on shared resources.
Service statelessness: Services must have no state, that is, a service does not retain information about the data processed. All the data needed comes from the input parameters every time it is consumed. The information needed during the process dies when the process ends. Managing the whole amount of state information will put its availability in serious trouble.
Service discovery: With a goal to maximize the reutilization, services must be able to be discovered. Everyone should know the service list and their detailed information. To achieve that aim, services will have metadata to describe them, which will be stored in a repository or catalog. This metadata information must be accessed easily and automatically (programmatically) using, for example, Universal Description, Discovery, and Integration (UDDI). Thus, we avoid building or applying for a new service when we already have a service, or several ones, providing that information by composition.
Service composability: Services with more complex requirements must use other existing services to achieve that aim instead of implementing the same logic that is already available in other services.
Service granularity: Services must offer a relevant piece of business. The functionality of the service must not be so simple that the output of the service always needs to be complemented with another service's functionality. Likewise, the functionality of the service must not be so complex that none of the services in the company uses the whole set of information returned by the service.
Service normalization: Like in other areas such as database design, services must be decomposed, avoiding redundant logic. This principle may be omitted in some cases due to, for example, performance issues, where the priority is a quick response for the business.
Vendor independent: As we discussed earlier, services must not be attached to any technology. The service definition must be technology independent, and any vendor-specific feature must not affect the design of the service.

SOA organization

Once we understand what is SOA and its principles, the next question is How to apply SOA to a standard organization?

Well, what SOA tries to say to a big organization that has a large number of systems with a high level of integration (remember the spaghetti dish), is, Stop turning a blind eye! That spaghetti dish is also your business!

What we mean by this sentence is that companies add new systems on demand as their business grows. In this scenario, the typical way to proceed is to add new systems and configure all the integrations needed with the current systems of the company. This way, little by little, the number of systems grows until they achieve the spaghetti dish.

What is wrong with this scenario is the thinking that interconnections do not play any role in the company business, but only play the simple role of an information socket. At this point, all the interconnections of the company have enough dimension and impact on the business that they should be treated like another part of the business.

SOA tells the company that all that system connections is a business that has to be managed. The company must go deep inside the spaghetti dish to understand the business behind it. Then, that business must be digested to turn a large number of wires between applications into business services. The beautiful thing about this is that after taking a look at these system connections and discovering the business behind them, the business that we find is the essential business of the company, which defines its identity and what the company is and does.

Once we realize that, let's see how we can bring SOA to our company.

The very first step to do this is service prospection. In service prospection, we look deep inside the connection network to look for the business behind it and turn it into business services. We need to know which systems produce information and which ones consume information. This analysis can be made with the following two strategies:

Bottom-up: This approach starts analyzing the integration from the point-to-point connection (bottom) existing between the systems, building the integration map. These connections are linked together to result in another one in a higher level of abstraction, which will be the very first candidate services. We iterate several times over these connections/services, building high-level services according to the SOA principles until we achieve the business services (up) of the company.
Top-down: This approach is the opposite of the bottom-up approach. It starts designing high-level business services (top) according to the business expert. Starting from that point, we iterate over them, increasing the level of detail in each iteration and splitting them according to the SOA principles. We stop when no new services result from the previous iteration (down).

Take a look at the following diagram:

Neither of the strategies are perfect and each one has its pros and cons. The top-down approach is theoretical and lacks the real-world vision, while the bottom-up part is data or real world but does not consider the business theory. Finally, each strategy has its advantages and disadvantages; so, the best practice is to apply both the approaches and stop where they both meet.

For example, we start by defining the high-level business service (top) and identifying the point-to-point connection (bottom); these are the ideally desired business services for the business. We follow this by adding detailed information to top-level services and splitting the services into other services that compose them. On the bottom level, we link or compose the services to generate higher-level ones; these are real-world services that currently form the business. Both the processes continue iterating until they meet each other at a point, where both the processes merge, obtaining the final set of detailed business services. These are the candidate services that have the ideal or desired vision of the business, but use the real-world services that are part of the current business.

These groups of business services will be the SOA catalog. The catalog is a registry where we can find, at the very least, the following information:

The available services
The services in progress
Relationships between services
Detailed information about each service

Once we have the service catalog of our future SOA organization, the following steps do not differ much from other typical IT projects:

SOA life cycle

We will start with the analysis phase where we determine what to do in each service. In the first iteration, part of this analysis has already been done during the service prospection.

This is followed by the design phase, where we define how to do it in detail. Then, services are implemented according to that detailed design and tested to move them to the production environment of our company. Finally, we are in the maintenance phase, where we control and monitor the business.

There is a new phase that is called SOA Governance, and it is present in all the services. The aim of SOA Governance is to define policies, standards, principles, and processes to uphold the SOA principles; in other words, its role is to ensure that SOA benefits the organization.

Once again, there's nothing new here; governance is a term from politics that is applied to the IT world, but instead of controlling and managing a country, it controls and manages the service catalog of the SOA organization:

Achieving an SOA organization

The last element needed to build our SOA organization is the component that enables the consumer to discover which services are available in the organization, their functionalities, and how they can consume them; this component is known as the Universal Description, Discovery, and Integration registry:

UDDI

So now, we are ready to turn our standard organization into an SOA organization.

Technology for SOA

As we discussed earlier, SOA is an approach and tells us nothing about which technology to use to implement it. Each organization is free to choose the desired technology as long as it ensures the SOA principles.

The most common implementation is based on the following standards:

XML: This is the extensible markup language we all know. It is the typical language used to model the organization's business entities; another option quite popular nowadays is JSON.
HTTP(s): This is the protocol used to exchange information between systems.
SOAP: Simple Object Access Protocol (SOAP), is based on XML and defines the structure that will be used to exchange information between systems. A SOAP message contains the following elements:

SOAP message

Envelope: The top-level structure that includes everything else
Header: Extra information related to security, how the message must be processed, and the quality of service
Body: The data it sends to the producer, or the response that it sends to the consumer
Fault: Information about the errors produced while processing the message
WSDL: Web Services Description Language (WSDL), and it is the language for defining the contract of the service. In other words, it defines the input and output of the service black box without any attached vendor technology.

Putting all these standards together, we get the web service technology. Web service is a process that aims to interchange information with other systems. This information exchange is defined by a contact, which is built using the WSDL.

Web service technology will be the instrument that is finally used to build the business services we have been discussing in this chapter. It is important to note that at this point, we can fully define a business service using this technology, without tying it to any specific vendor or technology, such us Java, .Net, or Oracle.

At this time, we can finally introduce the central element of the SOA IT infrastructure ESB. The name is an analogy to the hardware bus we can find in a motherboard, which enables the communication between all components. In this case, as we discussed earlier in the chapter, it communicates software components transporting messages of data instead of communicating hardware components transporting bits of data.

The ESB is a middleware where the SOA services are deployed and published and are ready to be consumed by other systems that may be within the company, in another company, in another country, or even in another continent.

In the following diagrams, we can see an approach of the SOA infrastructure of our company:

Spaghetti dish organization

In contrast with the spaghetti dish, we can see the following:

SOA organization

The ESB allows us to accomplish the SOA principles with the following features:

Deploying, managing, and controlling the SOA services
Synchronous communication between systems, where the consumer is waiting for the response in the same channel used to send the request
Asynchronous communication between systems, where the consumers do not wait for the response in the same channel used to send the request
Fire-and-forget communication, where the consumers do not need a response
Services orchestration, which wraps business logic that involves several services into a single service
Complex business services
Security to ensure authentication and authorization of the consumers
Quality of the service, such as message signing and encryption, reliable messaging, and throttling
Applying enterprise integration pattern. You can find out more about it at: http://www.enterpriseintegrationpatterns.com.
Standardized platform
Protocol conversion
Technology isolation

There are many vendors offering different implementations of the ESB in the market with different pros and cons, and with many different kinds of licenses. The ESB chosen for this book is the WSO2 Enterprise Integrator (EI) by WSO2.

As we can read on Wikipedia about the history of the company (https://en.wikipedia.org/wiki/WSO2):

Based on the experience of working with WSO2 products for more than five years, we have to follow how the company has grown and expanded due to its innovative culture. Along with this experience, we have witnessed the evolution of the WSO2 components since the successful early version of the ESB and DSS to the current ones, and the brand new WSO2 EI.

In the following chart from the WSO2 site, we can see the WSO2 vision of the SOA infrastructure:

WSO2 ESB (http://b.content.wso2.com/sites/all/technology-microsites/integration/images/integration-micro-complete-enterprise-ready-integration.png)

Here, we can see how WSO2 ESB is the heart of the infrastructure and relies on the other components:

WSO2 Data Services Server: This is a very successful component that lets you expose data from a very diverse source, such as web services or REST resources, in a couple of minutes without any code.
WSO2 Message Broker: This is a message broker, as its name says, that supports the Publish/Subscribe model, Java Message Service (JMS), and Advanced Message Queuing Protocol among other features, which allows us to build complex business services.
WSO2 Business Process Server: This a business process engine. Business processes are a higher level of abstraction called Business Process Management (BPM). BPM is based in business services that compose business processes, and supports BPMN 2.0, WS-BPEL 2.0 , WS-Human Task 1.1, and BPEL4People 1.1 standards.

The SOA component missed in the preceding chart is the UDDI registry. WSO2 offers this UDDI registry with its component called WSO2 Governance Registry; this component is more than a simple UDDI registry. It also provides a service catalog as well as a repository for the whole set of governance elements, such us policies, documents, schemas, and web services descriptors, for the SOA Governance.

As we said earlier, the WSO2 is a very dynamic company that tries to be cutting edge. The last component for SOA from WSO2 is the WSO2 EI 6. In this component, they put the most important SOA functionalities in just one component. Those functionalities were previously encapsulated in the following components:

WSO2 Enterprise Service Bus
WSO2 Data services Server
WSO2 Message Broker
WSO2 Business Process Server
WSO2 Application Server

So, this brand new component offers a new vision of the SOA infrastructure:

WSO2 Enterprise Integrator (http://wso2.com/integration)

We have already discussed all these components in the preceding figure about the WSO2 SOA infrastructure, except WSO2 Application Server. WSO2 Application Server, as the name says, is a Java application container for web applications, web services, and RESTful services. This container is called the Carbon platform. According to the level of this book, we will focus on the following WSO2 EI functionalities:

Service Integration
Data Integration
Messaging
Analytics
Tooling

In the subsequent chapters, we will learn how to install WSO2 EI, create web services and data services, and how to test and manage them. We will also learn how to build services that use message queues, data services, and connectors to integrate with third-party applications.

WSO2 Update Manager

WSO2 Update Manager will allow us to download many of the WSO2 products with the latest improvements and fixed bugs applied. They can be downloaded from http://wso2.com/update/wum/download, where we have to just choose the right version for our operating system and follow the instructions provided to install it.

Once installed, we can access WSO2 Update Manager by opening a terminal and typing:

>wum

And if everything is okay, we will see this in the console:

WUM keeps WSO2 products up-to-date.* Find more information at http://wso2.com/updateUsage:wum [command]Available Commands:init         Initialize WUM with your WSO2 credentialssearch       Search products containing specific keyword(s)add          Add or download a productcheck-update Check for new updates for productsupdate       Update products in your local repositorylist         List products in your local repositorydescribe     Show details of products in your local repositorydelete       Delete products in your local repositoryconfig       Change WUM configurationversion      Display wum version informationFlags:  -h, --help      help for wum  -v, --verbose   enable verbose mode

    Use wum [command] --help for more information about a command.

This tool allows us to download and update our products so that we will always be up to date with fixed bug and improvements. It is important to note that, as we can see on the product page, WUM is not Apache 2.0 like the rest of the components, and it can only be freely used in non-production environments. For production environments, it is necessary to pay a subscription.

We will go through the most useful command needed to download and update our server.

The mandatory first step we always have to perform is to initialize the tool using a WSO2 user account. When we do this, we type this:

>wum init

Then, we will be asked for a WSO2 username and password. In case we have already launched the init command, only the password will be requested in the console:

You need a WSO2 account to start using wum. Don't have one yet? Sign up at https://wso2.com/user/register:

Please enter your WSO2 credentials to continueUsername: [email protected]Password for '[email protected]': Authenticating...Done!-- Welcome to WUM 1.0-beta --* Find more information at http://wso2.com/update* Please contact us for further information at http://wso2.com/contact
What's next? Have a look at the following list of wum commands. Add WSO2 products to your product repository:
wum search            Search WSO2 products
wum add               Add or download WSO2 products

Update WSO2 products available in your product repository:
wum check-update      Check for new updateswum update            Update your WSO2 products

Manage WSO2 products available in your product repository:
wum list              List WSO2 productswum describe          Show details of WSO2 productswum delete            Delete WSO2 products

Use wum [command] --help for more information about a command.

Now we are ready to download or check for updates for our products. By default, the tool places the repository in the operating system user's home directory; to avoid this, we will change the location of the repository to our working folder.

We will place our repository in the wso2/products/ path; so, we type as follows:

>wumconfiglocal.product.repo wso2/productsNew product repository is \wso2\products

In this repository, WUM places all the distributions in the download, and the new distributions are generated when there is a new update.

At this point, the most common tasks will be as follows:

Adding an existing downloaded product to the repository.
Downloading a new product.
Checking for updates of the server added to the repository.
Deleting a product from the repository.

Add an existing product to WUM repository

In cases where we have downloaded the product from the product page of wso2.com, we can add this download to the repository so that we can check for updates.

If we have previously downloaded, for example, WSO2 ESB 5.0.0, the previous version of WSO2 EI, and we want to add it to the repository to check whether any update is available for it, we type the following:

C:>wum add --file wso2esb-5.0.0.zipConnecting to WSO2 Update...Adding product wso2esb-5.0.0...Successfully added to following location:C:\wso2\products\wso2esb\5.0.0\wso2esb-5.0.0.zip
    What's next?
    use wum check-update wso2esb-5.0.0 to check for updates
    use wum update wso2esb-5.0.0 to install latest updates

Remember that in Mac, you need to put the full path when typing the name of the file; in this case, we need to type the full path for the file, wso2esb-5.0.0.zip.

Now, we have WSO2 ESB 5.0.0 in the local repository. The ZIP containing the original server will remain in the source location, and an updated/patched copy will be placed under [product.repo]/wso2esb/5.0.0/ in the repository location.

Next step, as we can read in the message displayed in the console, is to check for updates.

Download a product using WSO2 Update Manager

We can download a product using the WUM tool. This is our recommendation as it is the best way to keep your server up to date. In case you need to install an update in a server that is manually downloaded, you will have to install WUM and add that server to the repository anyway, so it is better to use it from the beginning.

To download WSO2 EI using WUM and add it to the repository, we just need to type the following in the console:

>wum add wso2eiConnecting to WSO2 Update...The following product(s) will be downloaded.wso2ei-6.0.0.zipAfter this operation, 600.8MB of additional disk space will be   used.Do you want to continue? [Y/n] ySuccessfully added to following location:6MB/600.8MB]
    What's next?
product wso2ei-6.0.0... [566.7MB/600.8MB]
    use wum check-update wso2esb-5.0.0 to check for updates
    use wum update wso2esb-5.0.0 to install latest updates

When the download is complete, we will have WSO2 EI 6.0.0 in our repository. We can list the products existing in the repository by typing as follows:

C:\wso2\products>wum listProduct                 Updated         Filenamewso2ei-6.0.0            -               wso2ei-6.0.0.zipwso2esb-5.0.0           -               wso2esb-5.0.0.zip

What's next? Use wum describe [<product-pattern>] to get more details of products. Also, if we ask for details, we get this:

C:\>wum describe wso2ei-6.0.0Filename:               wso2ei-6.0.0.zipProduct Name:           wso2eiProduct Version:        6.0.0Kernel Version:         4.4.14Last Updated Time:      -Product File Path:      C:\wso2\products\wso2ei\6.0.0\wso2ei-6.0.0.zip

We can get the absolute path in the filesystem where the product is located using this command. We can verify that the ZIP file containing the server is available in the path where we located the WUM repository in the filesystem.

Check for product updates using WSO2 Update Manager

Now that we have all of our products in the repository, as we can check with the wum Gstlist command, we are ready to check for updates. To achieve that, we type the following:

C:\ >wum check-update wso2ei-6.0.0Connecting to WSO2 Update...Checking for latest updates for wso2ei-6.0.0...Product is up to date.

In this case, WSO2 EI is up to date, so no action is required. However, if we check for updates for the other product, WSO2 ESB 5.0.0, in our repository, we have this:

>wum check-update wso2esb-5.0.0Connecting to WSO2 Update...Checking for latest updates for wso2esb-5.0.0...37 updates are available[WARNING] 17 critical security updates. WSO2 strongly recommends that 
    you install these updates now.

What's next? Use wum update to install the latest updates.

In that case, due to WSO2 ESB 5.0.0 being released in 2016, there are updates available. In spite of the fact that it is not the product we are focused on in this book, we will show the update procedure using this product since the procedure is the same for all the products managed by WSO2 Update Manager.

So, to install the latest updates in all the products in the repository, we type this:

C:\>wum updateConnecting to WSO2 Update...Validating your subscription status for product wso2ei...[WARNING] Your credentials are not associated with an active WSO2 Updating wso2ei-6.0.0...Product is up to date.Validating your subscription status for product wso2esb...[WARNING] Your credentials are not associated with an active WSO2
    subscription.

Please remember that updates are not licensed for use in production without a valid WSO2 subscription. See http://wso2.com/update for more details:

Updating wso2esb-5.0.0...37 updates are availableDownloading updates... [36/37]Installing updates...Preparing update summary...Building updated product...Update summary:  Installed updates: 37  * [WARNING] WSO2 strongly recommends to use this updated product for 
      production as soon as possible.  Security updates: 17Updated Product: C:\wso2\products\wso2esb\5.0.0\wso2esb-
      5.0.0.1488178152499.zip  * More information about updates are available inside the above
      product archive  Update summary(pdf): (product-archive)/updates\summary-2017-02-
      28T19-31-14\update-summary-1488178152499.pdf

What's next? Use wum list [<product-pattern>] to list products in your local repository. Use wum describe [<product-pattern>] to get more details of products.

The result is a new ZIP generated with all the updates ready to be installed.

Deleting a product from the WSO2 Update Manager repository

We can remove a product from the repository in a very simple way; we just have to type as follows:

C:\>wum delete wso2esb-5.0.0The following product file(s) will be deleted.wso2esb-5.0.0.1488178152499.zipwso2esb-5.0.0.zipDo you want to continue? [y/N] y

This command removes the ZIP file containing the product as well as all the releases generated as a result of new updates to the product.

Installing WSO2 Enterprise Integrator

Once we have obtained the up-to-date product ZIP, we are ready to install it. According to the level of the book, we will assume a target reader who will use WSO2 EI for development or testing purposes. We will also assume that the reader has a basic knowledge of WSO2 EI or this is their first contact with WSO2 EI. Advanced users may need to refer to the official product documentation at https://docs.wso2.com/display/EI600/WSO2+Enterprise+Integrator+Documentation.

We will describe the installation under the following assumptions that are not recommended for a production environment:

WSO2 EI runs using the default H2 embedded database
WSO2 EI H2 embedded registry
Standalone configuration

The prerequisites are quite simple, and for most cases, for a development or testing environment, these are enough:

Memory of 2 GB minimum and 1 GB heap size.
Space disk of 1 GB for installation. This is space just for the component; additional space is needed for logs and databases.
JDK 1.8.
JavaScript enabled browser.

If we meet these requirements, we will be able to run WSO2 EI. The installation for both the Linux and Windows operating systems are quite simple as well; we just need the JAVA_HOME environment variable properly configured, pointing to the JDK installation. Of course, there are several ways to create environment variables, but we will focus on a standard one, that is valid for most scenarios:

Linux:
1. Edit the BASHRC file in your home directory to add the JAVA_HOME variable definition at the top. The JAVA_HOME variable must point to the JDK installation folder. In this example, we are supposing that it is installed in the following:

/usr/java/jdk1.8.0_121: 
export JAVA_HOME=/usr/java/jdk1.8.0_121 
export PATH=${JAVA_HOME}/bin:${PATH}

2. Verify that the JAVA_HOME variable is properly set in the console using the echo $JAVA_HOME command. The result must be the JDK installation path we set earlier.

Windows:

1. Go to System properties | Advanced | Environmental variables.

2. In the System Variable section, add new.

3. Set the name as JAVA_HOME

4. Set the value as the JDK installation path, for example, C:\Program Files\Java\jdk1.8.0_121.

5. Verify that JAVA_HOME is properly set, executing the set JAVA_HOME command in the console. The result must be the JDK installation path.

Now we are ready to launch WOS2 EI. As we discussed in the previous sections, WSO2 EI encapsulates six products that were previously independent of the following:

WSO2 ESB.
WSO2 Data Services Server (WSO2 DSS).
WSO2 Application Server (WSO2 AS). In EI 6.1.1 and so on, it is not a feature anymore.
WSO2 Analytics.
WSO2 Business Process.
WSO2 Message Broker.

The first three products have been merged into just one component called WSO2 EI, so WSO2 EI is made up of the following:

WSO2 EI
WSO2 EI Analytics
WSO2 EI Business Process
WSO2 EI Broker

So, when we talk about starting the WSO2 EI, we have several options:

Start the four components manually in a specific order
Start the four components automatically using the script provided
Start the component on demand. According to the level of the book, WSO2 EI Business Process and WSO2 EI Broker require a deeper knowledge of WSO2, so we miss them in this book

Starting components manually

We can start the component one by one on demand. For example, we cannot create business processes in this book, so we do not need to start WSO2 Business Process. In such cases where we do not need a message queue, we do not need to start the WSO2 EI Broker. WSO2 EI Analytics will be useful to check message traces when invoking a service and to get the services statistics, so we will use this component.

We need to start the components in a specific order, as follows:

WSO2 EI.
WSO2 EI Analytics.
WSO2 EI Business Process.
WSO2 EI Broker.

In the following sections, we will see that all of them start and stop the same way.

Starting/stopping WSO2 Enterprise Integrator

To start WSO2 EI, we have to follow these steps:

Open a console and go to the <WSO2EI_HOME>\bin directory.
Type the following command:
- On Linux or macOS: sh integrator.sh
- On Windows: integrator.bat --run

You will see a log that the boot up is generating. The server will start up when you see a message log as shown:

      WSO2 Carbon started in X sec

We will also see the URL to access the WSO2 EI console in a message as follows:

      Mgt Console URL  : https://<EI HOST>:<PORT>/carbon/ 
      Where <PORT> is: 
               8243 for WSO2 EI version 6.0.0 
               9443 for WSO2 EI version 6.1.0 o higher

This port number will not change depending on the version for the rest of the components provided with WSO2 EI.

We will see, similar to the following screenshot, that the server is up:

WSO2 EI boot up finished

In this screenshot, we can see the Management Console URL. We can see that the URL is shown using the host IP. When we access the console from the host that WOS2 EI is running on, we use localhost instead of the host IP. We will use the host IP to access the console from other hosts in the LAN or from internet:

WSO2 EI 6.0.0:
- https://[your_IP]:8243/carbon/
- https://localhost:8243/carbon/
WSO2 EI 6.1.0 or greater:
- https://[your_IP]:9443/carbon/
- https://localhost:9443/carbon/

You may have noted that the Management Console uses the HTTPS protocol. This is a secure protocol that encrypts the traffic between the browser and the server for security reasons; to do this, a certification is needed. All WSO2 products are provided with a default certificate that is not signed by a well-known certification authority but is okay for testing purposes. For this reason, when we access the Management Console, the browser will show an untrusted connection warning. We have to accept that certificate to access the Management Console. For a production environment, we will need a certificate generated by a well-known certification authority and then, there will not be any warning when accessing the Management Console.

All these considerations are valid for all the components, so we have to proceed in the same way when starting the other components.

Now we are able to browse to the Management Console:

WSO2 EI Management Console

Now we just have to type the default username and password to sign in:

Username: admin
Password: admin

Then, the browser will show the WSO2 EI home page:

WSO2 EI home page

WSO2 EI can be shut down/restarted using the console. We can find this option in the Main tab by clicking on the Shutdown/Restart option:

Shutdown and restart from Management Console

We also can shut down the product this way:

On Windows: Using Ctrl+C in the command window
On Linux/macOS: Using sh integrator.sh stop from the <EI_HOME>/bin/ directory

Starting/stopping WSO2 EI Analytics

The procedure to start this component is quite similar to the WSO2 IE one; we just need to follow these steps:

From Command Prompt (Windows) or shell (Linux), go to the <EI_HOME>/wso2/analytics/bin directory.
Type the following command:
- On Linux or macOS: sh integrator.sh
- On Windows: integrator.bat --run

You will see a log that the boot up is generating. The server will start up when you see a message log as illustrated:

      WSO2 Carbon started in X sec

We will also see a URL to access the WSO2 EI console in a message, as follows:

      Mgt Console URL  : https://<EI HOST>:port/carbon/

The same considerations for accessing WOS2 EI Management Console are valid here. The port, in this case, is offset by 1, so we will browse to https://localhost:9444/carbon. Take a look at the following screenshot:

WSO2 EI Analytics Management Console

We sign in with the Username as admin and Password as admin to get the WSO2 EI Analytics Home:

WSO2 EI Analytics Home

WSO2 EI Analytics can be shut down/restarted using the console. We find these options in the Main tab by clicking on the Shutdown/Restart option. We also can shut down the product this way:

On Windows: Using CTRL+C in the command window
On Linux/macOS: Using shwso2server.sh stop from the <EI_HOME>/wso2/analytics/bin directory

Starting/stopping WSO2 EI Business Process

The procedure to start this component is quite similar to the WSO2 IE one; we just need to follow these steps:

From Command Prompt (Windows) or shell (Linux), go to the <EI_HOME>/wso2/ business-process/bin directory.
Type the following command:
- On Linux or macOS: sh.ws02server.sh
- On Windows: ws02server.bat
You will see a log that the boot up is generating. The server will be started when you see a message log as follows:

      WSO2 Carbon started in X sec

We will also see a URL to access the WSO2 EI console in a message, as illustrated:

      Mgt Console URL  : https://<EI HOST>:9445/carbon/

The same considerations as for accessing WOS2 EI Management Console are valid here, so we will browse to https://localhost:9445/carbon. Take a look at the following screenshot:

WSO2 EI Business Process Management Console

We sign in with the Username as admin and Password as admin to get the WSO2 EI Business Process Home:

WSO2 EI Business Process Home

WSO2 EI Business Process can be shut down/restarted using the console. We can find this option in the Main tab by clicking on the Shutdown/Restart option. We can also shut down the product this way:

On Windows: Using CTRL+C in the command window
On Linux/macOS: Using sh wso2server.sh stop from the <EI_HOME>/wso2/ business-process/bin directory

Starting/stopping WSO2 EI Broker

The procedure to start this component is quite similar to the WSO2 IE one; we just need to follow these steps:

From Command Prompt (Windows) or shell (Linux), go to the <EI_HOME>/wso2/ broker/bin directory.
Type the following command:
- On Linux or macOS: sh.wso2server.sh
- On Windows: wso2server.bat

You will see a log that the boot up is generating. The server will start up when you see a message log as shown:

      WSO2 Carbon started in X sec

We will also see a URL to access the WSO2 EI console in a message, as illustrated:

      Mgt Console URL  : https://<EI HOST>:9446/carbon/

The same considerations for accessing WOS2 EI Management Console are valid here, so we will browse to https://localhost:9446/carbon. Take a look at the following screenshot:

WSO2 EI Broker Management Console

We sign in with the Username as admin and Password as admin to get the WSO2 EI Broker Home:

WSO2 EI Broker Home

WSO2 EI Broker can be shut down/restarted using the console. We can find this option in the Main tab by clicking on the Shutdown/Restart option. We also can shut down the product this way:

On Windows: Using CTRL+C in the command window
On Linux/macOS: Using sh wso2server.sh stop from the <EI_HOME>/wso2/ broker/bin directory

Starting all the products

In cases where we need to start all the components, we can do it using just one command; we just need to follow these steps:

From Command Prompt (Windows) or shell (Linux), go to the <EI_HOME>/bin directory.
Type the following command:
- On Linux or macOS: shstart-all.sh
- On Windows: start-all.bat

The four components will start using a 10-second gap between each launch. After a few seconds, all the components will be up and each Management Console will be available.

To stop the components, we have to stop each of them manually, using each Management Console or using the console instructions.

WSO2 EI Configuration

There are some configuration tasks that must usually be done when developing real-world services:

Configuring JDBC drivers
Configuring transports
Configuring message formatters and message builders

Configuring JDBC drivers

This is a mandatory configuration task when using data services. JDBC drivers allow WSO2 EI to connect to databases. By default, there are many JDBC drivers installed, so we have to configure the drivers' need for connecting the data sources that will provide us with the data in the orchestration.

We can do that with these simple steps:

Locating the correct JDBC drivers for the databases that will play in our orchestration.
Copying them to the <EI_HOME>/lib/ folder.
Restarting the server.

Once the server is restarted, we will be able to connect to the databases.

Configuring transports

WSO2 EI supports several transports that we can use when building our services, such as the following:

JMS transport: This enables sending and receiving messages to queues and topics that implement the JMS specification
Mailto transport: This enables sending emails
VFS transport: Virtual File System (VFS) transport allows us to process files in a directory of the filesystem

We can enable this transport and many others in the <EI_HOME>/conf/axis2/axis2.xml file. All the transports available can be found in that file with a default configuration. We have to configure these transports for input and output connections. The input transports are configured using the transportReceiver XML tag, while the output transports are configured with the transportSender tag:

<transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender"/>

<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
<parameter name="myTopicConnectionFactory" locked="false">
<parameter name="java.naming.factory.initial" locked="false">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>
<parameter name="java.naming.provider.url" locked="false">conf/jndi.properties</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">TopicConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>
</parameter>

<parameter name="myQueueConnectionFactory" locked="false">
<parameter name="java.naming.factory.initial" locked="false">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>
<parameter name="java.naming.provider.url" locked="false">conf/jndi.properties</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
</parameter>

<parameter name="default" locked="false">
<parameter name="java.naming.factory.initial" locked="false">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter>
<parameter name="java.naming.provider.url" locked="false">conf/jndi.properties</parameter>
<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>
<parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
</parameter>
</transportReceiver>

Configuring message formatters and message builders

Message formatters and Message builders are the components that allow us to send and receive different types of messages according to the content type specified in the request. It is important to enable the content type used in the messages exchanged because if the content type received is not configured in the system, the message will not be understood.

Message builders are used to process incoming messages, and Message formatters are used to build the outgoing messages.

We can enable the Message formatters and builders in the file located in <EI_HOME>/conf/axis2/axis2.xml. Message formatters are under the same name in XML tag, and so are Message builders. Most values that messages content type can be received are already configured, but they are commented. We just have to find the one we need and uncomment it for Message formatters and/or message builders, depending on the case.

For example, to enable us to send and receive JSON messages, we have to enable that content type for input and output messages in axis2.xml:

<messageFormattercontentType="application/json" class="org.wso2.carbon.integrator.core.json.JsonStreamFormatter"/>

<messageBuildercontentType="application/json" class="org.wso2.carbon.integrator.core.json.JsonStreamBuilder"/>