Move to Wordpress

I moved my blog to http://javamaster.wordpress.com

 

New J2EE Architecture with Oracle Coherence.

 

2009-06-01

Oracle Korea Consulting / Principal Consultant

Terry.Cho (byungwook.cho@oracle.com)

 

Intro

Web Application Server(WAS) is introduced about 5 years ago. It is already matured. So there is no difference between products from different vendor. To get a higher position in WAS market place. We need to develop difference from other vendor's product.

 

In case of Korea success rate of JEE based SI projects are getting lower. A one of the reason we can think about is framework. Most JEE projects are using framework like spring,hibernate,JAX-WS,XML framework etc. It helps us to leverage efficiency of implementation and provide more higher quality of architecture. In contrast it makes our system to more complicate. It causes a trouble in production system. Makes system hang, bring high cpu usage, memory usage etc

In this document, I will introduce WAS and coherence based architecture to raise up quality of JEE architecture.

 

Pain point of traditional JEE architecture

Traditional JEE architecture is like below.

All of business logics are packaged into one application (1 WAR or 1 EAR file) including user interface. BIZ logic is implemented with EJB or POJO (Spring etc. POJO is current trend). UI is implemented Servlet/JSP of MVC framework.

 

This traditional JEE architecture has weak points.

Requires lots of redundant memory.

Current requirement for IT is being increased and complex is increased. So size of application and memory usage of application is being increased. The problem is JVM has a limitation in memory size. In 32bit JVM, Java can use usually maximum 1~1.5 GB Heap memory. Even if we can use 64bit JVM. We still have the problem in Garbage collection (GC) time. In full GC time, JVM suddenly stopped. If we increased the java heap size, the full GC time also increased. From JVM 1.4 there are many GC tuning options like Concurrent GC (CMS), Parallel GC. We are not free from the GC time.

In the traditional architecture all of biz logics are packaged into one application. Memory usage is getting increased. Especially it makes a burden in JVM permanent memory area. (The area that classes are loaded.)

 

Trouble affects other business Logic

Another problem of traditional JEE architecture is trouble propagation. Because all of business logics are running over same WAS instance, if some business logic makes a problem like high CPU usage, high memory usage affects the other business logic. It means if only one business logic has a problem, whole system can be stopped. Even if our weblogic server provides workmanager and clustering feature which prevent problem propagation, the problem from business logic can not be prevented.

 

How to solve this pain point?

So how can we solve this problem? Answer is easy. Separate the business logics to separate WAS instance.

 

It helps us by

Effective memory usage

Because only one business logic is deployed to one instance. Memory utilization is became low.

 

Prevent affection of problem between different business logic

Business logic is separated, so if one business logic has a problem, only the business WAS instance (or cluster) has a trouble. It doesn't provide any effect to other business.

 

Efficient resource allocation

In addition separating business by WAS instance can make us to use hardware resource efficiently .

In Hard ware platform which supports CPU partitioning, system administrator can assign CPU to specific business logic. In traditional JEE architecture, it is hard to assign the resource to business. Because all business logics are running over same WAS instance, so cannot select process which CPU is assigned to.

 

So, even if the separation of business logic by WAS instance has a lot of architectural, why the traditional JEE architecture doesn't implement it?

Problem comes from shared data & state information management.

 

In JEE architecture needs to share business data across the business logics or keep user state data by using Http Session. But if we separate business logic by WAS instance, it cannot share the session data and shared information across the WAS instances.

It is a reason why our suggested architecture cannot be realized.

 

Separate business logic across WAS instance with Oracle coherence.

Finally with Oracle Coherence solution we can realize the architecture.

Coherence is Data Grid Solution which is not just Caching solution but is extendible, high performance, high availability large shared memory with multiple interface.

Basically Coherence support HTTP session replication. It gives us more Http session capacity and solves the problem to sharing user state data.

 

By using coherence we can solve all of the pain point in traditional JEE architecture.

We get additional advantage from this architecture. We can plan a capacity based on business logic. (Assign WAS instance to specific biz logic based on capacity.)

And it also give us freedom of redeployment. We don't have to redeploy whole of WAS instance and don't have to restart whole instances.

 

Conclusion

It is hard to win a deal with only WebLogic server in JEE application market. We need a approach to suggest integrated package of our product and especially with ARCHITECTURE which can solve pain point of our customers.

WebLogic and Coherence are very excellent product from before acquired by Oracle. Acquisition should provide advantage with chemical interactions. This is the case that can make the chemical interactions and will provide a higher position in biding with other vendors.

 

 

Programming WebService with JAX-WS

Overview

This is a programming guide for component developer to make a webservice easily.

Technical Standard

This is webservice creation guide for component team.
We will use webservice development standard like this.

• JAX-WS
• JAXB
  

Standard IDE is WebLogic workshop 10.3. You can use any IDE that you want. JAX-WS is standard.WebLogic provides eaisest development environment. If you want to find any alternatives, i want to recommend CXF in Apache. It is easily integrated with Spring framework

Restriction

Here is webservice development restriction for more easy development

• Synchronous WebService Call
  You can just use synchronous message exchange pattern. If you want to use async,pub-sub, or queuing etc. Please talk with Application Architect.
  

• DataTypes in Value Object has a restriction.
  Because of we are using JAXB. We have a restriction to use data type in Value Object that is used as parameter or return type of webservice.
  Here is list of data type you can use it to define ValueObject

  Java Data Types	XML Scheme Data Type
  boolean	boolean
  byte	byte
  double	double
  float	float
  long	long
  int	int
  String	string
  java.math.BigInteger	integer
  java.util.Calendar	dateTime
  java.util.Data	dateTime

  In addtion you properbly need a use a some of array type. Here is a guide. You have to remember that you cannot use Map or Hashtable data type. You have to covert the types into ArrayList etc.

  Java Data Types	XML Scheme Data Type
  java.util.Collection	Literal Array
  java.util.List	Literal Array
  java.util.ArrayList	Literal Array
  java.util.LinkedList	Literal Array
  java.util.Vector	Literal Array
  java.util.Stack	Literal Array
  java.util.Set	Literal Array
  java.util.TreeSet	Literal Array
  java.utils.SortedSet	Literal Array
  java.utils.HashSet	Literal Array

Programming Steps

1. Make a ValueObject
Make a ValueObject.You can easily make a ValueObject with datatype decribed above.
You have to use some notation to provide more clarified WSDL generation.

Item.java

package osp.sample.valueObject;

import javax.xml.bind.annotation.*;


@XmlAccessorType(XmlAccessType.FIELD)
@XmlType (propOrder={
 "name",
 "price",
 "quantity"}
)

public class Item {
@XmlElement(required=true)
String name;


@XmlElement(required=true)
int price;


@XmlElement(required=true)
int quantity;

public String getName() {
 return name;
}

public void setName(String name) {
 this.name = name;
}

public int getPrice() {
 return price;
}

public void setPrice(int price) {
 this.price = price;
}

public int getQuantity() {
 return quantity;
}

public void setQuantity(int quantity) {
 this.quantity = quantity;
}



}

• First import java.xml.bind.annotation.* to use JAXB annotation
• specify XmlAccessorType with XmlAccessType.FIELD
• specify XmlType and propOrder. This for sequence of element generated by Java Class.
• Each of java variable. Specify XmlElement and required=true. This is just for the Xml parsing performance. If the data type is List or Array. You don't have to specify XmlElement
  
  

2. Create WebService
Create WebService in WebLogic Workshop IDE and implement a Webservice.

OrderWebService.java

package bcho.sample.webservice;

import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;



import bcho.sample.valueObject.PurchaseOrder;



@WebService
public class OrderWebService {


@WebMethod
public @WebResult(name="purchaseOrder")PurchaseOrder addOrder(
  @WebParam(name="purchaseOrder")PurchaseOrder po)

{
 return po;
}

}

• You also import javax.jws.* to use JAX-WS annotation.
• Use @WebService annotation into a class to declare the class is webservice.
• Use @WebMethod annotation into method, the method will be automatically generated "operation" in webserivce.
• Use @WebResult and @WebParam in return value and parameter. As i mentioned earlier, please use one result value and one result parameter.
  The name of @WebResult and @WebMethod should be equals with ClassName.

3. Run a Test
You can test a webservice in WebLogic Workshop IDE.

4. Package and deploy

• After testing, in WebLogic Workshop . Select a project in Project explorer.
• Click right button of Mouse and select Export.
• Select war in export menu. The *.war file automatically generated.
• Please delpoy the war file into Pilot Project environment (WebLogic Server)

Here is a short tutorial movie of WebService development.
[pilot:JAX-WS based Web Service Programming Guide^WebLogic 10.3 JAX-WS Creation guide.mpeg]

Reference

JAXB Development http://edocs.bea.com/wls/docs103/webserv/data_types.html#wp223908
JAX-WS WebService Development http://edocs.bea.com/wls/docs103/webserv/jws.html#program_jws_file

Test Management

Overview

Our testing model in based V model. It is extension of traditional water fall model.
Left side of model represent software development life cycle and right side means testing model.
In left side, deliverables are static things such as document or source code.

To verify these static deliverables, it has be done by human.
Verification means “review & inspect artifacts by human”. For example pair-review, code inspection etc.

From right side, we have a actual software system that really works. In this stage we validate real software with automated testing tools.

The test is concentrated on this validation process.

Validation phase is separated into four steps.

Level	Description	Testing Type	Responsibility
Unit Test	Verify software component	White Box	Dev Team
Integration Test	Verify integration between components. Verify software flow, interfaces and interactions.	White Box	Dev Team
System Test	Test system over production environment to verify system itself and production environment. Include Functional & Non functional Test. ( Availability, Stability, Extendibility, Performance etc).	Black box	QA Team
Acceptance Test	Verify customer’s requirement, legal issue, maintenance issue etc.	Black box	QA Team or Customer

Each of test step has a test cycle.

Ø Pre Test

Pre Test phase is preparation phase of testing.

In thin phase prepare target testing system, test scenario and mock up system for testing etc.

Ø Main Test

Execute test. Failed test is logged in test case management system.

Ø Conformation Test

The test that is failed during main test and then fixed later is tested in this phase.

Ø Regression Test

Regression test is for check up the impact from change. This test is done every time when it has a new change.

To execute regression test, all of test case should be automated. Every time execute regression test without automation is a lot of works.

1.1. Unit Test & Integration Test

This section describes Unit Test and Integration Test. As I mentioned earlier this test is done by development team by themselves.

1.1.1. Development and Testing Environment

In the test environment perspective Unit test and integration test are almost same. Only difference is scope. Unit test concentrate on individual component itself. Integration test’s scope is functionalities that are made by composition of individual component.

1.1.2. Unit Test

The goal of acceptance test is get
Unit test is testing for verifying software component works well. In J2EE world software component is Java class,EJB component etc.
This test is done by developer itself.

Every time developer develops software component. It is tested by developer itself before committing the code into SCM.

There are many types of software component, so there are many type of unit test framework as well.
Here is functional unit test framework.

Ø Basic Testing framework

Basic Testing framework. It is broadly used to test software component.

Junit is used for basic testing framework.

Ø In container Testing framework

One of extension of basic testing framework. J2EE application needs a application server like weblogic etc. When we tests software component we need a application server as well.

For example, getting session information or getting servlet context can not be run our of application server.

In container testing framework helps test case to run on container.

Ø DB Testing framework

Extension of basic testing framework. It is specialized for Data testing.

For example when we try to test DAO (Data Access Object) ,

1. we have to initiate DBMS

2. Run a logic

3. Compare result with expected data

4. Roll back the database to previous state

For example when we try to test DAO (Data Access Object) ,We can implement this test scenario with pure java & jdbc but it takes a lot of resource.And managing a test data is big burden for developer and DBA as well.

To simplify this process we use DB Testing framework. ,

1. Dump data from existing data base or write a data to external file such as excel or xml

2. Initiate DBMS with the file

3. Execute test case.

4. Compare data in DBMS with data (XML file,Excel or file dumped from DBMS)

5. Recover data with the file.

Ø UI Testing framework

Web application test needs a UI test.
For example querying profile application test scenario is

1. Input a name to text box

2. Press submit button

3. Check the result that has a expected name and phone number etc.

Automating UI Testing is very hard to framework. UI Testing framework execute these kinds of web navigation process automatically.

In addition some of component should be tested with performance. To support this we have Performance unit test framework

Ø Performance unit test framework

Most important factors of transformation or document parsing application is performance. Some component’s main factor is performance. To validate this performance testing framework can give a stress(heavy load) to component and can check the response time etc.

There are many issue to generate load for component like threading issue etc.

Performance unit test framework helps developer to execute this performance unit test without changing or making a new code for performance test. This framework just reuse existing unit test case and just add a load.

Even if developer tested software component, we cannot know what components are tested or not. This is testing for test. There are many methodology to validate test itself. But “Test coverage” strategy is mainly used.

Test coverage is validated how much test target is tested during the test.

For example how many source code lines are tested. How may requirements are tested. How may classes are tested etc.

In Unit test, we uses “Code coverage”. Code coverage means ‘How many source codes are tested in unit test”

By using this result we can estimate confidence of unit test.

Ø Code coverage analysis framework

Mechanism of code coverage tool is incrementing software code so gathering coverage data in runtime.
There is two kinds of incrementing mechanism.

1. Static instrumentation

Static instrumentation works in compile time. When compiling source code , coverage tool insert their code to compiled binary. In this case after deployer have to manage two kinds of achieve file. One is for test coverage, the other is for runtime.

2. Dynamic instrumentation

Dynamic instrument works in runtime. It hooks JMV itself so it put instrument code when class is loaded into memory. It doesn’t need to make different version for runtime and coverage.

But it has a runtime overheard for incrementing

In addition Enterprise application is composed of huge lines of code, so it is very hard to achieve 100% code coverage.

So we will employ risk-based test strategy, which was introduced by Eric Van.

We define Risk as follows:

Risk = likelihood * impact

Likelihood can have factors like “Need high skills to implement”, “ Complex functionalities” , “Not validated framework” etc.

Impact can also have factors like “Can lose revenue”, “Lost a user royalty” etc. It is mainly related to business.

Then, make a table for each test item with both likelihood factor and impact factor. A test item could be a class, module or package.

	Likelihood		Impact
	factor	factor	factor	factor
Test item
Test item

Figure 1 Risk Analysis Table

Calculate risk value with likelihood and impact. Then, we can rank a test item as below:

Figure 2 Risk Analysis Matrix

STA area has a high probability of occurrence with big biz impact. This should be tested.

ITA area also has a high probability of occurrence yet with small biz impact. In general, ITA area is related to technical issues so that it should be handled in development phase.

SSTA area has a low probability of occurrence with big biz impact. Therefore, preferably it should be handled in system testing and acceptance testing.

By using this technique, we can verify the items that should be tested in development phase (Unit Test and Integration Test). And the test items should meet a coverage rate of around 80~90%.

After we define test items, they should be managed in terms of coverage. In this regard, we need to measure code coverage.

Ø Mock Test Framework

Mock Test framework is used for making a mock up object of component.

Unit testing tests component in isolation. But most units do not work alone. To test unit in isolation. We have to simulate the collaborators in the test.

Mock Test framework creates mock up object based on component interface (eg. Java interface). Record a action and replay a action during running unit test.

If unit test tests software runtime. Static test tests source code in compile time.

Code inspection by human is very effective but not efficient. There a huge line of code and there are many change as well. So it is unable to inspect all of code by human. Static test tool helps this.

Ø Static Test

Based on pre-defined defect pattern, the tool validate the source code and find a defect. Even if the tool cannot find all of the defect. A lot of risky defect can be found by this tool. It is very efficient.

There is many static test technique. Defect pattern matching, naming convention check, code complexity analysis (by using cyclomatic complexity analysis.). Feature is depending on Static Test tool.

Here is composition of recommended unit testing framework product that I mentioned above.

Category	Product	Provider
Basic Testing	JUnit	http://www.junit.org/
DB Testing	DBUnit	http://www.dbunit.org/
In container Testing	Cactus	http://jakarta.apache.org/cactus/index.html
UI Testing	Selnium	http://selenium.openqa.org/
Coverage analysis	Cobertura	http://cobertura.sourceforge.net/
Mock testing	EasyMock	http://www.easymock.org
Static anlaysis	FindBug	http://findbugs.sourceforge.net/
Code complex analysis	CheckStyle	http://checkstyle.sourceforge.net/

1.1.3. Integration Test

Integration Testing tests interface between components, interactions with different parts of a system. For example Operating system, JVM, Middle ware, inter component etc.

In our process there is no additional integration test step. Integration is occurred every day by automated build and regression test of unit test.

Compare to unit test. Unit testing test fine grained test. Integration testing tests functionalities. But in test case perspective, functionalities testing is just big component testing followed by function.

Integration Testing is done in staging system. The system has very similar environment compare to production system. Same middle ware, same os, same jvm etc.

During the integration testing, we can find a defect from integrating component.

1.2. System Test & Acceptance Test

System Test and Acceptance test is executed by QA Team.

Here is solution architecture of System & Acceptance Test environment

1.2.1. System Test

System testing tests whole system as defined by the scope of development.

Testing environment of system test are same as production environment as much as possible to minimize the risk of environment specific defect.

System test scenario is based on risk on requirement spec, business impact , technical hardness.

In system test we execute below four testing.

Ø Functional Testing

Functional Testing tests system functionalities only over production environment.

It validate functionalities from system requirement. Even if functional testing is done in integration test by development team, this system test is done by QA team with different view.

Ø Performance Testing

Performance testing test system performance such as throuput, response time etc.

To execute performance test, test scenario is very important. For example if we have 50 performance test scenario. The load for each scenarios are different. This is from “work load model”

The model is prediction that is based on previous service log or estimated usage pattern.

Ø Availability Testing

Availability testing tests fail over (Clustering failover or HA) , recovery thing etc.

Under heavy load, down the system suddenly. Validate fail over functionalities.

Restart the server again and check the recovery and transaction consistency etc.

Ø Scalability Testing

Scalability has two types.

Vertical scalability is about resources like CPU, Memory etc. By adding hardware resources in a box. Horizontal scalability tests scalability by adding new instance(process).

This test result is used for basis of capacity planning.

1.2.2. Acceptance Test

The goal of acceptance test is get a confidence of system. This testing is done by customer or end user of system. Acceptance test contains validation of deliverable , legal issue and company policy as well.

Acceptance test has

Ø User Acceptance Test (UAT)

Test system functionalities based on requirement. This test can be regression test with automated UAT tool

Ø Maintenance Test

This test is done by customer’s maintenance team. They check up deliverables for OAM and tests a OAM functionalities of the system.

Ø Contract and regulation Test

Validate legal issue and company policy.

Ø Beta Test

Beta Testing is done by system end user.

1.3. Test Case Management

Followed by V-Model, there are many test level and many test cases.

This test cases are very hard to manage. QA needs a systemized tools for handling these test cases.

Functionalities of Test case management are

Ø Manage Test case

Create and manage test case (not actual program, logical scenario, plan etc)

Categorize test case and register test case runner.

Priories test case based on priority and critical level.

Ø Run Test case

Run a test thro test case management tool.

Ø Record Test result

Record a test result in system and report it.

Ø Manage relationship with requirement and defect.

Test case is from requirement. Requirement management system manage requirement. Or we can use excel sheet either. Test case management system provides traceability from requirement to test case.

During a testing if new defect has been found, the defect is registered in defect management system. Test case management system also handles this relationship.