Best practices when using Apache openJPA entity manager

We are using Apache openJPA excessively in Apache Airavata project. All the communication between data layer and frontend is done via OpneJPA. We recently found that we are facing some memory leaks with some openJPA classes. When we analyze the memory dump, below is the culprits that we got.

Problem Suspect 1

546,326 instances of "org.apache.openjpa.kernel.FinalizingBrokerImpl", loaded by "sun.misc.Launcher$AppClassLoader @ 0x7007c7bc0" occupy 1,031,525,848 (56.39%) bytes. These instances are referenced from one instance of "java.util.concurrent.ConcurrentHashMap$Segment[]", loaded by ""

Keywords
java.util.concurrent.ConcurrentHashMap$Segment[]
sun.misc.Launcher$AppClassLoader @ 0x7007c7bc0
org.apache.openjpa.kernel.FinalizingBrokerImpl

Problem Suspect 2

546,300 instances of "org.apache.openjpa.kernel.LocalManagedRuntime", loaded by "sun.misc.Launcher$AppClassLoader @ 0x7007c7bc0" occupy 680,034,240 (37.17%) bytes. These instances are referenced from one instance of "java.util.concurrent.ConcurrentHashMap$Segment[]", loaded by ""

Keywords
java.util.concurrent.ConcurrentHashMap$Segment[]
sun.misc.Launcher$AppClassLoader @ 0x7007c7bc0
org.apache.openjpa.kernel.LocalManagedRuntime

This is how we used OpneJPA entity manager when we are getting this exception. 

And below is the memory graph with the memory leak.

We wanted to fix this memory leak so badly since it took lot of memory of the system. With help of OpenJPA community, we get to know that we are not closing OpneJPA entity manager properly in the final block. Below is the correct way to close the entity manager.


After fixing all the places, we are able to solve the memory leak issue and below is the memory graph that we get after the fix.

As you can see, it is much better memory graph compare to previous one. 

Getting rid of extra authentication window poping up in a JNLP

Today I'm going to describe you about a problem that we faced few days before and it almost took two days to come up with a solution. We have a server which is authenticated with basic authentication. There is a java client which talks to this server through a JNLP. JNLP (Java Network Launching Protocol) is a file defined with XML schema which specifies how to launch Java Web Start application. When it tries to connect to the server for the first time, it pops up an extra authentication window as below. 

This problem has faced by many people and you will see there are lots of questions posted regarding this. Most of them were not answered, as well as most of the solutions posted are not that helpful either. 

Solution is simple. You just need to add a simple property to your JNLP file. Then this annoying authentication window will go away. The magic property is this. 

As a developer when I saw this property, I thought that the value should be set as "false" in order to prevent that window. And I tested with that value, but no success. Then I came across this bug that has been reported. So the value should be set as "true".  

Hope this post will helpful to someone and hope you will not end up wasting your time like me :)  

Enable SSL with Apache HTTPD server 2.4.3

To enable SSL in apache httpd server, you need to have mod_ssl module installed. Below commands will install apache with mod_ssl.

Then you need to create key file and the cert files.

Create the key file

Create the cert file

.key file is your private key. After generating key file and the cert file, you can configure httpd.conf file.  

Configure httpd.conf

Make sure mod_ssl.so is enabled in modules list section.

Then add following lines to your httpd.conf.

Now restart apache server and now your http server will only accepts https requests. To work with apache HTTPD server 2.4.3, you need to have openssl-1.0.1e or higher. If your current openSSL version is minor to this, you will have to set LD_LIBRARY_PATH to a newer version.

Scripting with Perl

Perl is very good scripting language since it has lot of in-built modules to do lot of complex tasks that you would not be able to do with shell scripts. For example, if you want to insert data in to mysql data store or if you want to connect to a AMQP channel and get messages, you can use inbuilt modules in perl.

In this blog, I'm summarizing some very useful functionalities that I was able to do with perl.

1. Read a config file and assign values to variables from the file 
2. Connect to MySQL data store
3. Execute a query in MySQL data store
4. Creating a directory
5. Go to working directory
6. Replace a string in a file with another string

1. Read a config file and assign values to variables from the file

Suppose name of the configuration file I have is "config.ini". In here we use , Config::Simple perl module. If I have a parameter called "host" in my config file, below is how I read the config file and assign the value for host parameter to a defined variable in the perl script.

$cfg is a hash map. It will contain all the parameters that are defined in that config file.

2. Connect to MySQL data store

In here we use, DBI and DBD::mysql perl modules.        

In here, $host is where your MySQL server is running,  $userid is the database user and $passwd is the password for the given db user. Once you execute this, you will connect to the MySQL store that is running in the given host with given user name and password.

3. Execute a query in MySQL data store

Once you connect to the MySQL data store, you can execute queries.   

In here, $connection is the connection object that you get, when you connect to MySQL data store.

4. Creating a directory

To create a directory, you can use File::Path module of perl.

In here, it creates a directory called IUGateway and another directory called UBMoD inside IUGateway directory.

5. Go to working directory

Suppose you want to do a maven build by going to your source directory. Below is how you can jump to whatever directory that you wish.




6. Replace a string in a file with another string

There are lot of occasions where you want replace strings in a file with some other string. Here is a example how you can do it using perl.

If you have only single instance of that string, here is what you can use.     

Here $user_substr is the original string in the file. $http_running_user is the value that the original string will be replaced. When you replacing, make sure you don't have special characters such as ":", "/" inside that string. $httpd_conf_location is the location of the file. In perl you can run any command that you run in the command line inside "system".

If you have multiple occurrences of the same string and you want to replace all the occurrences, here is the command that you can use.     

All the above commands are extracted by googling. It took me lot of time since I'm very new to perl. I hope this post will be helpful for someone like me.

If you are a newbie to REST...

Last couple of months, I was working on implementing a REST interface to one of our java API. Here I would like to share some of my experiences I came across on that process. Being a total newbie, I had to do a lot of readings before start implementing the REST service.

As all of you aware, REST is a style of software architecture. So everything depends on how you design your resource objects. In our case, I need to come up with a REST interface for existing registry API. In registry API, what we basically deal with back-end database. It has a data structure which we design according to what kind of data will be present in a science gateway. Here is our data structure.

gateway *
  |- descriptors *
  |- published workflows *
  |- gateway-worker *
  |       |- workspace

  |       |      |- workflows
  |       |      |- projects *
  |       |             |- experiments *
  |       |                  
  |
  |
  |
  |
  |- name
  |- owner
  |
  |

users

So in the Registry API, we have lot of methods to facilitate this data structure. Basically we have several interfaces for each and every basic models. 

• ConfigurationRegistry
• DescriptorRegistry
• ProjectRegistry
• ProvenanceRegistry
• UserWorkflowRegistry
• PublishedWorkflowRegistry

So I need to come up with a REST interface for all the methods exposed by above interfaces. 

There are several problems that I need to find solutions before start implementing the REST service.

• Java interface is written without intention of using them in a REST interface in the future. Due to that most of the objects that are returned and taking as parameters are complex types which are not possible to serialize in to an XML. So I have to make those objects simple and annotated. Example class would be look like this.

  @XmlRootElement
  @XmlAccessorType(XmlAccessType.FIELD)
  public class WorkflowInstance {
  
      public WorkflowInstance() {
      }
  }
  

• In a Java API,  you can have overloaded methods. But in a REST API, you can not have two methods with the same name. 
• You should only expose that are useful for an outside user. In tha Java API, there are lot of intermediate methods. Those methods should not be exposed.   
• What is the best REST framework that we should use. There are several options you can try. Some are Jersey, Restlet, RESTEasy, Apache CXF, Apache Wink etc. They all have their pros and cons. Before selecting a framework, there are several things you need to consider.  
• Whether they have both server and client support 
• Whether they have custom annotations other than basic REST annotations
• How easy, it is to integrate (whether it is too bulky )
• Whether the REST framework license compatible with the license of your software

So in summary, if you are planning to have a REST interface, you should have your Java APIs as much as simple you can. They should not contain complex object types. And also you should make sure that the associations between the objects should not create cyclic dependencies among them.    

Some tips on working with MySQL

I was working with MySQL for last last of couple of weeks and thought of sharing some important stuff that I found to be quite useful. I'm sure you can find all these by googling, but thought of having them in a single place. Hopefully this will save someone's time in the future.

1. Start MySQL using .tar binary
2. Start MySQL with enabling logs
3. Change root password 
4. Create user and allow admin privileges
5. Allow remote IP to connect to your database
6. Check table status 
7. Repair a corrupted table
8. Create a database dump and restore to a new database
9. Pass a REGEX to select query

1. Start MySQL using .tar binary

I downloaded the mysql-5.5.27-osx10.6-x86_64.tar from official mysql site and extract it to a location. I go to extracted folder in the command line and start mysql in demon mode using following command.

sudo ./bin/mysqld_safe --basedir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64 --datadir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data/ --user=chathuri

Please note that, you need to specify admin user as --user and also make sure data directory is existing inside the extracted mysql directory.

Now you will see the following lines printed in your console and mysql server is being started in the demon mode.

121008 09:20:17 mysqld_safe Logging to '/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data//140-182-209-114.dhcp-bl.indiana.edu.err'.

121008 09:20:17 mysqld_safe Starting mysqld daemon with databases from /Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data/

Now you should be able to connect to the mysql server that is being started in the deamon mode. To connect to mysql server run the mysql script in the bin folder.

140-182-209-114:mysql-5.5.27-osx10.6-x86_64 chathuri$ ./bin/mysql

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 2

Server version: 5.5.27 MySQL Community Server (GPL)

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Now you will see that the mysql prompt is there and you can run basic mysql commands to create databases and etc.

2. Start MySQL with enabling logs

You can configure mysql to write all the logs in to a log file. Suppose I have a file called log.out inside the bin folder of mysql extracted location. To enable logs, you need to add --log in to the start command that we specified in the previous step. Now the server starting command will look like this.

sudo ./bin/mysqld_safe --log=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/bin/log.out --basedir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64 --datadir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data/ --user=chathuri

Now if you do a tail to that file, and try to connect to mysql server as before, you will see all the logs are being written to that log.out file.

/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/bin/mysqld, Version: 5.5.27-log (MySQL Community Server (GPL)). started with:

Tcp port: 0  Unix socket: (null)

Time                 Id Command    Argument

/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/bin/mysqld, Version: 5.5.27-log (MySQL Community Server (GPL)). started with:

Tcp port: 0  Unix socket: (null)

Time                   Id Command    Argument

121008  9:43:32        1 Connect    chathuri@localhost as  on 

                       1 Query      select @@version_comment limit 1

121008  9:43:47        1 Query      show databases

3. Change root password

By default mysql does not have a root user. You need to add a root user and set the root password. To set up root password, you need to use mysqladmin script located in the bin folder in the command as below.

./bin/mysqladmin -u root password NEWPASSWORD

If you already have a root user and password, you can update password using mysqladmin as well. Suppose my current root password is "abc" and I want to change it to "root123", you can run the following command in the command prompt.

./bin/mysqladmin -u root  -p'abc' password root123

Note that there are no spaces after -p in the above command.

4. Create user and allow admin privileges

You will need to add user accounts. To do that, connect to mysql server as the root user and then you can add other user accounts using following commands.

mysql> CREATE USER 'jim'@'localhost' IDENTIFIED BY 'jim123';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL ON *.* TO 'jim'@'localhost' WITH GRANT OPTION;

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql> exit;

Now try to connect to mysql server as user 'jim' and try to run some queries. 

5. Allow remote IP to connect to your database

You may need to allow some remote IPs to connect to your database. To allow that, connect to your database as an admin user and run the following commands.

Suppose I have a user called "airavata" with the password "airavata" and I want to allow 140.182.198.158 IP to connect to my database, run the below commands.

mysql> GRANT ALL PRIVILEGES ON *.* to airavata@'140.182.198.158' IDENTIFIED BY 'airavata';

mysql> FLUSH PRIVILEGES;

If you want to allow any IP to connect to your database, use the following commands. 

mysql> GRANT ALL PRIVILEGES ON *.* to airavata@'*' IDENTIFIED BY 'airavata';

mysql> FLUSH PRIVILEGES;

6. Check table status

You can check the status of a database table using check table command. Suppose I have a table called

"Host_Descriptor" and I want to check the status of that table.

mysql> check table Host_Descriptor;

+---------------------------------+-------+----------+----------+

| Table                           | Op    | Msg_type | Msg_text |

+---------------------------------+-------+----------+----------+

| persistent_data.host_descriptor | check | status   | OK|

+---------------------------------+-------+----------+----------+

1 row in set (0.00 sec)

This command is quite useful if you have a corrupted table in your database. 

7. Repair a corrupted table

If one of your table being corrupted, you can use the following command to repair it.

mysql> repair table Host_Descriptor;

8. Create a database dump and restore to a new database

In most of the times, you will need to create database dump and restore it to a new database. To create the dump file, you need to use mysqldump script inside bin folder of mysql extracted location.

Suppose I have a database called "persistent_data" and I want to restore it to newly created database called "testDB". "airavata" is an admin user of my database and its password is "airavata".

//to create the dump

140-182-209-114:mysql-5.5.27-osx10.6-x86_64 chathuri$ ./bin/mysqldump -u airavata -pairavata persistent_data > airavatadump.sql

//restore from the created dump

140-182-209-114:mysql-5.5.27-osx10.6-x86_64 chathuri$ ./bin/mysql -u airavata -pairavata testDB < airavatadump.sql

9. Pass a REGEX to select query

In mysql select query, you can pass a regular expression as well.

mysql> SELECT experiment_ID FROM Experiment_Data  WHERE username REGEXP '^admin$';

Note the two characters that are being highlighted in red (^ and $). This will return experiment ID which has a username as admin. You can pass any kind of regular expression between ^ and $ characters. 

Start Apache Derby in server mode programmatically

Derby can be started in the embedded mode and server mode. In this post, I will explain how to start derby in the server mode. You can start derby in the server mode using the script startNetworkServer which is located under bin folder of derby binary. But suppose you want to start the derby server from your program without downloading derby binary. There are two ways to do this.

1. You can create your own script which will wrap the original startNetworkServer 
2. You can start derby in the server mode programmatically

For both cases, you need the following jars to be in your classpath.

• derby.jar
• derbyclient.jar
• derbynet.jar
• derbytools.jar

Please note that you can't have the jars with their versions. If you are adding those jars using a maven project, you will need to rename them by removing the version part of those jars. 

1. Write a script to wrap startNetworkServer 

Suppose I create a script file called derby.sh. Here is what is inside my script. 

export DERBY_HOME="path where the above jars located"
./startNetworkServer $*

In this I have a copy of startNetworkServer at the same location where my derby.sh located. If you run the derby.sh and if the above jars in your classpath, you will be able to start derby in the server mode in it's default port 1527.

2. Start derby in the server mode programmatically

Following is the code snippet that allow you to start derby in the server mode programmatically. Please note that you need to have above jars in your classpath.

System.setProperty("derby.drda.startNetworkServer", "true");
// start derby in port 20000
// db user - airavata
// db user password - airavata
 server = new NetworkServerControl(InetAddress.getByName("localhost"), 
                                   20000, 
                                   "airavata", 
                                   "airavata");
java.io.PrintWriter consoleWriter = new java.io.PrintWriter(System.out, true);
server.start(consoleWriter);


//if you want to shut down the server
// server.shutdown();

Now the derby will start on port 20000.

Hope this will helpful and save some time of you for searching how to do that.