If you are a newbie to REST...

Last couple of months, I was working on implementing a REST interface to one of our java API. Here I would like to share some of my experiences I came across on that process. Being a total newbie, I had to do a lot of readings before start implementing the REST service.

As all of you aware, REST is a style of software architecture. So everything depends on how you design your resource objects. In our case, I need to come up with a REST interface for existing registry API. In registry API, what we basically deal with back-end database. It has a data structure which we design according to what kind of data will be present in a science gateway. Here is our data structure.

gateway *
  |- descriptors *
  |- published workflows *
  |- gateway-worker *
  |       |- workspace

  |       |      |- workflows
  |       |      |- projects *
  |       |             |- experiments *
  |       |                  
  |
  |
  |
  |
  |- name
  |- owner
  |
  |

users

So in the Registry API, we have lot of methods to facilitate this data structure. Basically we have several interfaces for each and every basic models. 

• ConfigurationRegistry
• DescriptorRegistry
• ProjectRegistry
• ProvenanceRegistry
• UserWorkflowRegistry
• PublishedWorkflowRegistry

So I need to come up with a REST interface for all the methods exposed by above interfaces. 

There are several problems that I need to find solutions before start implementing the REST service.

• Java interface is written without intention of using them in a REST interface in the future. Due to that most of the objects that are returned and taking as parameters are complex types which are not possible to serialize in to an XML. So I have to make those objects simple and annotated. Example class would be look like this.

  @XmlRootElement
  @XmlAccessorType(XmlAccessType.FIELD)
  public class WorkflowInstance {
  
      public WorkflowInstance() {
      }
  }
  

• In a Java API,  you can have overloaded methods. But in a REST API, you can not have two methods with the same name. 
• You should only expose that are useful for an outside user. In tha Java API, there are lot of intermediate methods. Those methods should not be exposed.   
• What is the best REST framework that we should use. There are several options you can try. Some are Jersey, Restlet, RESTEasy, Apache CXF, Apache Wink etc. They all have their pros and cons. Before selecting a framework, there are several things you need to consider.  
• Whether they have both server and client support 
• Whether they have custom annotations other than basic REST annotations
• How easy, it is to integrate (whether it is too bulky )
• Whether the REST framework license compatible with the license of your software

So in summary, if you are planning to have a REST interface, you should have your Java APIs as much as simple you can. They should not contain complex object types. And also you should make sure that the associations between the objects should not create cyclic dependencies among them.    

Some tips on working with MySQL

I was working with MySQL for last last of couple of weeks and thought of sharing some important stuff that I found to be quite useful. I'm sure you can find all these by googling, but thought of having them in a single place. Hopefully this will save someone's time in the future.

1. Start MySQL using .tar binary
2. Start MySQL with enabling logs
3. Change root password 
4. Create user and allow admin privileges
5. Allow remote IP to connect to your database
6. Check table status 
7. Repair a corrupted table
8. Create a database dump and restore to a new database
9. Pass a REGEX to select query

1. Start MySQL using .tar binary

I downloaded the mysql-5.5.27-osx10.6-x86_64.tar from official mysql site and extract it to a location. I go to extracted folder in the command line and start mysql in demon mode using following command.

sudo ./bin/mysqld_safe --basedir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64 --datadir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data/ --user=chathuri

Please note that, you need to specify admin user as --user and also make sure data directory is existing inside the extracted mysql directory.

Now you will see the following lines printed in your console and mysql server is being started in the demon mode.

121008 09:20:17 mysqld_safe Logging to '/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data//140-182-209-114.dhcp-bl.indiana.edu.err'.

121008 09:20:17 mysqld_safe Starting mysqld daemon with databases from /Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data/

Now you should be able to connect to the mysql server that is being started in the deamon mode. To connect to mysql server run the mysql script in the bin folder.

140-182-209-114:mysql-5.5.27-osx10.6-x86_64 chathuri$ ./bin/mysql

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 2

Server version: 5.5.27 MySQL Community Server (GPL)

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Now you will see that the mysql prompt is there and you can run basic mysql commands to create databases and etc.

2. Start MySQL with enabling logs

You can configure mysql to write all the logs in to a log file. Suppose I have a file called log.out inside the bin folder of mysql extracted location. To enable logs, you need to add --log in to the start command that we specified in the previous step. Now the server starting command will look like this.

sudo ./bin/mysqld_safe --log=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/bin/log.out --basedir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64 --datadir=/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/data/ --user=chathuri

Now if you do a tail to that file, and try to connect to mysql server as before, you will see all the logs are being written to that log.out file.

/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/bin/mysqld, Version: 5.5.27-log (MySQL Community Server (GPL)). started with:

Tcp port: 0  Unix socket: (null)

Time                 Id Command    Argument

/Users/chathuri/airavata/tools/mysql-5.5.27-osx10.6-x86_64/bin/mysqld, Version: 5.5.27-log (MySQL Community Server (GPL)). started with:

Tcp port: 0  Unix socket: (null)

Time                   Id Command    Argument

121008  9:43:32        1 Connect    chathuri@localhost as  on 

                       1 Query      select @@version_comment limit 1

121008  9:43:47        1 Query      show databases

3. Change root password

By default mysql does not have a root user. You need to add a root user and set the root password. To set up root password, you need to use mysqladmin script located in the bin folder in the command as below.

./bin/mysqladmin -u root password NEWPASSWORD

If you already have a root user and password, you can update password using mysqladmin as well. Suppose my current root password is "abc" and I want to change it to "root123", you can run the following command in the command prompt.

./bin/mysqladmin -u root  -p'abc' password root123

Note that there are no spaces after -p in the above command.

4. Create user and allow admin privileges

You will need to add user accounts. To do that, connect to mysql server as the root user and then you can add other user accounts using following commands.

mysql> CREATE USER 'jim'@'localhost' IDENTIFIED BY 'jim123';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL ON *.* TO 'jim'@'localhost' WITH GRANT OPTION;

Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.00 sec)

mysql> exit;

Now try to connect to mysql server as user 'jim' and try to run some queries. 

5. Allow remote IP to connect to your database

You may need to allow some remote IPs to connect to your database. To allow that, connect to your database as an admin user and run the following commands.

Suppose I have a user called "airavata" with the password "airavata" and I want to allow 140.182.198.158 IP to connect to my database, run the below commands.

mysql> GRANT ALL PRIVILEGES ON *.* to airavata@'140.182.198.158' IDENTIFIED BY 'airavata';

mysql> FLUSH PRIVILEGES;

If you want to allow any IP to connect to your database, use the following commands. 

mysql> GRANT ALL PRIVILEGES ON *.* to airavata@'*' IDENTIFIED BY 'airavata';

mysql> FLUSH PRIVILEGES;

6. Check table status

You can check the status of a database table using check table command. Suppose I have a table called

"Host_Descriptor" and I want to check the status of that table.

mysql> check table Host_Descriptor;

+---------------------------------+-------+----------+----------+

| Table                           | Op    | Msg_type | Msg_text |

+---------------------------------+-------+----------+----------+

| persistent_data.host_descriptor | check | status   | OK|

+---------------------------------+-------+----------+----------+

1 row in set (0.00 sec)

This command is quite useful if you have a corrupted table in your database. 

7. Repair a corrupted table

If one of your table being corrupted, you can use the following command to repair it.

mysql> repair table Host_Descriptor;

8. Create a database dump and restore to a new database

In most of the times, you will need to create database dump and restore it to a new database. To create the dump file, you need to use mysqldump script inside bin folder of mysql extracted location.

Suppose I have a database called "persistent_data" and I want to restore it to newly created database called "testDB". "airavata" is an admin user of my database and its password is "airavata".

//to create the dump

140-182-209-114:mysql-5.5.27-osx10.6-x86_64 chathuri$ ./bin/mysqldump -u airavata -pairavata persistent_data > airavatadump.sql

//restore from the created dump

140-182-209-114:mysql-5.5.27-osx10.6-x86_64 chathuri$ ./bin/mysql -u airavata -pairavata testDB < airavatadump.sql

9. Pass a REGEX to select query

In mysql select query, you can pass a regular expression as well.

mysql> SELECT experiment_ID FROM Experiment_Data  WHERE username REGEXP '^admin$';

Note the two characters that are being highlighted in red (^ and $). This will return experiment ID which has a username as admin. You can pass any kind of regular expression between ^ and $ characters. 

Start Apache Derby in server mode programmatically

Derby can be started in the embedded mode and server mode. In this post, I will explain how to start derby in the server mode. You can start derby in the server mode using the script startNetworkServer which is located under bin folder of derby binary. But suppose you want to start the derby server from your program without downloading derby binary. There are two ways to do this.

1. You can create your own script which will wrap the original startNetworkServer 
2. You can start derby in the server mode programmatically

For both cases, you need the following jars to be in your classpath.

• derby.jar
• derbyclient.jar
• derbynet.jar
• derbytools.jar

Please note that you can't have the jars with their versions. If you are adding those jars using a maven project, you will need to rename them by removing the version part of those jars. 

1. Write a script to wrap startNetworkServer 

Suppose I create a script file called derby.sh. Here is what is inside my script. 

export DERBY_HOME="path where the above jars located"
./startNetworkServer $*

In this I have a copy of startNetworkServer at the same location where my derby.sh located. If you run the derby.sh and if the above jars in your classpath, you will be able to start derby in the server mode in it's default port 1527.

2. Start derby in the server mode programmatically

Following is the code snippet that allow you to start derby in the server mode programmatically. Please note that you need to have above jars in your classpath.

System.setProperty("derby.drda.startNetworkServer", "true");
// start derby in port 20000
// db user - airavata
// db user password - airavata
 server = new NetworkServerControl(InetAddress.getByName("localhost"), 
                                   20000, 
                                   "airavata", 
                                   "airavata");
java.io.PrintWriter consoleWriter = new java.io.PrintWriter(System.out, true);
server.start(consoleWriter);


//if you want to shut down the server
// server.shutdown();

Now the derby will start on port 20000.

Hope this will helpful and save some time of you for searching how to do that.