Showing posts with label DB2. Show all posts
Showing posts with label DB2. Show all posts

Saturday, November 26, 2011

Mapping SQL terminology with IBM i terminology

As a IBMi developer coming with background of modern SQL RDBMS, I was always try to "translate" iSerience term like "physical file", "logical file" into SQL terms. Here, I found a table, which maps these terms.

Mapping SQL terminology with IBM i terminology
SQL termIBM i term
TABLE PHYSICAL FILE
ROW RECORD
COLUMN FIELD
INDEX KEYED LOGICAL FILE
VIEW NON-KEYED LOGICAL FILE
SCHEMA LIBRARY
LOG JOURNAL
ISOLATION LEVEL COMMITMENT CONTROLE LEVEL
PARTITION MEMBER


The table is cited from document IBM DB2 for i indexing methods and strategies

Friday, November 18, 2011

IBM DB2 extension has no limitation on number of opened persistent PHP DB2 connection.

To further study how to optimize PHP DB2 connection, I looked into PHP DB2 extension again recently. I want to find out the best way to use persist connection with considering about ODP reusing.

After reading the source code of PHP DB2 extension, I noticed that there is no limitation of maximum number of pconnection from PHP DB2 extension. In the C code, we can see only UserID, password, and Database name are used to compose hash code. You locate the place by search following code in the source code.

sprintf(hKey, "__db2_%s.%s.%s", uid, database, password);

This is interesting as it implies that I should avoid allowing huge number of different user profile to be used inside one Zend Server.

I am thinking that we will have difficult to reuse ODP when only userId, password, and database name are used to generate hash code for pconnect because ODP will be reused based on not only user name and password. For example, I think the ODP will be rebuilt if different requests use same User Name and password but different library list to get pconnection. PHP Db2 extension allows us to pass in library list. So, if developers, for some reason, happen to pass in different library list to run query on same library, the ODP will be rebuilt for sure.

Some other things I do not have answers are,

1) What happens to pconnection if its associated QSQSRVR job is terminated for some reason?

2) I saw something about ODBC version occur in C code. Does PHP DB2 extension works with ODBC in any way?

Tuesday, November 8, 2011

A simple example about creating DB2 SQL stored procedure to return multiple Result Set

There are plenty of examples showing how to get multiple result sets in languages like Java or php. But, there are not so many simple stored procedure examples, which return multiple result sets. Here is a very simple DB2 stored procedure, which return two result set. Just following the steps, you will have a multiple results DB2 stored procedure for testing.

Step 1: Create a table:
CREATE TABLE DEPT
     (DEPTNO   CHAR(3)     NOT NULL,
      DEPTNAME VARCHAR(36) NOT NULL,
      PRIMARY KEY(DEPTNO))

Step 2: Populate data into table:
INSERT INTO DEPT (DEPTNO, DEPTNAME)
     VALUES ('B11', 'PURCHASING'),
            ('E41', 'DATABASE ADMINISTRATION') ;

Step 3: Create SQL store procedure:
CREATE PROCEDURE TESTMULTIRS
--We do not use this input parameter.
(IN i_cmacct CHARACTER(5)) 
RESULT SETS 2
LANGUAGE SQL
BEGIN 

DECLARE csnum INTEGER;

--Declare serial cursors as serial cursor consume less resources
--and we do not need rollable cursor.
DECLARE getDeptNo CHAR(50); --Be careful about estimated length here.
DECLARE getDeptName CHAR(200);
DECLARE c1 CURSOR WITH RETURN FOR s1; 
DECLARE c2 CURSOR WITH RETURN FOR s2;

SET getDeptNo = 'SELECT DEPTNO FROM DEPT';
SET getDeptName = 'SELECT DEPTNAME FROM DEPT'; 

PREPARE s1 FROM getDeptNo;
OPEN c1;

PREPARE s2 FROM getDeptName;
OPEN c2;

END;

Step 4: Call stored procedure in iNavigator:
call testmultirs('jia');

Then we can see outpu as below in inavigator:
and

Wednesday, October 19, 2011

Why does DB2 not use Index I created?!

Recently, I heard a guy cried that the DB2 for iSeriese does not use index he created. Then, I asked him if he has check the size of table. The answer is NO!

So, it seems that some developers do not aware that a RDBMS' query optimizer may not use index though indexes are created to optimize SQL query speed. In fact, DBMS normally have its own algorithm to decide when to use index scan or table scan in a SQL query. In general, DBMS should use index scan instead of full table scan only when size of index is smaller than size of table. Otherwise, there is no point for DBMS to use index. This is true for DB2, MySQL, Oracle, MS SQL etc.

How to create index or combined index and how many index should be created for a table is big topic. But, we shall know that index may not be always used when SQL query optimizer thinks it is not worthy. This is most likely true for a new launched application, which has not received/created many data in tables. So, do not cry that DBMS is suck that it does not use indexes. Check your table size first. Then, check your DBMS configuration. For example, MySQL has configurable threshold value, which determine when query optimizer can use index instead of full table scan. But, you can not force SQL query optimizer to use index. The most we can do is configuring table to prefer index scan than table scan.

Here is a more detailed discussion about "magic number".

Wednesday, June 8, 2011

how to convert TIMESTAMP format to different date format

It is very often that we need to get specific format of date from TIMESTAMP column. We can certainly get timestamp as string and do some string operation on it to compose a new string with desired formated. However, it will be just nice to be able to get what I want directly from SQL query output. Here is a brief conclusion on how to get formated Date string from TIMESTAMP.

Basically, two DB2 functions are used. DATE scalar function and CHAR scalar function. In CHAR function, we can specify five different format: iso, usa, eur, jis, local.

Below is sample output for different format,

SELECT CURRENT_TIMESTAMP, char(date(CURRENT_TIMESTAMP), eur) 
  FROM SYSIBM.SYSDUMMY1; 
Output EUR format: 
2011-05-28 15:36:12.178455 28.05.2011

SELECT CURRENT_TIMESTAMP, char(date(CURRENT_TIMESTAMP), usa) 
  FROM SYSIBM.SYSDUMMY1; 
Output USA format: 
2011-05-28 15:38:57.487165 05/28/2011

SELECT CURRENT_TIMESTAMP, char(date(CURRENT_TIMESTAMP), jis) 
  FROM SYSIBM.SYSDUMMY1; 
Output JIS format: 
2011-05-28 15:40:00.455681 2011-05-28

SELECT CURRENT_TIMESTAMP, char(date(CURRENT_TIMESTAMP), iso) 
  FROM SYSIBM.SYSDUMMY1; 
Output ISO format: 
2011-05-28 15:41:12.159374 2011-05-28


SELECT CURRENT_TIMESTAMP, 
char(date (CURRENT_TIMESTAMP), local)
  FROM SYSIBM.SYSDUMMY1; 
Output LOCAL format: 
2011-05-28 15:42:41.802835 05/28/11

Wednesday, May 18, 2011

DB2 for i5/OS V5R4 have no buit-in UPSERT support.

Recently, I need to do UPDATE_or_INSERT (UPSERT) in DB2 for i5/OS V5R4. But, this version DB2 does not have built-in UPSERT function.

In MySQL, I have two ways to implement UPSERT. One is using REPLACE INTO. The other is using ON DUPLICATE KEY UPDATE

It seems that MERGE statement can be used for UPSERT purpose though it is designed for different purpose. But, MERGE statement is not support DB2 v8 for i5/OS V5. It requires 6.1 of the OS.

The following is an article about how to write SQL for both DB2 and MySQL. I bookmark it here as reference: Writing SQL for both MySQL and DB2

Tuesday, April 12, 2011

IBM i5/OS V5R4 prestart job and database connection tuning.

Database connection tuning is an important part of application performance tuning, particularly, when applications uses connection pool. I noticed that some iSeries user does not care about this too much as I can not get answer when I ask them what is the limitation of database connection iSeries can support. The answer may not be simple. Or it can be as simple as no limitation as the default maximum number of QSQSRVR job and QZDASOINIT job is *NOMAX. This wont be good answer as the number of job should be optimized to a number, which fits your applications requirement.

On IBM i5/OS, there are two jobs used by database connection. They are SQL server mode prestarted job QSQSRVR and Database server prestart job QZDASOINIT job. QSQSRVR job is used when Database connection is coming through Command Level Interface (CLI). For example, PHP DB2 extension uses CLI to connect to DB2. Therefore, its corresponding job is QSQSRVR. Native iSeries JDBC driver (type 2) is using QSQSRVR job too. QZDASOINIT job is used when the connection is from ODBC or type 4 JDBC. Therefore, access from jt400.jar goes through QZDASOINIT job. When the type 4 JDBC is configured to use SSL, QZDASSINIT job is used instead of QZDASOINIT.

Both QSQSRVR job and QZDASOINIT job have their deamon job, which creates them. QSQSRVR job's deamon job is QYPSJSVR. Command WRKJOB JOB(QYPSJSVR/QYPSJSVR) can be used to do further investigate. For QSQSRVR job itself, we can use command WRKSBSD SBSD(QSYS/QSYSWRK) or command WRKJOB JOB(QSQSRVR) to do deeper investigation. For QZDASOINIT job, its deamon job is QZDASRVSD. We can use command WRKJOB JOB(QZDASRVSD) or WRKSBSD SBSD(QSYS/QSERVER) work with it further. Command CHGPJE can be used to change the configuration of prestart jobs. Also, don't forget that iNavigator is a convenience tool for us to investigate too: Work Management -> Server Jobs . The database server prestart jobs (QZDAINIT, QZDASOINIT, and QZDASSINIT) by default are shipped to run in subsystem QSERVER QZDAINIT) and QUSRWRK (QZDASOINIT and QZDASSINIT).

Also, there is another prestart job called QRWTSRVR, which is used by DB2 Connect driver. DB2 Connect uses DRDA protocol that is used to reduce the cost and complexity of accessing data in different DB2 supporting DRDA. In fact, in Work Management menu of iNavigator, you can see that iNavigator uses QRWTSRVR job. However, its license fee is expensive for enterprise edition. QRWTSRVR job typically run in QSYSWRK subsystem.

I made a diagram to present these three ways to connect to DB2 for iSeries. It is helpful for having a clear concept how the database connection is connect to DB2 and what is under those licensed and unlicensed program indeed. For example, people always ask why IBM gives type 4 JDBC driver for free and charge DB2 Connect. From this post, we can see jt400.jar, php db2 extension, and DB2 Connect use different protocol connect to DB2 for iSeries and different server jobs serve them. Furthermore, different protocols are designed for different usage. For example, DRDA is designed to support topology in complicate distributed network environment.


And, in Zend server for IBMi, there is a package called i5 Toolkit. It can be used to connect to i5/OS and access data too as DB2 for i5/OS is part of OS. A daemon called i5_COMD server is shipped together with Zend Server. i5 toolkit client communicate with it-COMD server for accessing i5 Data object. However, I feel it emulates 5250 terminal protocol and uses Remot Command Server prestart job QZRCSRVS. I am not able to discuss it here. I will spend some time to dig it later.

Here are some links from IBM for prestarted job on i5/OS V5R4 and checking job log of the appropriate "Host Server" and a PTF for improved management of QSQSRVR jobs.

Thursday, April 7, 2011

SQL naming must be used when calling user defined function in SQL Stored procedure on UDB DB2 for i5/OS

Today, I got a small tip, which let me aware that when SQL naming must be used when calling a Java User Defined Function in SQL stored procedure.

I wrote a UDF in Java for DB2 for i5/OS. A dummy code sample like below,

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

/**
* @author Yiyu Jia
*/
public class DummyUDF {

 boolean debugtableOn = true;
 public static String helloJia(String name)throws SQLException, Exception {
  //Connection con = DriverManager.getConnection("jdbc:default:connection");
  return "hello" + name;
 }
}

Then, we register it as DB2 user defined function as below,
CREATE FUNCTION YOURSCHEMA.DummyUDF ( 
NAME VARCHAR(500) 
) 
RETURNS VARCHAR(1000)   
LANGUAGE JAVA 
SPECIFIC YOURSCHEMA.DummyUDF 
NOT DETERMINISTIC 
MODIFIES SQL DATA  
RETURNS NULL ON NULL INPUT 
EXTERNAL NAME 'DummyUDF.helloJia' 
PARAMETER STYLE JAVA ;

However, when I call it in either iNavigator or SQL stored procedure, I was reported error message, which says
SQL State: 42704
Vendor Code: -204
Message: [SQL0204] DummyUDF in *LIBL type *N not found. ...


The statement used to call DummyUDF is as below,
values(YOURSCHEMA/DummyUDF('Yiyu'));
Since my iNavigator is configured to use system naming. I use "/" to link the schema name and function name. However, this wont work!After research and trying, I found it is extremely simply to solve this problem. That is, using SQL naming instead of System naming to call UDF though iNavigator is set to use system naming already. Below code works,
values(YOURSCHEMA.DummyUDF('Yiyu'));

Tuesday, March 22, 2011

Zend Server for IBM iSeries platform does not support multiple persistent connection to DB2

_
_
Edit: This post's content may only apply to i5/OS V5R4 without latest PTF. Please refer to comments for detailed info.

Persistent connection in PHP has its advantage and disadvantage. According to my observation about creating DB2 connection on i5/OS, time used to set up connection via normal connection and persistent connection can be 50 ~ 100 times different. However, persistent connection has its disadvantage as well. Some php programmer even call it evil. Also, it might not be worthy to take risk from persistent connection if the access traffic is not so high.

In a brief way to describe, the causer for persistent connection has disadvantage is that PHP itself is a stateless script language and database connection is usually stateful. That is, php script's life time only exists from the request coming and request end. Meanwhile, database connection will hold some state like table lock, user defined variables etc. So, the problem will occur when a php script use a persistent connection as a fresh new connection and it is not. PHP persistent does not support a kind of "private" connection, which will be used by one user session only. This kind private connection implementation can be found in easyComm i5 toolkit.

Back to what I observed recently. I find that Zend Server (ver 5.0.4) for IBM i5/OS V5R4 does not supports having multiple persistent connection in one request to php scripts. Below is a simple testing code. When you run it, you can see that the output Current Schema is not ALWAYS as what you specified in db2 connection string. I guess the hash function used Zend Server for resource manager may has some problem. Or, DB2 extension has a bug if it does not intend to support only one persistent connection in one PHP script request.

Change the db2_pconnect to db2_connect to see different output. You need to run this script many times to see that you can not ALWAYS get the right schema as your current schema. Most likely, you need to take 5 minutes to 10 minutes break during the testing and see the wrong result to come out.


<?php 

$sysStr = 'select CURRENT SCHEMA from SYSIBM.SYSDUMMY1 ';

$params = array(
 'username' => 'accountNameOne', 
 'password' => 'accountPwdOne',
 'dbname' => '*LOCAL', 
 'driver_options' => array('i5_lib' => 'libraryOne')
);
$connOne = db2_pconnect($params['dbname'], $params['username'], 
$params['password'], $params['driver_options']);
$sysStmt = db2_prepare($connOne, $sysStr);
db2_execute($sysStmt);
$returnObj = db2_fetch_object($sysStmt);
var_dump($returnObj);

$params2 = array(
 'username' => 'accountNameTwo', 
 'password' => 'accountPwdTwo', 
 'dbname' => '*LOCAL', 
 'driver_options' => array('i5_lib' => 'libraryTwo')
);
$connTwo = db2_pconnect($params2['dbname'], $params2['username'], 
$params2['password'], $params2['driver_options']);
$sysStmt2 = db2_prepare($connTwo, $sysStr);
db2_execute($sysStmt2);
$returnObj2 = db2_fetch_object($sysStmt2);
var_dump($returnObj2);

Wednesday, March 16, 2011

buit-in datatype supported by DB2 for i5/OS and MySQL

To complete one of my work, I need to know list of data type supported by RDBMS. So, I wrote a simple Java application to fetch it. Below is java code and output.

Db2MetaData.java using jt400.jar. (This is just a quick work)
package jia.blog.db2;

import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Yiyu Jia
 */
public class Db2MetaData {
   Connection conn = null;
   DatabaseMetaData dmd = null;

    public Db2MetaData() {
     
    }

  public void openConnection () {

      try {                  
          String url = "jdbc:as400://172.0.0.1"; //change ip address
          DriverManager.registerDriver(new com.ibm.as400.access.AS400JDBCDriver());
          conn = DriverManager.getConnection(url, "accountName", "password");
      }
    catch (Exception e) {
         Logger.getLogger(Db2MetaData.class.getName()).log(Level.SEVERE, null, e);
       }
  }  

 
  public void closeConnection () {

    if (conn != null) {
       try {
           conn.close();           
           System.out.println("Connection closed");
           }
       catch(SQLException e) {
          Logger.getLogger(Db2MetaData.class.getName()).log(Level.SEVERE, null, e);
          }
       }
  }  


    public void displayMaxConnection() {
        try {
            DatabaseMetaData metadata = conn.getMetaData();
            int maxConnection = metadata.getMaxConnections();
            System.out.println("Maximum Connection = " + maxConnection);
        } catch (SQLException ex) {
            Logger.getLogger(Db2MetaData.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
    
    public void displayTypeInfo() {
        try {
            DatabaseMetaData metadata = conn.getMetaData();
            ResultSet resultSet = metadata.getTypeInfo();
            while (resultSet.next()) {
                String typeName = resultSet.getString("TYPE_NAME");
                System.out.println(typeName);
            }
            resultSet.close();
        } catch (SQLException ex) {
            Logger.getLogger(Db2MetaData.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

  public static void main(String args[])  {
      Db2MetaData db2Meta = new Db2MetaData();
      db2Meta.openConnection();
      db2Meta.displayMaxConnection();
      db2Meta.displayTypeInfo();
      db2Meta.closeConnection();
  }

}

Supported data type printed by JDBC
DB2 i5 R6V1 MySQL 5.5.8

CHAR
CHAR() FOR BIT DATA
GRAPHIC
LONG VARGRAPHIC
LONG VARCHAR FOR BIT DATA
VARCHAR
VARCHAR() FOR BIT DATA
VARGRAPHIC
DATE
TIMESTAMP
TIME

DECIMAL
DOUBLE
FLOAT
INTEGER
NUMERIC
REAL
SMALLINT
BIGINT
ROWID
DECFLOAT
DATALINK
BLOB
CLOB
DBCLOB

BIT
BOOL
TINYINT
TINYINT UNSIGNED
BIGINT
BIGINT UNSIGNED
LONG VARBINARY
MEDIUMBLOB
LONGBLOB
BLOB
TINYBLOB
VARBINARY
BINARY
LONG VARCHAR
MEDIUMTEXT
LONGTEXT
TEXT
TINYTEXT
CHAR
NUMERIC
DECIMAL
INTEGER
INTEGER UNSIGNED
INT
INT UNSIGNED
MEDIUMINT
MEDIUMINT UNSIGNED
SMALLINT
SMALLINT UNSIGNED
FLOAT
DOUBLE
DOUBLE PRECISION
REAL
VARCHAR
ENUM
SET
DATE
TIME
DATETIME
TIMESTAMP
Below is a picture linked from IBM web site. It listed out built-in data type supported by DB2 for i5/OS RvV4.source
Here are links for DB2 data type to jdbc data type mapping and MySQL data type to JDBC data type mapping

Tuesday, February 22, 2011

PHP DB2 extension should support features like timeout checking.

I ran into very troublesome problem when I use DB2 connection from PHP to call RPG stored procedure. What happened is described in the following diagram.
  
As you can see, if calling a bad configured RPG stored procedure from PHP through DB2 extension on the same machine where Zend server and DB2 are installed, Zend Server may hang and it can not even be restarted from Zend Menu in green screen of i5/OS.

The appeared phenomenon is that QSQSRVR jobs is under MSGW status and seems the waiting status will never be broken. I am not sure which part exactly causes this. But, I think adding a time-out check mechanism in PHP DB2 extension on IBM i could be considered. The detailed discussion can be found from Zend Forums.

Friday, January 21, 2011

Did MySQL for IBM i5/OS platform die?

On Zend Forum, there is a hot discussion about MySQL announce EOL support for MySQL on IBM i. This is the link to Zend forum post. This is MySQL EOL announce. However, after reading those discussion, I feel some of them are worrying something which they do not need to worry about.

I think the EOL support only means that there will be newer version of MySQL for IBM i platform. It does not mean developer can not use existing latest version of MySQL on IBM i. The only thing missed  is that you can not BUY support from MySQL because they do not supply this service. However, will this be big issue? I do not think it is going to big problem because many people want to use MySQL because it is FREE. I think I will be happy with MySQL as long as its features are enough for my application and it is in a product quality.

So, I would like to ask people, who worry about the EOL support for MySQL on IBM i, to think about one thing. What do they want, always upgrading to latest version of MySQL or using one stable version to build their own product? Obviously, most of people's target is the second one: build their own application on top of free MySQL RDBMS. Therefore, I would like to say MySQL does not have its EOL on IBM i. Developers can still use it as long as they are satisfied with its quality and features.

I am interesting in MySQL on IBM i is because I am looking for a way to walk around the DB2 Connect license fee issue. To remotely connect to a IBM i DB2, developer need to buy DB2 Connect license from IBM if they are not using free type 4 JDBC library like jt400. Therefore, I am thinking that let PHP script to remotely connect to MySQL server with DB2i storage engine might be a good way to walk around this license issue. With this solution, PHP can write and read data which is stored in DB2. And the data can be subsequently accessed from DB2 side like RPG programm. But, the limitation will be that developer can only write stored procedure in MySQL and this stored procedure can be remotely accessed by PHP through MySQL interface.

Wednesday, January 19, 2011

Some notes about QSQSRVR job on IBM i5 OS V5R4


QSQSRVR runs in subsystem QSYSWRK. But, with "V5R4 PTF—SI33298" or "V6R1 PTF—SI33949", it is possible to configure QSQSRVR to run in different subsystem. I copied some commands here for my convenience.

There are two ways to request to use SQL Server Mode in programm. One is, in SQL CLI API, call SQLSetEnvAttr() via the SQL_ATTR_SERVER_MODE attribute. The other is using QWTCHGJB() Work Management API, via the "Server mode for Structured Query Language" key 1922 within the JOBC0200 format.

By default, five QSQSRVR jobs are active. Two more QSQSRVR jobs will be created if fewer than jobs are unused. The following command is to increase the initial number of jobs, threshold, and job number increasing interval.
CHGPJE SBSD(QSYS/QSYSWRK) PGM(QSYS/QSQSRVR)

Management Central starts 18 SQL Server Mode connections. Each of the two servers (QYPSSRV and QYPSJSVR) currently establishes nine connections with a QSQSRVR job to process its SQL requests to these databases. The following commands are used to find the controlling job:

1)Using command line: WRKJOB JOB(QYPSJSVR/QYPSJSVR)
2)Or using iNavigator: Work Management --> Subsystem --> Active Subsystems --> Qsyswrk
3)Using SQL monitor: STRDBMON OUTFILE(MYLIB/QSQMON1) JOB(*ALL/*ALL/QSQSRVR)

In order to run QSQSRVR job in same subsystem as application runs, we need to do following steps,

1)install PTF—SI33298 for i5 OS V5R4 or PTF—SI33949 for i5 OS V6R1. Use command WRKPTFGRP to list all installed PTF.

2) set environment variable QIBM_SRVRMODE_SBS to be "*SAME" or "". Here is the command to add this environment variable, ADDENVVAR ENVVAR(QIBM_SRVRMODE_SBS) VALUE('*SAME') LEVEL(*SYS). To remove environment variable, command RMVENVVAR is used.

Finally, can I use this to optimize Web application running inside Zend server? It seems I can not because the QIBM_SRVRMODE_SBS environment variable is ignored whenever the name of the application subsystem is 'QHTTPSVR' or 'ZEND'. However, this environment variable should be seen in subsystem where application use native JDBC driver to connect to DB2. PHP db2 extension and native JDBC driver (in IBM Java toolkit) connect to DB2 through DB2 SQL server mode. Using DB2 SQL Server Mode, each connection has a QSQSRVR job. QSQSRVR job inherits attributes user profile and running priority from connection request initializing.

references
1) DB2 for i5/OS: SQL Server Mode Primer
2) TechTip: Grab Control of the DB2 QSQSRVR Jobs

Sunday, December 12, 2010

Zend Framework resource plug in and avoid using Zend_Application_Resource_Multidb in wrong way

Personally, I do not think it has significant advantage to use configuration file (application.ini) from Zend Framework because 1) I do not see big difference between PHP script file and plain text configuration file in PHP enviroment. Probably, placing configuration directly in PHP script is better in terms of performance, easy maintenance and flexible. 2) the cost of following design of Zend Framework, for example resource plugin, is that programmer has to be very careful about each components' life cycle in the system. In other words, the programmer has to understand how the thing happens insight.

Here, I have an example. I was given an authentication library as resource plugin for Zend Framework. It asks for multidb resource configuration in application.ini. It has two databases resources. The second one gets the database connection confidential parameters by calling through the first one (default db).

After checking the source code of Zend Framework. We can see that bootstrap will initialize all database adapters from multidb resource specified in application.ini. So, here is the problem. What will happen to the second database connection in multidb when bootstrap tried to initialize it without properly database connection parameters? Fortunately, Zend Framework does not really call php database connection function ( in my case db2_(p)connect) when the DB adapter is initialized. Database connection function call is only called when php script tries to get connection from DB adapter.

However, it is really confusing to specify wrong connection parameters in application.ini as multidb plugin. Then, correcting parameters in some place else. Using multidb as resource plugin might be good practice when multiple database adapters are needed in multiple places in the code and all connection parameters are known. Otherwise, it is neat to create database connection when it is needed.

Simple should be always one of main considerations of software design. Especially, for an interpret language like PHP, this could be more important.

Friday, December 3, 2010

an opened cursor can not survive between two different database connection session

I was shown a DB2 stored procedure design, which has a input parameter called "FIRST". I feel it is odd. After further enquiry, I confirm it is a wrong design.

What they expected is as below, 1) if the "FIRST" parameter is past in, it means it is the first time for calling application to call this stored procedure. 2) Therefore, the stored procedure will run and open the cursor. 3) if the "FIRST" parameter is false, it means it is not the first time for calling application to call this stored procedure. Therefore the stored procedure will "reuse" the cursor opened in the first calling.

This sounds wonderful as they told me they can avoid opening cursor frequently. Then, they can gain performance improve from it. However, I will say this is a bad design if it is not wrong because,

1) an opened cursor obviously consumes resource from DB2 server. DB2 memory cache size is limited. Furthermore, the chance for concurrency access to DB2 will be increased if each opened cursor last too long time.

2) an opened cursor should not be able to survive between two different connection session. This is actually the most risk part of this "FIRST" design because their existing Java Web app use connection pool, which can not promise always giving back same connection to servlet serving same longin customer.

So, as we can foresee, if the Web app use a different database connection to call the stored procedure without "FIRST" parameter, the stored procedure has to deal the calling as the first time call although the developer of Web app think they are "reuse" the opened cursor. In fact, the last opened cursor does not exist now.

The interesting thing is that they do not notice this problem as they configure the initialized available JDBC connection as small as one. So, the Web app is actually always using same JDBC connection although the connection is fetched from the connection pool.

So, this "FIRST" parameter for stored procedure is incorrect. The stored procedure only needs the parameters for telling stored procedure about start row and total number of rows it wants. However, in AS400 DB2 UDB, we only have function "FETCH FIRST". It does not have function like "limit", which MySQL has. However, we can use the DB2's row_num() to implement this as below,

select * from (select col1,  row_number() over() as rownum  from schemaname.tablename)end  where rownum < 10 and rownum > 6 ;

Saturday, November 20, 2010

Do we need database constraints?

I noticed that the DB2 system belong to one project I am working on does not have any constraints set up. One of my colleagues thought it is too bad that the database does not have constraints like foreign key etc. Do we need database constraints in our system excepts the primary key, and index constraints? Well, I think there is different answers for different people who plays different roles in a software development life cycle.

According to my understanding, the constraints in relational database is constraints set up by Database Designer for Application Developer. Database designer set up the foreign key constraints, null constraints, check constraints etc. Then, developer must follow the design to avoid break constraints.

Properly setting up constraints will be helpful if developer want to use some OR mapping tools to automatically generating database abstract layer. Or, it is useful if the application will depends on RDBMS to cascade delete records. However, I do agree that it could be harmless to remove constraints like foreign key in the product environment if the application takes care of natural key very well to avoid having orphan records. I think it could pay us some performance increasing and computer resource usage decreasing as return. Probably this is the reason for MySQL did not implement foreign key constraints when it is first invented and it still become widely used?

Monday, October 25, 2010

failure on passing SQL statement as parameters into stored procedure and the way to walk around

Recently, I got code 3 error message when I tried to call a stored procedure on IBM i DB2 through System i Navigator. The purpose of this stored procedure is to perform a series queries that are passed in as the input parameter under a predefined schema.  Below is the way to call the stored procedure,
call myStoredProcedure('update dummyTbl set columnA = 1 where columnB = ''Jia''  ')

The error message says that

Message: [SQL7008] dummyTbl in schemaA not valid for 
operation. Cause . . . . . :   The reason code is 3.  
Reason codes are: 
1 -- dummyTbl has no members. 
2 -- dummyTbl has been saved with storage free. 
3 -- dummyTbl not journaled, no authority to the 
journal, or the journal state is *STANDBY.  Files 
with an RI constraint action of CASCADE, SET NULL, 
or SET DEFAULT must be journaled to the same journal. 
The interesting thing is that I get error free when I run the same SQL query in System i Navigator. 

After doing research, I found that, on IBM i DB2, a table is journaled by default if it is created with SQL statement in i Navigator. On the other hand, the table is not journaled by default when it is created under green screen. And the walking around is adding the following statement in the stored procedure before executing the input string that is actually sql statement,
SET TRANSACTION ISOLATION LEVEL NO COMMIT

This could only be temporary walking around solution. To be serious, it is better to review tables setting about journaling and isolation level. However, for an existing complex database system, this walking around might be a good choice too.

Wednesday, August 4, 2010

Using user defined variable to simulate DB2 RANK function failure in MySQL server

I recently tried to simulate RANK function from DB2 by using user defined variable. It is very nature for developer to think about using MySQL user defined variable as counter to simulate RANK function. However, it is not successful. I got random rank number in the result set. Below is the sample SQL code.
SET @jia = NULL ;
SELECT id, sum( sales ) AS sales, count( * ) AS sig, @jia := IFNULL( @jia , 0 ) +1 AS rank
FROM brandpromo
GROUP BY id
ORDER BY sales DESC
LIMIT 100 ;
I was only aware that the user defined variable is only connection session scope alive. I did not know that it is dangerous to use user defined variables together with GROUP BY and ORDER BY clause. So, DO NOT use user defined variable together with GROP BY and ORDER BY in MySQL server. At least, be careful when you want to use it in that way.

Actually, I found a blog post that talks about this topic in detail. For more info, we can read this.