CA Gen Java EJB server (Unicode/UTF-8) using Mainframe DB2 database with EBCDIC Arabic Code Page 420 has problems with Arabic strings.
Article ID: 201282
Updated On:
Products
Issue/Introduction
In the process of modernising CA Gen block-mode CICS applications under z/OS which connect to DB2 on z/OS.
Pilot project converts applications into EJB Webservices.
Built a sample server procedure into an EJB Webservice, assembled and deployed onto WAS 9.0.5.5 (installed on Linux).
Testing the sample EJB Webservices using SOAPUI with UTF-8 encoding.
Facing Issues as below because of Arabic Strings.
- The READ statements which have a hard-coded Arabic string in qualifiers are not retrieving any data.
- READ on entities which have a relation to other entities and the relationships are implemented as string columns which have Arabic string values are also not retrieving the data.
- When sending any Arabic string as input (provided in SOAP UI request message) to a procedure which does a simple Insert of values onto the z/OS DB2 database, The Arabic texts are getting changed to a different text than what was originally provided in the request input.
What are the steps, configuration changes, user exit changes and any other setup/changes to be done that are needed to support Arabic strings for EJB Web Services?
Environment
Release : 8.6
Component : CA Gen Enterprise Java Beans
CA Gen Construction Toolset and Build Tool installed on Windows 10 with Arabic Codepage 1256.
DB2 database on z/OS with EBCDIC Arabic Codepage IBM-420.
WebSphere Application Server 9.0.5.5 on Linux
DB2 JDBC Driver
Resolution
Theoretically the DB2 JDBC driver should be able to handle the required character conversion between the Gen Java Unicode program and the Mainframe DB2 database e.g. Db2 for z/OS 12.0.0 > Administering Db2 > International data > Application programming with Unicode data and multiple CCSIDs > Java applications and Unicode data
However, a similar problem with Arabic strings for a Gen Java program under WebSphere 6.x running against a Mainframe DB2 database under EBCDIC Arabic code page 420. had been previously reported. Shaping problems with Arabic characters were encountered and IBM provided some routines/code to add to modify input/output strings. Initially, that solution was implemented with EABs but a new Gen user exit "DataConversionExit" was then created to allow modification of the strings in that exit instead. The "DataConversionExit" documentation reference is here:
CA Gen 8.6 > Reference > User Exits > Web Generation > User Exits > DataConversionExit - Web Generation Data Conversion Exit
The file DataConversionExit.java is located here: "C:\Program Files (x86)\CA\Gen86\Gen\classes\com\ca\gen\exits\common\"
The DataConversionExit is called by the Gen Java runtimes in the bindParameterString method right before an SQL statement is executed and is also called in the getStringFromResultSet method right after the SQL statement has been executed.
Any string output from a DB2 query would go through the getStringFromResultSet method and any string qualifiers used in a DB2 query would go through the bindParameterString method.
RESULTS:
1. For input strings the DataConversionExit was modified to use IBM class AS400Text to shape the strings. With that change, able to convert the strings provided in SOAP UI as input and store them onto DB2 database with proper shaping successfully. For READ statements which had Arabic Strings hard-coded, as the generated Java Code contains the Arabic string of different Shaping and/or possibly different code page, the READ was not retrieving any data even though there is data that qualifies the SQL in the z/OS DB2 database. That was resolved by storing the hard-coded string in a local view first and use the local view for qualification in the DML statements. Gen Engineering did find a note from IBM, where they said "Avoid using quoted strings in SQL statements; use host variables whenever possible". The note said using quoted strings containing bidirectional data in SQL statements could cause the SQL statement to be modified in a way that might make it unrecognizable by DB2. So setting up some site standards to use local views instead of hard-coded strings in SQL statements is likely the best solution going forward. However, a Gen "backlog" defect has been created to investigate the possibility of handling the conversion of hard-coded strings in the future. When the analysis is done it may show that although it is possible to change the generators to generate a host variable instead of using the hard-coded string, the change could be risky since it is likely to impact the generator's SQL optimization logic. So it is possible that it will be decided that the risks associated with this change may outweigh the benefits, especially since the workaround to move hard-coded strings into local views is available (although tedious).
2. For output strings no modification to the DataConversionExit is required as by default, the DB2 JDBC driver is correctly converting the output string to the UTF-8 encoding.
However, there were still some residual Permitted Value violation issues for a few strings, which was worked around by disabling PV checks on READ.
Additional Information
Feedback
