I am not a code page guru, but where are you seeing the junk characters? Viewing the table in BODS or via the SQL client?
I know that when I was doing double byte extraction/loading, I had to convert the varchar to nvarchar data types in the table definition order to get proper character set in the database, which was MS SQL SERVER.
I believe as long as your Data Service job server is set to the proper code page, you are good from the DS side.
I can view the data from the from the source table using view data option. I can see the data design viewer option also. But after loading the data , I can see junk characters in the IQ target table.
Earlier I have handled from MS SQL server to IQ with Arabic data and from Oracle AL32UTF8 code page to IQ.
I have tried to load a flat file with both options cp1256 and UTF-8 and still seeing junk characters in the flat file also.
Step 1: If you are seeing junk characters in IQ outside of viewing the data in DS, is there any way to load some special character data into a table not using DS? This test will make sure your database is set up to accept that data using its code page settings.
Step 2: Once the database is verified to accept this data not using DS, then make sure the job server is using a correct code page. This is a windows setting or a unix setting depending on your platform. Verify that notepad can store and view these characters correctly.
Step 3: Bring DS back into the equation and make sure the code page is the same as the ones that you are using in steps 1 and 2. Remember that if you are using the designer on a different server than the job server, make sure this client is also set up with proper code page. Set the DS flat file code page in the file format editor.
Remember to always view the data outside the DS viewer just to be sure what is actually in the table.
We have resolved the issue. We need to consider below things here . I hope these steps will help for resolving the code page issues.
Source database code page and need to set datastore code page settings accordingly.
Target database code page and need to set datastore code page settings accordingly.
Use the environment variables related to databases if required. For example oracle database NLS_LANG environment variable required. For MS sQL server,Sybase ASE & IQ not required of any environment variables.
Set the charset options in the ODBC settings. ODBC settings are depending on the database. For Sybase IQ setting are required but for Oracle , SQL server not required.
Set the region & language settings in the control panel.
As per the region & language settings in the control panel need to provide the settings in the Data Services Locale selector. Data Services Locale Selector settings written to the DSConfig file. System regional & language settings and DSConfig locale settings should match.
Restart the system as we are changing the system settings .
(BOB member since 2009-04-30)
(BOB member since 2008-03-11)