File Watcher In Data Integrator

dataservices job design
1

Hi All,

Our ETL job will be done every morning and a .txt file will be created in a network folder. Once the file is available, I would like to kick off the DI jobs, like File Watcher in BCA. I don’t see any feature in the current version XI-R2, Is there any other work around to accomplish this file watcher method?

Thanks for your help.
Ravi Pazhani.

Ravi_Pazhani (BOB member since 2002-08-20)

2

Yes…kind of.

I think DI has an Access Server configuration that allows you to create a polling from the admin tool to look for a file to start a job. I didn’t have a lot of luck with this but a simple DI job can be used.

This can be done a few different was, but what i have done is create a DI job to look for a file and when the file is found execute a bat file to launch the other DI job. Once the job has started rename or remove the file.

…or find the file in a conditional loop and then fire the workflow
…or create a DI job that looks for various files to execute various bat files for other di jobs.

cduey :us: (BOB member since 2002-09-05)

3

Hi Ravi,

In addition to cduey’s suggestion:

…or find the file in a conditional loop and then fire the workflow<<

Don’t know which DIXI version you’re on, but version 11.7 would help make this slightly easier with the availability of the new “wait_for_file()” function, this a sleep function that ends once a file does exists within a given time window. So just schedule your ETL job daily to start early morning, then let the job run.

…I don’t think there is any direct equivalent of ‘BCA filewatcher’ for DI, but one suggestion that will require further investigation, for version 11.7.2, the DI scheduling is available from the BO CMC, it looks like there is the choice to use existing DI scheduling functionality or Business Objects Enterprise scheduling, could BOE scheduling provide that filewatcher feature? Can anyone confirm this?

PS. Will add to post if I get answer.

sae37 :uk: (BOB member since 2005-05-10)

4

Thanks for the info. I am working on DI version 11.7.0.0 and it does have wait_for_file function. I am having hard time to get the right syntax. If any one of you have the right syntax, please let me know.

Also, the DoneFile.txt file is available in the network folder. How should I check the availability of the file in the network folder in this syntax? DI has access to the network folder.

Thanks
Ravi.

Ravi_Pazhani (BOB member since 2002-08-20)

5

Please ignore…I figured it out the syntax and it sort of works well.

Ravi_Pazhani (BOB member since 2002-08-20)

6

:mrgreen:

If I am not mistaken, the CMC of Business Objects Enterprise does have file watcher capabilities. Schedule a 11.7 job in the CMC rather than in the jobserver and modify the CMC schedule then.

Werner Daehn :de: (BOB member since 2004-12-17)

7

So, Can I schedule a DI job with the BOE scheduler? If possible, could you let me know how should I call a DI job in the BOE scheduler? I have used this only for report scheduling.

Thanks
Ravi.

Ravi_Pazhani (BOB member since 2002-08-20)

8

Has anyone used the wait_for_file function to check the file at the ftp server location? It doesn’t work for me eventhough the server has access to the FTP location of the file. It just simply skips the script which has wait_for_file function and execute the jobs and sometimes gives the error ‘Permission denied’. Any suggestions? or work around to fix the issue?

Thanks
Ravi.

Ravi_Pazhani (BOB member since 2002-08-20)

9

I am in the same boat. I need a solution for file watching. We have BOE XI R2 but I’m not quite sure how to create an event that will trigger the DI job .exe.

jemarqu :us: (BOB member since 2006-11-15)

10

Hi Ravi,

I would suggest that you try and troubleshoot the file access or permissions issue you are experiencing. This is because the obvious work around that is available would probably include using the while_loop and file_exists() function, and I’m sure you will experience the same issues (unless there is a bug specififcally with the wait_for_file function…)

A little info on your environment would help?
What OS does the DI JobServer run on? And what is the FTP area on?

However, to help troubleshoot: When you execute the DI job, the ftp area would be accessed using the DI Jobservers credentials (remember that the DI Job server is run under a user account, which maybe different from your own user account). What I would do is logon into your machine using the DI Job server user, and check to see if you can view file/access ftp drive/folder. This may highlight permission issues?

Also, if you are using Unix JS, has your DI Job server been restarted since the JobServer user account was given rights to the ftp area, if not you will need to restart the DI service for new rights to take effect (don’t know if this was a bug but we have to do it for 11.5 for Unix Job server.)

HTH

sae37 :uk: (BOB member since 2005-05-10)

11

jemarqu,

I’ve been able to enable BOE scheduler and then use that to schedule DI jobs. The benefit from that is that with the BOE scheduler, I can create an event.

The event that I create is when a file appears. My DI job starts when that file appears.

Hope that helps.

Tony

webtechie :us: (BOB member since 2007-10-10)

12

I am also working on same task. Once file arrives in specified folder on AIX UNIX box I have to trigger DI JOb and remove the file from folder.

Can any one please explain sequence of steps how to do above task.

ramanagh (BOB member since 2009-07-07)

13

Hi,

I am having requirement like you .Could you please give me the sequence of steps need to be executed.

My scenario will be .I need to check the dependency in source files .as per the depency of source files I need to execute my jobs.

Can you please give me the steps how i needd to work on this in detail.

I will appreciate If you send me the sequence of steps.

Regards,
Kumar

kvkumara582 (BOB member since 2009-11-22)