UNIX Shell Scripting DWH Engineers Must Know

UNIX shell scripting; listed here the things you can do in Informatica projects. This is useful for every data warehouse engineer.

UNIX scripting; Things You Can do.

  1. Before processing the flat file for Informatica, you can check data according to the requirement specified(like Header, Trailer data or how many details records, etc).
  2. Send mail to the concerned person, if the files having issues against the requirement.
  3. Using the PMCMD command, you can run the workflow if all are fine(eg. start the workflow, Abort the workflow, etc).In 8.6 we have an extra option for creating the folder, label, user using informed. You can check the COMMAND REFERENCE under administrator Guide in Informatica help.
  4. After each successful run of Session in Informatica, you can use Unix with a combination of Oracle to Delete, Update, insert, select for different scenarios either in pre, post-session command or using Command task to call the Shell script.
  5. For e.g to remove, move, rename, compress the cache, log, parameter, data file u can use Unix shell script.
  6. Dynamically u can create Paramare files, Zero byte files(no data), Giving permission to the file using Chmod, Chown, etc.

Additional Points.

  • Informatica server can be installed on Windows and Unix OS. But in most of the projects, you will see that the Informatica server is installed on a Unix OS because of the advantage that Unix has over Windows OS as an OS at the enterprise level. Now since your Informatica server is running on Unix box you need to understand how to schedule jobs. how to access Unix directories and etc to use the Informatica efficiently on the Unix box.
  • You can use the PMCMD command to execute a workflow and you can create a shell script with all your processing logic and a reference to the PMCMD command to execute your workflow. You will also need some of the basic file handling commands to play with Source files and read/analyze the bad files which are created on the Unix side.
  • Remember it is not a must to know Unix to develop Informatica mappings but if you know Unix commands and shell script it will help you in getting a grip on what you are doing and you may be able to do things in a better way.

Keep Reading

Author: Srini

Experienced software developer. Skills in Development, Coding, Testing and Debugging. Good Data analytic skills (Data Warehousing and BI). Also skills in Mainframe.