Run batch file .bat or .vbs after Scribe Insight Integration Process runs in Scribe Console

Four years ago, I have written a post to talk about running batch files before or after running a Scribe DTS job. Here is a link to the old post:

http://www.mohamedibrahim.net/blog/2009/08/11/scribe-console-renaming-source-text-files-before-running-a-job-after-processing-and-regularly-changing-source-file-name/

Today, I was trying to do the same thing and I referred to my post and I was faced with a strange problem. I found that the Scribe Console integration process fires the DTS scribe job and then post processing, it runs the batch file. But the process never runs again. No matter what you do this process doesn’t run again until you reset it, change and apply change to it or pause it and then resume it. I found this to be really interesting. I tried few options including making sure Scribe has full access to the location, using UNC path instead of folder path, adding “Exit” to the end of the batch file and a number of other options to ensure the batch is processed and then hands back to the scribe process. All of that didn’t work and the process only ran once and after the batch file has run, it never runs again. If I ran the batch separately it works fine and if I run the process without the batch, all works fine.

After a lot of trials and thinking, and I mean a lot of thinking, I remembered that I had exactly the same issue 4 years ago. I looked through my files and I remembered that it was the exact same problem back then. I have just failed to mention it in my post 4 years ago (link above).

Checking back my files, I found that I used the pre job processing instead of the post processing of the dts package. So if I wanted a scribe dts job to run and then a batch to run to copy the source file to an archive location, rename it, time stamp it and then deletes it from its original location, if that’s what I want, I do it the other way. The solution is that you setup you batch to run before the job. i.e. the batch will run, copy the file to archive folder, rename it, timestamp it, and then the project runs. Once the project run, I used the option to delete the event file after execution which does the deletion for me (my source file is also my event file).

So the solution in short is: Setup your batch file to execute before your scribe console DTS job process runs instead of after it. and it works….

In regards to the different options I have tried to get the batch to work after processing the dts package, I found the following links on Scribe Open mind useful for coming up with ideas on how to get it to work (none of them worked for me unfortunately):

https://openmind.scribesoftware.com/topics/prepost-processing-commands-using-vbscript-vbs

https://openmind.scribesoftware.com/topics/pre-job-batch-script-not-running-in-file-based-int

 

Finally, I know as an MVP, I need to report this back to Scribe and I’ll try to do this. Saying that, I won’t be surprised that I am just doing something wrong in my batch or configuration that is causing the issue but as you can see from the 2 open mind links above, there are a number of people who have exactly the same issue.

 

 

Scribe: Moving DTS from one location to another and changing source file location in Scribe

A challenging issue with Scribe is how to move a DTS (job) and source files (in case source is a text batch file) from one location to another or from one server to another without loosing mappings, corruptingdata links, lookup criteria, user variables, loosing fields names and basically corrupting the whole DTS.

Again, I have looked online and could barely see any information or help in this regard (probably my mistake as I didn’t search properly!). I found out that the key for moving Scribe project files including source across to a new location or a new server, the key is always the QETXT.ini file. This file is vital for pointing the DTS to which source file it should be looking at. QETXT.ini files have all the field names, so mapping S1 to the name that you have chosen for the first source field. It also has the source file name, the ODBC name and the table name. From there you can do almost everything.

When you move the files across, you will obviously need to re-point to the new source location, but by editing QETXT.ini, you will be able to put back all the source (S1 to field name) mappings, point to the new source file name, ODBC name and everything else.

This has proved very efficient and have worked now with my whole deployment.

One more important piece of advice, always try to get a dedicated folder for every source file. So if you have more than one DTS jobs, make sure that the source for each DTS job is in a separate dedicated folder. This will ensure you have separate QETXT.ini file for each one of them, hence, you can easily update the information inside it. It will still work with one large QETXT.ini file but it’s always better to separate the sources and their associated QETXT.ini files. You can always manually split this file into source specific files and put each source in a separate folder later on (which is what I have done after “inventing” this best practice of having separate source folders!).

Scribe Console: Renaming Source Text files before running a job and after processing and regularly changing source file names.

This post applies for Scribe Insight version 6.5.1. It may well apply to all Scribe version 6.5.x

Have you ever looked in Scribe Insight for a way to rename a source file before processing it. Scribe console creates collaborations where integration processes can be configured so that they wait for a file to be added to a specific location and then run a specified job. Once the file is added the job will run and process the file. Also, Scribe DTS jobs can only be setup to process a source file that has its name always fixed and unchanged. So a DTS can be setup to process a source file named: customersdata.txt. It will never run if another source file is added to the location the Scribe console is looking at. In this case, if you get a source file with the date (and time) stamp in its name will need to rename it so the the DTS can detect it and run. So if the source file comes with time and date stamp that varies every day (for example: customers_1453_21092009.txt), you will need to rename “customers_1453_21092009.txt” to “customersdata.txt” only to get the DTS to work.

After some research, I found out that you can only do this using pre and post processing commands step of the process. Every integration process has step 2 in it called pre and post processing commands. In this step, you can specify a pre-processing and post-processing commands or scripts. This feature lets you specify a pre and post processing file that can do this renaming for you. Accepted pre and post files are: *.vbs, *.js, *.vbe, *.bat, *.cmd, *.exe, *.com

You will then create a pre-processing script that finds all files that start with “customers*” in our example and rename it to customersdata.txt which is the source file name the job is expecting. Post processing can be to rename the file to something else so that you keep a record of processed files.

In step 3 of the Integration process, Scribe also gives you two options (in the form two check boxes) that I find very useful. You can either select to delete the source file after processing or you can select to rename it. So the source file will be processed and renamed to something like customersdata.L1.txt. Unfortunately Scribe doesn’t give you the choice to choose the new name of the file. Hence, you will need to write a post processing script again for renaming it afterwards if you are after a specific file name.

I’m not sure if there is any other way of renaming source files (also called event files by Scribe) before processing them or specifying a new name for them after processing.