Any perl experts out there?

[BobWeaver]

We had a solution running in the office that exported the contents of a field into a text file. In FM7 the resulting file is 2 byte unicode, and there is no option to export it any other way. We needed to convert the resulting text file to ascii. The quick and dirty method is to scan the file and simply delete any occurrence of the bytes 00, FE and FF (hex). On the Mac, I use a one line shell script to do this with TR, and we were using a perl script on the Windows machine. That is, until the IT guys came and refomatted the disk without asking anyone, and wiped out the only copy of the script. The guy that wrote the script is no longer around. So, now I need to recreate the perl script that scans the file and deletes those characters. It should be fairly simple for anyone familiar with perl, but I'm not. Can anyone help?

[Wim Decorte]

Any objection to doing this with the native Windows tools? This is a VBScript that will strip out Hex 00 (decimal 0), hex FE (decimal 254) and hex FF (decimal 255).

This approach (reading the text file character by character) could be slow on large files. In that scenario you could read all of the content of the file in memory at once and run 3 replaces.

Set objFSO = CreateObject("Scripting.FileSystemObject")



Set inFile = objFSO.OpenTextFile("c:source.txt", 1)

Do Until inFile.AtEndOfStream

    strCharacter = inFile.Read(1)

	If strCharacter = Chr(0) Or strCharacter = Chr(254) Or strCharacter = Chr(255) Then

		' do nothing

	Else

		' write out the character

		data = data & strCharacter

	End If

Loop



Set outFile=objFSO.CreateTextFile("c:cleaned.txt")

outFile.write data

outFile.close

[BobWeaver]

Thanks Wim. We only used perl because of our programmer's familiarity with it. He's gone now, and we have no reason to stay with it.

I'm not a Windows guy, so I have a couple of questions:

1. Do we just save the VB script as a text file and then call it from a command line?

2. We need to tell the VB script the output file name. Can we include the file name as a command line parameter? If so, how do we access the command line parameter from within the script?

[BobWeaver]

Okay I did some experimenting and got it working with command line parameters.

Unfortunately, our files are about 2Meg. each, and this script takes up to 2 minutes to run compared to a couple of seconds in perl. We generate about 100 of these files at a time. So, it needs some optimization. I will try buffering the reads and writes to see if that improves things.

BTW Does anyone know if there is a version of the TR utility that runs on Windows? That would be ideal.

Edited April 19, 2006 by Guest

[Wim Decorte]

If you do away with the loop and use the Replace function in VBscript it should be a whole lot faster.

[BobWeaver]

Yes, that's pretty much what I ended up doing although I didn't remove the loop completely. It now only takes only a couple of seconds on our slowest computer. So, that's as good as we were getting with perl, or maybe better. This is what I ended up with:

Set Arg = WScript.Arguments

Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objDEST = CreateObject("Scripting.FileSystemObject")

Set inFile = objFSO.OpenTextFile(Arg(0), 1)

Set outFile = objDEST.CreateTextFile(Arg(1))

Do Until inFile.AtEndOfStream

	outFile.write replace(Replace(replace(inFile.Read(2048),chr(0),""),chr(255),""),chr(254),"")	

Loop

inFile.close

outFile.close

Is it possible to put the entire VBscript in the command line so that I don't have to save it out as a separate file? If so, what would the syntax look like?

[Wim Decorte]

Set objDEST = CreateObject("Scripting.FileSystemObject")

You don't really need this line. You can reuse the other obj. Nothing dramatic, the way you have uses a bit more memory.

You can't pass VBscript syntax on the command line. Not yet at least, the upcoming Monad will solve that issue.

For now you need to do what you're doing: have the VBscript live outside FM and call it with the parameters through a Send Event.

Or you can have FM create the VBscript file dynamically, run it and then delete it. But that's a lot more work. There's an example of this in this forum somewhere. I uploaded one in a thread about getting the list of files from a folder.

[BobWeaver]

Yes, I saw your comment in another thread about storing the script in a global field and then exporting it and running it. So I have done that. That solves the problem of having to preload extra files on someone's machine when they want to use the solution.

Thanks very much Wim.

[Wim Decorte]

You're welcome. I'll have a real VBscript plugin out in a couple of weeks. That one will allow you to execute VBscript syntax directly out of a FM field.

[cjaeger]

Why not create a foud set containing just the current record, exporting the field in question only? Can be easily scripted and gives you a choice of export formats ...

Should not be much slower. But beware: It will add a trailing return to the field's content.

See Example

ExportFieldContents.fp7.zip