The name "Analyzer" was already taken

[BobWeaver]

Edited 2004-05-21:

Please note that the file that was attached here was an early version which would not document scipts when run under OSX. I've deleted the attachment to prevent the old version from circulating. For the latest version, please go to this link:

New Thread and Latest Version

And please post your comments to that thread so that this one can sink to the bottom of the pile and die a graceful death

[kennedy]

I clicked on "Attachment" and it promptly downloaded "download.php" to my Desktop; then it tried opening that, which opened up Dreamweaver and opened that file... which of course was full of gibberish.

What was supposed to happen?

[Kurt Knippel]

Try it again. Sometimes what happens is that the database which stores the attachments is not updated as quickly as that which stores the messages. So the attachment is not immediately available. Also make sure that you just click on the link and DO NOT right click on it. What you are actually clicking on is a php script which will then download the files.

[kennedy]

Okay, I just tried it again... again by left-clicking it... and again I got a window of gibberish in Dreamweaver. Ideas?

[Kurt Knippel]

Try a different browser, or clear your browser cache.

[BobWeaver]

Yes. The same thing happened to me. This is over 12 hours after I uploaded it. So, there must be something else wrong.

[BobWeaver]

I just realized that even though the file downloads as "download.php" it is actually the correct file. It is a zip archive so you can rename it to "softdoc.zip" and then run it through Stuffit Expander or Winzip.

[kennedy]

I just realized that even though the file downloads as "download.php" it is actually the correct file. It is a zip archive so you can rename it to "softdoc.zip" and then run it through Stuffit Expander or Winzip.

Yep, it worked! I'd have never tried that. Thanks... I'll be checking it out right now...

[kennedy]

The analysis step worked fine.

I printed the scripts to PostScript just fine (well, its a bit tedious).

And then I hit the button to process the scripts, brought up the dialog to select the corresponding script file... navigated to it just fine... but the file was grayed out. I can't figure out how to tell it to load the file. Any ideas?

I am running this on OS X.1; are you doing anything to require a specific Type or Creator? Can we avoid that, or tell it to allow any file?

Thanks.

[BobWeaver]

Well, postscript files are just text files, but I have heard of problems with Filemaker running under OSX getting not recognizing text files when importing. I seem to remember something in the Importing/Exporting forum about that. I'll have a look and see if there is an answer there.

[BobWeaver]

Okay, here is the link to the topic:

http://www.fmforums.com/threads/showflat.php?Cat=&Board=UBB13&Number=45210&page=1&view=collapsed&sb=5&o=31&fpart=1

Unfortunately, he never said whether or not he solved the problem. You could change the extension of the spool files to .txt to see if that helps. It seems to me that OSX has some very strange quirks when it comes to file types. Trying to act like Unix and MacOS at the same time must make it schizophrenic.

[kennedy]

Okay, I solved that problem by manually setting the Type and Creator to TEXT. So, now its importing...

But the first file it imports, it goes through 3 processing dialogs (the last being Replace), and then pops up a dialog saying:

"This script file doesn't appear to belong to the current application, or it has already been processed. Script processing will be aborted."

Any clues?

[kennedy]

It shows current script to be "!" right before giving the above message.

Is there any way to have it abort that particular script file, but continue processing, moving onto the next file. I can't figure out how to get it to try another file... it always wants to try the same one first. And that one, being part of my authentication system, may be peculiar.

[BobWeaver]

Go into the script called "SaveScriptInfo" and about 3/4 of the way down is a step"

If[gCurrentDB != gDetectedDbaseName]

change it to:

if[0]

This will bypass the routine that checks to see if it's reading in the correct file. It obviously doesn't work under OSX but disabling it shouldn't hurt anything else.

Sorry, for the nuisance, I really should have fired up OSX and tried it myself before I posted it.

[kennedy]

Thanks, Bob... have it working now. But you should mention that its damned slow! It was absorbing 200,000 records out of those PS files. Anyway, few hours later it finished (and that on my 10% finished project).

The results are very nice. I'll be adopting it into my Admin functionality... but I think I'll figure out how to use AppleScript to script the script printing and then, even more important, the script file processing. That way I can run it unmanned overnight, and have the answers ready in the morning.

Any issues that would prevent me from scripting this?

Anyway, thanks for posting this... its awesome!!

[kennedy]

Although it churned on the script output, it didn't seem to work. None of the fields that are referenced by scripts are marked as being referenced by scripts. In fact, as best I can tell, all that script processing didn't make a single change to the resultant database.

Any ideas?

[BobWeaver]

If you go to the script records, do they look like scripts, or do they look like garbage?

Regarding speed, how big is your project? How many fields, layouts, scripts etc., and what kind of computer are you running it on?

The largest project that I've tested it on had:

21 files

700 fields

183 scripts

56 layouts

111 relationships

26 value lists

It processed everything in about 7 minutes on my 400Mhz G4. It sounds like you have a much bigger project or a much slower computer.

[BobWeaver]

Okay, I did what I should have done in the first place. I booted OSX and tried running it. I realized that I don't even have a print-to-file option. I only get print to pdf which is a totally different thing. Mind you I still have OSX 10.0.4 which is ancient. Does your version of OSX allow printing to an actual postscript file?

[kennedy]

Yes, I have two choices: Print to PDF or to PostScript.

However, I pulled up the PostScript files in TextEdit... I don't see anything that resembles the text of the script. I assume it has formatting around each letter.

I am running 10.1.5.

What does the PostScript output look like in OS9? Is the script plainly visible in the Postscript source file?

Attached is one of mine.

Begin.ps.txt

[kennedy]

Regarding speed, how big is your project? How many fields, layouts, scripts etc., and what kind of computer are you running it on?

I'm sure the speed problem has more to do with the unexpected form of the Postscript files. My project will soon be larger than what you listed, but currently it is smaller...

10 databases

548 fields

29 layouts

31 relationships

16 value lists

It only found 11 scripts... one per file... though, in reality, there is probably 200 scripts.

And I'm running on a dual 500 G4.

Brian

[BobWeaver]

Postscript files contain a tremendous amount of formatting code. The actual script text doesnt' usually appear until about two thirds of the way through the file. Even then there is a lot of formatting material around the text. If you open it up in a text editor and then do a search for a piece of text that you know is in one of your scripts you can see how it looks.

In order to parse the text I had to make some assumptions based on a bunch of output that I generated in different versions of printer drivers. I only used things that were consistent across all of them, but I never tried it with OSX. So, I assume that the driver in OSX generates some different formatting codes that confuses my parser routine. The slow processing on your system is obviously due to that confusion.

I'm going to get a new version of OSX and install it shortly, and then I will test things out.

[BobWeaver]

Sorry, I hadn't noticed that you had attached a copy of the postscript file. I had a look through it and found that it is set up completely different from the postscript files generated in Mac OS8 and OS9. I don't see any quick fix for this. Maybe you could print in classic mode and that would generate the earlier style file.

I definitely will be looking into making the program compatible with OSX because I don't intend to abandon it, and I will be forced to switch to OSX at some point. But, for the time being, I'm afraid that this is OS8 and OS9 compatible only.

Thanks again for the feedback.

[BobWeaver]

Any issues that would prevent me from scripting this?

Since the script analysis failed in OSX, I guess this is strictly academic at this point but, if you're interested in running under OS9, there is no reason why it can't be scripted.

I originally experimented with scripting the entire print to file and import, thinking that I could use applescript to tell each database to print the scripts, but I found out that Filemaker doesn't get any print setup details when it gets a print command via applescript. That is, you can't say:

tell document "MyFile.fp5" of application "Filemaker Pro"

Print all scripts

end tell

...unless that has changed with version 6.

However, you should be able to at least have applescript send the print command and then manually set the print to file and file name. And then, use applescript to import in all the spool files.

Meanwhile, I see that a couple of commercial documentor utilities are using Print2pict to extract scripts. I may modify SoftDoc to use it as well. I didn't want to do this, because I wanted it to be a standalone utility. And, I think Print2pict only works in classic mode anyway. So, there's likely no advantage over using postscript.

[Bob White]

You didn't say which browser you are using. However, is this hexadecimal looking type of gibberish? If so, on most versions of Explorer or Netscape, just hold down the Option key as you start the download and you will get a .sit or .zip file instead of miles of gibberish. Doesn't always work and works on more versions of browsers on Mac's than on Windoze. At any rate, try this next time you get gibberish and see if it works for you.

[BobWeaver]

Okay, I've had a chance to experiment with the OSX postscript file that you attached. I managed to write a new processing script which seems to work. I've attached it here. It's not the whole documentor thing, just a script reader. Since I still don't have OSX on my computer, I can't generate OSX postscript files to test. So, I would really appreciate it if you could try this with a few scripts and see what happens. If it works for you, I will incorporate it into the main program.

[kennedy]

So, I would really appreciate it if you could try this with a few scripts and see what happens. If it works for you, I will incorporate it into the main program.

*******, Bob... I'm amazed you were able to parse those files! It took almost an hour to process one of my larger files (165 scripts).

I did find a few errors, but its 99% correct... and good enough that it would rarely affect the analysis.

In the bodies of the scripts there were very few errors. Errors were a bit more common in the header info (File and Script names).

In the bodies:

1) All "/"s became "k"s. So, "PausekResume Script", "Exit RecordkRequest", "Go to RecordkRequestkPage", and "SelectkPerform".

2) The Mac not-equal operator came up as a blank character.

3) An extra line feed was introduced after "Set Field [" and before the rest in a few cases where that Set Field was the first line in the script.

4) One time, it came up "Toggle Window [ Maximiz ]" ("e" missing)

In the header info:

5) In about 15-20 of the scripts, the filename was missing the ".".

6) In one case, the filename had the added prefix "y,2002" in front

of the filename.

7) In maybe 10 cases, there was a missing character or two in the

script name. So, "Prevew Prevous" instead of "Preview Previous",

"Prevew Las" instead of "Preview Last", "Selecion" instead of "Selection",

"Refres Portal" instead of "Refresh Portal", "View Acunt" instead of

"View Account", and "Views k Layuts" instead of "Views / Layouts".

I am now, with great anticipation, looking forward to being able to

put those two halves together and being able to use your analysis

tool. There's several things I want to check out so that I can clean

up.

Thanks for working on this analysis tool, Bob... it will be a huge

productivity boost for many of us!!

[BobWeaver]

Thanks for the very detailed feedback.

Just for comparison purposes, the OS9 PS files seem to process about 20 times faster than the OSX ones. So, if you have the option of running this under OS9 or OS8.6 there would be a noticable improvement.

Once I got the basic parser working, I tried to optimize it as much as possible, but the way the OSX PS files are composed makes for a lot more processing. After getting past all the formatting preamble, there are two lines of code for every character of script text. And, that character must be looked up from one of at least two character encoding tables. And to slow things down even more, the character encoding tables are different for each page. So, if the file has 20 pages the encoding tables have to be rebuilt 20 times. Thank goodness for Loops.

Anyway, I will have a look at the missing/wrong character problems and try to get this all put together.

[BobWeaver]

I noted two bugs which could account for all the problems except for item 3.

Regarding item 3: The breaking of the Set Field line was something that had been occurring everywhere. I put some code in to prevent it, but apparently didn't work when the Set Field was the first step. I've changed the code somewhat hoping that this will fix the problem.

Here is a revised script processor.

If you still encounter problems, there is a script in the file called ExampleScript. It contains all possible script steps. You could print it to a PS file and post it here so I can analyze it.

(Uploaded revised script processor 2002-10-28)

[kennedy]

That fixed all problems but two:

1) All "/"s became "k"s. So, "PausekResume Script", "Exit RecordkRequest", "Go to RecordkRequestkPage", and "SelectkPerform".

6) In one case, the filename had the added prefix "y,2002" in front of the filename.

I noticed there are 3 fewer scripts (162 v. 165)... I'll get back to you on the proper number. I may have changed something... but I don't think so.

I have a second computer set up to run your script (since it takes an hour or so for each file). So, I am going to run a few more through.

Attached is the Example Script in PS form (with added .txt extension just so I can attach it as-is). Enjoy!

OSX-PSexperiment.ps.txt

[BobWeaver]

1) Arrrgh... Yes, I recognized the source of the bug but didn't fix it properly. But, I do have a proper fix for it.

6) There are a few issues involved in recognizing the file name from the data in the page footer. The extracted script line looks like

"October 27, 2002MyFileName.fp5 - SomeScriptName-1-"

with no space between the date and the file name. Also if the file name or script name contain hyphens it makes it difficult to determine where the file name ends and the script name begins. Not likely a major problem though, since the main program will have a list of file names and script names that it can use to sort everything out. In fact, I now realize that this is also a problem in the OS9 version.

Just curious, in the situation where you had the prefix of "y,2002" on your file name, what were the real file and script names?

[kennedy]

The first time the file was Families.fp5, but I didn't write down the script name.

On a more recent run, it happened again... this time:

File = Persons.fp5

Script = Update LOGFILE Parse

I do have identical scripts in Families.fp5... high probability it was the same one.

In Persons.fp5, similar happens here:

File = K, 2002Persons.fp5

Script = View Marketing

Also, I have one case where the File comes up blank.

And one case where the "5" in ".fp5" is replaced with a ".".

Doesn't seem to be related to the script names, nor to

the preceding or following script names.

It feels like a case of one variable writing on another.

HTH.

[BobWeaver]

Here's yet another version of the script reader. Hopefully it fixes the problems.

[kennedy]

Didn't fix them all. I still have a couple filenames that are blank and a couple that have prefixes. In all cases, the Page Footer text is odd.

In the cases of prefixes, the date was munged. For example, in one case, instead of year "2002" I see year "2..2" resulting in "..2" prefix to the Filename. In the other, the day is "2y".

In the cases of blank filenames, the page number at the end of the footer is munged. Instead of "-1-" (where I think its dashes instead of hyphens), I have "212" or ".1." or "h1h".

BTW, the latter dominates the former... if both are munged, you still get blank.

[BobWeaver]

Would you be able to send me a PS file with an example of this? I only have the two files you previously sent to work with, and they don't exhibit this problem.

[kennedy]

Well, I'll have to do some stripping... file stripping... and see if I can lose the proprietary without losing the bug. Let me see what I can do...