Can scripts run too fast for copy and paste steps?

[seanc]

Hi All,

I've just come across an odd problem.

I have a script that (simplified dramatically):

Loop

starts at record one in the found set

copy's from field Text_A

paste's into field GlobalText_B (without selecting)

Goto next field

End loop

In this way I end up with one global field containing text from every record. I need this so that I can copy and paste into a dos batch file, and exporting won't do the trick because it drops some of the characters.

Anyway, the problem is, every now and again, usually somewhere around record 10 to 15 (out of 25ish), it seems as if the paste step is done but the clipboard hasn't been updated by the copy step. So I end up with 2 copies of one record and one record missed entirely. I put a 1 second pause in before each paste and everything seems fine (albeit slooow).

Does this seem like acceptable, expectable behaviour??

Cheers... Sean.

[Lee Smith]

Set Field would be better.

Set Field [

[Vaughan]

It is more likely that you'll hit FMP 6's 64KB text field limit.

I agree with Lee that Set Field is a much, much better way of doing it.

[seanc]

Hi guys,

Yes I agree that set field is better, and I'll be changing that, but I'm still thinking theres some sort of timing glitch.

I don't know the technical definition of the 64KB limit you mention, but since this problem is occuring somewhere in the middle of the text field, it doesn't sound like the problem. Also, when I paste into a text file it is only 33KB...

I'm almost certain the script is tripping over itself, so I think out of interest the question stands: has anyone else noticed such a phenomenon?

Cheers,

Sean.

[aklodner]

Hi Sean,

I have seen this behaviour once in a great while, don't really have an explanation of how/when it occurs. Just letting you know, I feel your pain!

Here's another idea on accomplishing what you're aiming for. It'll actually avoid the whole looping set up. Make a layout, list view, with only TextA and the globalB on it. When you run your script, start out with clearing globalB, then use the copy all records step, then paste in globalB. You'll get all the values for your found set in one shot, no looping.

Adam

[Vaughan]

Adam's technique is good (I used it just last week myself) but it too can hit the 64KB text field limit in FMP 6 and earlier. (The limit is 2GB in FMP 7.)

To work out approximately how many records can be processed at a time, divide 64000 by the size of the string used for the key (remember to add 1 character for the return separator).

[seanc]

Hi Adam,

Thanks for that. A good suggestion, although I run into text block limits (which presumably is the 64K limit, and I shouldn't be anywhere near that for the test I ran so I probably did something wrong so I should probably try again) so it didn't work out. Never thought to use the Copy All Records step though, so thank you!

I have replaced everything with Set Fields instead of Copy Paste, and it all seems to work fine, so everything is under control.

I still find this a bit of a concern though. I'm sure I've had the same problem in the past, but I can't remember the details anymore. Something to watch out for anyway.

Thanks guys!

Sean.

[Baloo]

"I'm almost certain the script is tripping over itself, so I think out of interest the question stands: has anyone else noticed such a phenomenon?"

I see this all the time in FM 5.5 when I have complex script loops, and yes I've just grown accustomed to pause script; time = 1 sec to fix the problem. I generally both warn my users that the script will take a long time to run and have the script spit out a message reminding them and giving them the option to cancel out at the start.

In short I feel your pain and am hoping this is not a problem in FM7

[Vaughan]

The big problem with copy and paste is that they are layout dependent: the target fields must be on the current layout when the script step executes.

Set Field has no such dependencies, but its gotcha is that the data must be valid for the data type of the field: it chokes when trying to set a date range using "..." for instance, because the string "1/1/2004...1/1/2005" is not a valid date.

I have never come across scripts "tripping over themselves." There may be issues with copy/paste to the clipboard that are OS related but I have not encountered them -- except for record locking in multi-user systems, but this should be trapped for anyway.

[cobra]

Hi Everyone,

I've come across this behavior as well (from memory, this problem goes all the way back to version 4 or poss. version 3).

I think the problem may have something to do with the way FileMaker attempts to optimize scripts and that FileMaker's interface (even with version 7) does not seem to be completely separated from the data.

The last time I came across this issue was for something like the following (in FileMaker 7):

Tables (with corresponding layouts)

Parent <-> Child <-> GrandChild

Use a script to relate several child records to the parent by going to the Child's layout and setting the foreign_key_to_parent for the records on the child table.

The child table uses an auto-enter calculated text field (the calculation is set to always replace the existing value of the field) that gets its value based on a field in the parent table.

That value is used (along with another field) to relate the child to the grandchild.

The problem: The auto-entered calculation on the Child table doesn't refresh unless I'm either doing this manually, running the script in debug mode, or using a 1 second pause before (and after as well, I think) setting the value of the foreign_key_to_parent.

Why do I think this is caused by a combination of script optimization and lack of separation between the interface and data?

You'll notice that the first time you run a script, FileMaker will be a little slower than on subsequent runs. Often, this is because FileMaker doesn't bother going to other layouts when it knows that the script will return the user to the original layout upon completion. It doesn't seem to know this, though, until the script has been run at least once.

To perform certain functions (like copy/paste, updating calculations, etc.) on the layout that your script is jumping to, however, FileMaker often needs the layout to be open and the data within it to fully load/refresh.

The same can be said when FileMaker loops through a number of records when executing a script. To optimize execution, FileMaker doesn't bother refreshing the screen until the script has finished executing. When the screen isn't refreshed for each record, however, things don't always work as expected.

Sometimes, the script will run fine on first execution because FileMaker doesn't seem to attempt to optimize a script until after it is executed at least once.

I've observed this phenomenon for years and it looks like this is always going to be a problem, as FileMaker hasn't bothered to fix the issue over at least 4 versions.

A simple fix for FileMaker is to modify its script optimization techniques. When a script step absolutely requires that the layout's data be present (like copy/paste, for example), FileMaker should wait until the required data has loaded fully before executing the script step.

Alternatively, FileMaker can automatically translate a copy/paste script step to a Set Field that preserves formatting. With Calculated fields, these should not require the layout to be open before refreshing themselves. Instead, they should automatically be updated at a lower level.

Sorry for the lengthy post!

Regards,

Cobra

[Logixx]

The big problem with copy and paste is that they are layout dependent: the target fields must be on the current layout when the script step executes.

Set Field has no such dependencies

Is it just me, or if the target field is a related one in a portal, the portal row must be selected and thus the portal must be present on the active layout?

[-Queue-]

If you are referring to Set Field, then yes, the portal must be present, unless you are setting a field uniquely related to the current record or want to set the first related field (according to the specified sort order, if any), regardless of which related record it is associated.

[Vaughan]

"FileMaker should wait until the required data has loaded fully before executing the script step"

Does executing an Exit Record/Request step help?

[cobra]

Vaughan Said:

"Does executing an Exit Record/Request step help?"

Hi Vaughan,

As far as I can tell, that script step doesn't exists.

If you mean "Commit Records/Requests", then yes, I've tried this along with "Flush Cache To Disk" and whatever else I thought might help.

Thanks for the suggestion, tho!

Cobra

[H.P.]

This is a very common problem. I noticed the same problem when using VB in Excel for copy and paste with an external application. The only way to solve it is pauseTimer or to use set field. But watch out when you use pause timer though. Different machine will need different timer (depend on the system speed). Some need only 1 second, other might need 2 or 3 seconds pause timer.