Web pages Causing Crashes

[dscottsw]

I'm running a database where I've created a script that loops through a series of product web pages to check product availability - use the "get content" from the Web Viewer, and then parse the text to establish a base field, then set a mirror field with that information - so the next time the system navigates to that page, it can compare the two fields and trigger an alert of something has changed. All that works fine.

However, with one vendor (Victoria's Secret), the amount of system memory used triples over other vendors (like Abercrombie or Hollister). I've noticed their web pages are very "java intensive" and am wondering if that could be causing the huge memory differences. The web page source code doesn't seem significantly larger than the others. After about three hundred records, it actually shuts down the application (FM-11-Adv).

I'm not storing the webpage - it saves to a global field and then is cleared with the next record.

Using MAC Snow Leopard, 2 Gig Ram, 2 Ghz processor

Any ideas?? Similar problems?? Tarot card readings on the subject?

Edited July 16, 2010 by Guest

[fseipel]

You might want to try using ScriptMaster (free), cURL (free), or Troi URL plugin to get the HTML content without rendering the pages or executing any scripts. This may resolve the issue, since a browser will never render the page, and none of the Java code will execute. This ought to significantly reduce execution time for screen scraping as well as improve stability.

[Lee Smith]

I have not experienced the Web View crashing FileMaker. Can you post an example file so others can test it to see if they can recreate the problem?

[dscottsw]

Yes - That's exactly what I ended up doing. Scriptmaster's "GetURLasText" command - works fine. It gives me the source code, which I can then parse to get the product update information I need - without having to load the actual pages.

Thanks for the response! :laugh2:

[dscottsw]

Thanks for the response. I was able to replicate the problem on both my MAC and PC - so the problem was certain kinds of Javascripts within in the particular vendor's web page. My solution was to use the "GetURLasText" rather than load the page. Completely solved the problem and sped up the process too. Thanks for the response.