Online web grabs

[Esmo2000]

Hi everyone,

I'm fairly new to filemaker, and was wondering if something like this was possible. I basically want to be having a database that is going to have some sort of interaction with business websites to grab various pieces of information (e.g. Quarterly reports, annual reports, very consistent things) from websites. Are there ways to port things such as this into filemaker, and on top of that, check if the data being received is actually meaningful? (a very simple check).

For example, say I got the 2003 annual report, and I linked it into Filemaker. Poeple can see it when they use it. In a year, the company changes their file system, and the website moves. Obviously this will confuse FM, and it will not load a website.

Is there a way to check for something like that? Also, are there any sorts of business streamers people know about which might be directly linked to FM?

Thanks a lot everyone, I really appreciate your help,

Jim

[Leb i Sol]

Hello Jim!

Nice post and in line with some discussions....

you can not really relay on "busniess website's content" UNLESS you are the webmaster; so pulling "reliable" info from it is impossible. A semi reliable site would be Forums styles or Newspapers where info is published in patterns (most likely by some DB in back). "Drilling down" throuhg dynamic sytes is big topic of security and URL executions/injections....even for those who design it.

In dynamic site...."URL-Rewrite" is a must

Remember, FM is really the one that "holds" info rather than being a "site robot" and the one that scoops out the data.If your are talking about Intranet or site that you control then something could be setup in line of remote-ODBC-XML import so your 1st attempts could be in direction of "Importing"...

Don't get me wrong there are "ways" arround it but the road would not be paved...sonsdier also using some platform specific:

- Send MSG

- Execute DDE

- using DOS commands (eg. Ping http://www.google.com) and external utilities

I, too, am also a fan of filing a "good website material" as you never know about its availablity. My favorite choice was using Adobe Acrobat and capturing pages into PDF....which can be stored into FM

If you would like to talk more about this topic then post some specifics...as in sites you want to capture, from FM server? from antoher webserver that is searrch engine etc....perhaps we can extend this topic a bit more as I know that a few developers were interested into this....

Anyhow....

Take care!

[Garry Claridge]

You can use PHP to periodically check if the URI exists. The "file_exists()" function and a derivative "remote_file_exists()" are described at:

PHP Site

PHP offers great flexibility to FM.

All the best.

Garry

p.s. If you are using OS X, as well as PHP, you can use Applescript and Cron jobs to do the same thing.

[Esmo2000]

Hi guys! Thanks so much for the assistance, I really appreciate this because it is going to be a fairly large project and it is really great to grab all of this information in advance.

I think I should mention before I begin that if there was some really useful additional functionality that FM 7 might provide I would be able to upgrade without too much problem. Sticking with FM 6 for now would be optimal however.

Moving on, let me come up with one example:

Say we look at the BBC website for news http://news.bbc.co.uk/

Lets say for example ( a simple one ) I wanted to just display this website. That would be something I might need to do. Possibly even looking at the format and capturing only the main title, and putting it into FM.

Secondly, thank you for telling me about PHP, but how do I got about doing that? I am well versed in PHP itself, but i do not know how to run PHP from FM?

Thanks again SO much for your help guys!

Jim

[Garry Claridge]

Re: capturing only the main title, and putting it into FM

Do you intend to place this into a database, or just integrate it into a web-page?

Check the PHP forum for information about using FM with PHP.

All the best.

Garry

[Esmo2000]

Thanks a lot, I will check that out.

As for the top website, maybe have something like a database that every day stores away a copy of whatever today's head title was and puts it into a database. This would be accessible to the user as a record (preferably displayed in the same format that the web page was produced in).

Thanks a lot!

Jim

[DanBrill]

You may also want to look at the Troi URL plugin. You can grab the raw html of any web page and suck it into a FM field. For example, I've used this with the census bureau. I take an address from my contact database, submit this to the census bureau, and pull back demographics on the contact's census tract from the bureau's database. The plugin pulls in the raw html, and I parse this out into FM fields.

I tried a lot of ways to make this work without the plugin, without much success. Once I had put more unsuccessful development time into the project than the plugin actualy cost, I broke down and bought it. It was easy with the plugin.

Hope this helps,

Dan