Data Harvesting

[Papabjoe]

I have to use an online database to gather information of potential clients. The problem is that the database is not downloadable. Does anyone know of a script I can write that will go to the website, copy information from the site and then paste it inot the appropriate field in filemaker?

[transpower]

If the host computer is Windows 2000/2003 Server and you have its IP address (and it allows external access and you have the login name/password) you could log into Terminal Server by Remote Desktop Connection and get your information (for instance, by importing records or copying from the host to yours using UNC).

[Papabjoe]

Well let me be a little more specific. My frame of reference is a program called Happy Harvester. This program allows you to build a database of feilds. You direct it towards a url that is dependant on an id. The solution then loads that page and begins scanning the html code for the peices of information you are looking for. If I wanted a name for example, I would input the strip of text in the html code that comes immediately before and after the actual data I am looking for. As the software scans the source code and recognizes these strings, it copies the data between those strings and inserts it into the predefined fields. Once it has finished that page it adds a value of 1 to the id within the url, loads the next page and begins the process all over again.

[transpower]

So, Happy Harvester should be able to export its data in such a way that FileMaker can import it.

[geod]

It requires extra costs, but you might go to

http://www.acmetech.com/

and look into NetTools.

[Papabjoe]

The object is to build my own solution without purchasing other software.