Huge fields slowing down PHP?

[Thetcm]

So, I have a database that gets his informations from another database, then I load information from that first database.

There are fields on it that are huge, and contain up to 8 pages text if it was a standard text document. When I load my list of records I need to download between 50 and 100 of these records and it turns a 2 seconds search into a 30 seconds search. Really not user friendly.

Anything I can do to help the loading speeds?!f

Edited August 10, 2009 by Guest

[Genx]

It's got nothing to do with the amount of text in the fields -- you are trying to retrieve too many records at once.

FileMaker can realistically handle a full call for about 10-20 records max at any given time before you start noticing an exponentially increasing slow down - this has nothing to do wit quantity of data in your fields and more to do with a fairly problematic XML engine at the core.

To solve the issue you are going to have to either split your call in to 10 [take a look at the setRange([skip[, Max]]) API call] - if you actually need all of this at the same time and collate them (why you would I'm not sure - the usual practice is to integrate paging...) or try and dump the records into a portal and call the main record -- this may actually allow the full hundred to be returned at once though I haven't tried so don't hold me to that thought.

Your other option is to try ODBC.

[Thetcm]

I'm only loading 25 of them in my list. I'm not loading them all. It loads instantly if I remove the huge fields from the layout but it takes about 20 seconds if it's there. I'm pretty sure it has to do with the content of the field.

Anyway, I tried splitting the field into 10 smaller fields and go a much better result, loading times being between 1 and 2 seconds.

Edited August 11, 2009 by Guest

[Genx]

Not really sure how splitting one chunk of data from one field across 10 fields is likely to help given that you're still sending the same (well actually more) information back to the web server.

But hey, if it works it works.

[Thetcm]

I'm actually as baffled as you are. Since I'm doing 25 requests to the database, and the field is in a portal that has 4 different block of information, that means I'm actually requesting a big chunk of text 100 times.

By separating the field into 10 smaller fields, I'm now requesting a total of 1000 smaller fields instead of 100 big ones, and somehow, it loads nearly instantly, instead of taking 20 seconds.

I guess it's because of the way the PHP API analyzes the fields. It could be a bug. I asked on the official FileMaker forums to see if a developer knows anything about this.

[Baloo]

The API uses PHP's built in XML Parser to parse the result of the cURL request it sent to Filemaker's XML feed. So I doubt that's the hang up. Still given that every lag I've run into has been fixed by reducing the amount of XML returned by Filemaker Server this is interesting. If you get it figured out please post back.

Generally speaking I've found that I get the best results from the API by grabbing the absolute minimum of data possible on page load. I then use Ajax to pick up each individual piece as I need it, often creating a specific layout to maximize the efficiency of each call.

[Thetcm]

Interesting idea Baloo. That allows you to quickly show information, while the additional stuff loads. Unfortunately it can't help me here, since I'm actually searching, but I'll keep this in mind.

Anyway, on the official FileMaker forum, I was told it was normal, but to send my database and that they would test it out. I still haven't gotten around to doing that, but once I do and get an answer, I'll share the result.