I've taken some actions and performed some tests. I feel stumped.
1) I rebuild my Mac OS X hard disk from scratch with Mac OS X 10.1.5 and did permissions repair (I have been warned away from 10.2.x with FMPS) and then installed FMPS 5.5v2. I didn't install anything extra, such as Retrospect Client or Timbuktu. I also reduced the size of the cache to 10% of the total size of the databases involved.
2) In the application I use, the time to generate an aging report went down from 1h45m to 1h20m. Some progress.
3) If I run this same database application from my Xserve (with hardware RAID) as a local application, the aging report takes 11 minutes to generate (with TCP/IP turned off in FMP) (and 14 minutes if data is on a local hard drive). If I turn on TCP/IP on FMP and set up the files for sharing, aging report generation time goes up to 14 minutes. This leads me to believe that my network is performing fine (all computers in question access the network at 100baseT).
4) As a test, I had another computer access the FMP databases with my computer as host. Response time was awful, worse than with FMPS as host. But response time with my computer (the host computer) was fine.
I have ordred the June/July issue of FileMaker Advisor to see if its article on optimizing server performance will yield any clues, but FMPS performance just seems very poor. While I am sure adding a fast SCSI drive (I have a few cards available) would help, I don't know if anything short of adding a RAM disk, with backups every 15-30 minutes to hard disk) will really help. Basic lists and relational lists seem fine, but anything involving calculations seem to get very bogged down when using a "remote" FMP computer. Surely, for $1,000, FMPS has more to offer.
--Paul