Initial sync extroadinary slow :(

[Steve Cooper]

I am conducting an initial test of FM easy sync.  I am working on a system that I have in production with synchronization logic that I scripted myself (to iPads from FM Server).  The sync to the iPads using my logic is rather ponderous, so I am researching alternates to speed things up.

I carefully applied the changes described in the FMEasySync setup document to my server file & the client file.  I cleared all data from the client file - anticipating it would then do a full synchronization from the server to the client. 

The primary table has over 150,000 records, so I did not expect it to be quick, but I am puzzled at what happened.  It seemed to run without error, but was so slow, I thought it had stalled.  I waited over 24 hours and in that time the progress indicator said it has processed about 500 records.

I conducted a brief review of where it was going in the debugger, but could not see any obvious problems with how I had inserted the elements into my solution.

I figure, there must be something grossly wrong with how I put this together.  Any thoughts out there?

... Steve.

 

[dansmith65]

EasySync isn't designed to sync that much data at once. Also, the lack of performance you describe is what I would expect when pulling that much data, here is my post explaining why: http://fmforums.com/topic/97486-image-sync-speed-improvement/

I think you should try as hard as possible to NOT have the full set of records on the iPad's. Instead, only pull down the records you need, when you need them.

[Steve Cooper]

Thanks for your input Dan.  My scenario is for testing only - in production, I anticipate hundreds of records to be updated, not 1000's.  My concern is not so much about the total time, but the incremental time for each record.  It seems to be taking about 45 minutes for each record.  These are very small records and don't contain much data.

Are you saying that the processing time for each individual record gets higher as the total number of records to be processed increases?

 

[dansmith65]

9 minutes ago, Steve Cooper said:

Are you saying that the processing time for each individual record gets higher as the total number of records to be processed increases?

yes

[RyanESmith7]

I'll second dan's comments. Also, are there any containers in the data that you're sync'ing? 

[Lee Smith]

7 hours ago, RyanESmith7 said:

I'll second dan's

Of course you do, you  Smiths all stick together LOL

[RyanESmith7]

LOL! Join in and agree, Lee!

Steven, take note of the ES_Exclude flag in each of your tables. My flagship product sync's about 30 tables and some of them I'm able to filter out records older than 90 days (not relevant in the "offline" version of my app), and others I filter out based on the "active" state of the job. This means that my mobile solution is 10% of the size of my hosted database. Much quicker sync times. For me, my sync's take 3min and it almost doesn't matter how many records (ie, it does 1 record in 2:30, and it will do 100 records in 3:10, roughly speaking). 

 

[Steve Cooper]

Thanks for the feedback guys.  There are some images in the 150k records (the more recent ones). 

I'll try a couple of tests: first to reduce it to about 1000 records; second to eliminate all images.

I'll see what comes of that.

Note to Ryan:  This was a preliminary test - it is my intention to restrict the records to be put on the iPad as I move to a production system. 

[RyanESmith7]

Good move Steve. What I normally do when setting up a system for EasySync is do one table at a time. That helps me find the performance issues without having to get too involved. And be ready to change how you store data. As a result of my first offline system, I learned that I needed to move ALL my container data into separate tables in order to help with performance.