Alternative Delete Sync/Method Idea

[GisMo]

I've had to disable the native delete records sync because it takes WAY too long to do the DIFF with large tables. I've been looking at other ways to try to sync the delete and would like to table an idea that I have. The process would be something like this:

1. Each table would get a flagField called "delete". Initial value=0 or empty
2. When a record is deleted on the client by the user, the flag field gets marked for deletion(value=1). 
   • Maybe with some UI logic, we can hide these flagged records
3. User1 Sync with server push the flagged for deletion records to the server.
4. After the Sync is complete, the User1 client runs a script to actually delete all records that are flagged
5. User2 will Sync and pull down and new records and the records that are flagged for deletion
6. After User2 sync is complete, User2 client script will delete all flagged records locally
7. (Here is where I see an issue)I think the flagged deleted records will always need to live on the server, but they will never be able to be modified since the clients will delete them locally. This way, any tardy users will always have access to the deleted records. Maybe once per month, the server will purge the flagged records.

I'm going to be testing this out soon. Any thoughts on this process? Any ideas on how to make it more robust?

 

 

[Joshua Willing Halpern]

Hi. If flagging, the records should live on server indefinitely. However, I like the sound of this method a lot. 

I solved the Sync Check performance issue using javascript in a native webviewer. I've got it Sync Checking ~23000 records in 18 tables in a few seconds instead of 10 minutes. It works like:

1. Server 'Sync Check' script returns a FULL list of all its ES_UUIDs.
2. Client generates its own full list.
3. Client formats both lists as arrays and saves them in variables. It then saves these into a $url variable and drops them into a webviewer.
4. The webviewer uses a Javascript filter to find the diff and then calls another filemaker script and passes the diff as a parameter. This script simply returns the parameter to the 'Sync Check Client' script.
5. Finally, the 'Sync Check Client' processes the $diff list as usual.
6. (Also, I modified the sync check scripts to pull the Server's UUIDs in segments if the $$max_pull_segment_size is exceeded.) 

Maybe I can use you as a guinea pig? I just need to migrate the new scripts to a generic easysync file but when I do I'll share ASAP.

I love EasySync and I think it deserves some faster technologies like JavaScript as long as we stay true to its mission of working without plugins or extra hosted files.

[GisMo]

I'm all for trying out your method! It sounds interesting!

I agree, the server would need to keep a copy at all times. I think my method might be fast as well. I'll be testing it out shortly.

In one instance I had even more records than you... about 180k records. I ended up flagging older records to bypass sync, but still there are about 20k records remaining. It was a nightmare, so I disabled it. Now I need to solution to sync deletions.

[Joshua Willing Halpern]

Here is a generic version of my modified Sync Check idea. Please note the webviewer named "WebViewer" must be on the Easysync_Payloads layout for this method to work. (more info in "READ ME NEW.txt")

Try adding a bunch of records to the solution and then when ready, delete them from the server (normal delete is fine) and they should be removed from the client much more quickly than in the original version. For me, the sync check is often the fastest part of the sync now. Any feedback is greatly appreciated!

DISCLAIMER: I can't make any more promises as to the efficacy or safety of this product than Tim can so please consider any non-liability inherited from his license agreement.

EasySync v1.3 mod1 (jwillinghalpern@gmail).zip

Edited January 22, 2016 by Joshua Willing Halpern