Jump to content

Payload Timing


bcooney
 Share

This topic is 1846 days old. Please don't post here. Open a new topic instead.

Recommended Posts

Hey fellow syncers, stepping back from any method, I was wondering if there's an issue that all sync solutions have. The issue is this:

Client A pushes a payload, and it's processed and ready for ClientB to receive...all nice and transactional. However, ClientB gathers records to pull a table at a time.

Couldn't ClientB gather Orders - ClientA's processing completes - and ClientB gathers Order Line Items (thus, missing the parent record of some of those Order Lines since ClientB has already pulled Orders for his payload?).

We're seeing this and wondering if it's a design flaw in EasySync or indicative of any sync solution that allow multiple users to sync at a time.

Link to comment
Share on other sites

It's a thing!

Sync tools do not have paranormal psychological abilities. Its up to you to provide explicit instructions to the sync tool that control which records are available for syncing. In practice this is difficult because real world situations are complex.

It's much easier to tell people to sync frequently, as the second, third, fourth, etc, sync clears up problems in the first, second, third, etc, sync.

Link to comment
Share on other sites

Quote

Its up to you to provide explicit instructions to the sync tool that control which records are available for syncing.

Ah, but you really can't. Hmm.

We're thinking of one user sync at a time. Setting/clearing a system flag.

Link to comment
Share on other sites

Todd Geist suggested that the developer forces a parent record to be marked for syncing when a child record is added or modified.

Not sure if this is the problem you are describing.

I had to write code that finds orphan records but for a completely different scenario.

Link to comment
Share on other sites

Not exactly the issue we've encountered. It's a timing thing.  Easy sync ( or any solution that syncs by looping through tables ),  gathers records a table at a time. A client may have already gathered records from a parent table and then moves onto the child table. If records become available in the child table, they will be gathered, but their parents will be missed.  

Link to comment
Share on other sites

@bcooney I'm still not using EasySync in any client application's so I wouldn't have noticed this yet myself, but it does seem to be a limitation of the method used for syncing.

Quote

We're thinking of one user sync at a time. Setting/clearing a system flag.

I would suggest opening/closing a record instead of setting a field to control this. The problem with setting a field is when a client looses connection in the middle of a sync, or some other error occurs; that flag will remain set to "sync in progress". If you create a record (or table with one record) and open/close it when starting/completing syncing, then that lock will be more likely to clear on it's own. In the case of a client disconnect, for example, that record will be closed when the client disconnects (although I suppose it takes a bit for the connection to time out on the server).

 

The only other possibility I can think of is to store the time the sync start, and the time it finishes, then add one more process at the end of a sync to see if any records were changed between those two times. Then you could run another sync to get the missing parent records. Note, you may need to resolve the following issue before this method works as expected: https://github.com/dansmith65/FileMaker-EasySync/issues/24

Link to comment
Share on other sites

Oh boy, Dan. Thank you for helping me think this through. I learned from Mike Beargie on TechNet that this issue is referred to as "Collision Detection Logic," and he believes that Mirrorsync addresses it.

If I understand you correctly, you'd dedicate a one record table on the host as "the switch?" Open that record and commit it at the end. No one can sync if that record is locked? Interesting concept.

Thanks for pointing me to the other issue.

Perhaps time to look to Mirrorsync. :-(

Link to comment
Share on other sites

This topic is 1846 days old. Please don't post here. Open a new topic instead.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Similar Content

    • By 360Works
      How to Build FileMaker Web Apps for High Traffic using MySQL Backend and 360Works MirrorSync
      Why Sync FileMaker and MySQL?
      FileMaker is the best tool for rapid application development, but it is not the most reliable or scalable option for high traffic web sites. MySQL provides that reliability and scalability, but can't compare to FileMaker for development productivity and user friendliness.
      360Works MirrorSync bridges this gap, letting you use each tool for its best purpose. Quickly create layouts, scripts, and schema in FileMaker Pro, test and refine them, then use MirrorSync to publish all your record data to MySQL and keep both databases in sync moving forward. You can use industry-standard tools and frameworks (or hire from a vast talent pool) to create highly reliable and scalable web apps.
      In the event that FileMaker Server is down or rebooted for any reason (such as software upgrades), your MySQL web app will continue running and serving users. MirrorSync will detect when FileMaker Server comes back online and bring both databases back in sync.
      Video Demonstration
      This video, narrated by Junior Perez of 360Works, takes you on a step-by-step walk-through of how to sync between FileMaker Server and MySQL. Junior demonstrates how to start with a completely empty MySQL Server, create the database and tables, and set up an auto-scheduled sync to run every 60 seconds. The process is the same for any other JDBC-compatible database such as SQL Server, Oracle, or PostgreSQL. The license price for syncing one FileMaker Server with any one of these databases is $1,800, with discounts available for educational and non-profit organizations.
      If you would like to involve 360Works in configuring your sync, or in creating a web application backed by MySQL, please contact us for consulting and development assistance.
       
       
      Read More on MirrorSync or Download Now!
      360Works MirrorSync     Stay updated with 360Works
         
    • By 360Works
      New MirrorSync Update Now Available
      50 Improvements Made in MirrorSync 6.1
      360Works releases new point update for MirrorSync with 50 improvements and fixes to optimize FileMaker synchronization.

       
      360Works MirrorSync is the fastest and easiest synchronization tool for FileMaker. It allows everyday FileMaker users the power to synchronize their FileMaker Server files with FileMaker Pro, FileMaker Go, other FileMaker Servers, SQL databases, Salesforce, WordPress, Amazon Web Services Databases (DynamoDB and Redshift). MirrorSync was awarded FileMaker Solution of the Year in previous years and the product continues to be known for its scalability, reliability, speed, and ease of use.

      The latest update, MirrorSync 6.1, ships with 50 improvements and fixes. Here are some of the newest improvements by category:
       
      Container Fields and Large Data Improvements
      Optimized conflict resolution field merging; now about 100x faster for large tables! Now supports streaming for container uploads, so that you can sync any size container data (such as large video files) without running out of memory in MirrorSync Improved performance when syncing large numbers of fields (>500) Now truncates extremely long container filenames (converting a text field to container fields causes the text contents to also be used as the container filename, which can be too long to be practical) Performance improvements for configuration client Important fix: removed a deadlock that could occur when inserting large number of records that contained unusually large text data as well as container data  
      Syncing with Non-FileMaker Databases Improvements
      No need to do any time zone conversions when writing time and date values to Salesforce, you can now send them unmodified Improvements were made to SQL generation for syncing to a non-FileMaker SQL spoke database, such as MySQL or SQL Server Performance improvements for DynamoDB; prevented unnecessary update operations if fields are added or removed from a configuration  
      FileMaker Server to FileMaker Server Sync Improvements
      Improvements for externally stored container fields that have an .fmp12 file in them when transferring from hub to spoke during server-to-server syncs Much faster transfer of large multi-file solutions between servers for initial server-to-server syncs  
      Configuration Improvements
      Improved responsiveness on the configuration screen where tables, scripts, and layouts are copied and pasted Added extra information at the top of the sync status that always shows the name of the configuration and a description of both the hub and spoke databases Exporting a configuration from server A and importing to server B will now automatically change the configured MirrorSync URL to server B Smarter switching between using the internal and external IP addresses on the sync. This should make it so that syncing from outside the LAN is about 5 seconds faster, if there is a separate internal and external address configured. We now keep a history of all configuration changes, in case any issues arise  
      Improvements for Keys
      If you're using UUID primary keys with unique validation enabled, and if you change validation to 'always' (this is not set by default), then MirrorSync will automatically re-write inserts as updates to avoid duplicate records, or error messages about duplicates. Improvements to detection for foreign keys, as well as skipping over any foreign keys referencing tables or columns that are not part of the sync. Improved validation for foreign key configuration Fixed a problem when using custom text primary keys of varying lengths (not UUIDs or serial numbers, which work fine without this fix). If the column that a foreign key is based on is excluded from the sync, the foreign key is now removed from the configuration and skipped  
      Improved Error Reporting
      Improved error reporting to users if the sync fails due to a permission error or timeout [Worker script] Many improvements to automatically handle and retry network problems; better logging to identify problems caused by slow networks [Worker script] When the offline file has external data sources pointing to non-existent files, MirrorSync responds with better error reporting.  
      More Improvements
      Additionally, we have added support for configurations and synchronization with FileMaker Cloud 2 (Please note: due to the limitations of FileMaker Cloud 2, it is not ideal for synchronization. Contact support@360works.com for additional details if you plan to use FileMaker Cloud 2).
       

      Visit the MirrorSync changelog for the complete list of all 50 improvements and fixes available in 6.1!
       
       
      Why MirrorSync?
      MirrorSync allows users to work on their own offline copy of a database on their iPad, iPhone, or laptop computer, even when they have slow or no connection to the internet when working from home or out in the field. It can also be used for server-to-server syncing between servers in different locations and with other distributed server models (i.e., clustering, load balancing). MirrorSync documentation for further information is available at the 360Works Documentation Wiki.
      MirrorSync Hosting and Consulting
      We offer both FileMaker Hosting and MirrorSync Hosting. If you host FileMaker with us, you get MirrorSync hosting free!
      *Purchase of MirrorSync licensing is required. MirrorSync is completely free to sync FileMaker Server with FileMaker Pro or FileMaker Go with one device. Any additional configurations or seats are purchased as needed.

      Users running regular FileMaker Server on a Mac or Windows computer can install MirrorSync on their own server.

      We've designed MirrorSync to be something that even a novice user can integrate, but we're standing by when you need more advanced help. If you just want your sync integration to be completed as quickly as possible, we're available to do the work for you directly at our hourly consulting rates. Simply contact us to get started!
      Purchasing and Upgrading
      Existing MirrorSync 6 users can now upgrade to MirrorSync 6.1 at no additional cost. For users new to MirrorSync or past users with expired maintenance (maintenance expires one year after purchase date if not renewed), now is a great time to purchase a new MirrorSync license.

      If you need to check your maintenance on your MirrorSync license, visit the 360Works License Portal page. If you need to renew your maintenance or buy a new license, visit the MirrorSync product page.

      As always, MirrorSync is completely free to sync FileMaker Server with FileMaker Pro or FileMaker Go with one device.

      Free 14-day MirrorSync server-to-server configuration trials are available upon request. Please contact us to request your free trial today.
       
      --
      Get New Licensing, Upgrade, or Learn More: MirrorSync Product Page   Existing MirrorSync 6/Active Maintenance Users: Download the 6.1 Update Now 
       
      More questions? Email support@360works.com
       
    • By 360Works
      FileMaker DevCon Playlist and Products from 360Works to Help You Work Better
      Tune into Popular FileMaker DevCon Sessions from 360Works
      With more time at home, now is a great time to catch up on past recordings. Did you miss any of the FileMaker DevCon sessions from 360Works? We put together a playlist of our most popular sessions from the past two years so you can learn about FileMaker tools for sync and Amazon Web Services (AWS) tips all in one place.
      Watch the Videos About the featured products...
        360Works MirrorSync
      MirrorSync is an elegant synchronization product that can sync between any combination of FileMaker Pro, FileMaker Go, FileMaker Server, SQL database (MySQL, Oracle, SQL Server, or any database that supports JDBC), Salesforce, WordPress and Amazon RedShift and DynamoDB. For mobile devices, users can sync the hosted file with their FMGo or Pro clients, go off network to make changes, and sync again when a connection is available. For FileMaker Servers, MirrorSync allows a database to run in multiple locations, or to integrate different databases and make sure that changes to each server are reflected in the other server. MirrorSync 6 introduced tremendous improvements for FileMaker Developers such as automatic file transfers between servers, automatic version updates between servers, safer synchronization process, better handling of large files, and easier migration.
      360Works Zulu
      Zulu is an application that keeps your calendar events in sync with FileMaker records. It is able to sync with Apple Calendar, Google Calendar, and Exchange Outlook Calendar. You can manipulate the events in your calendar, and Zulu will write these changes to FileMaker records. The opposite is also true, updates to FileMaker records will be written to your calendar's events. Zulu 4 introduced improvements specifically for business workflows such as new support for Apple Reminders within FileMaker, one-click URL calendar subscriptions and improved publishing page for user-friendliness, better event handling, and improved filtering.
        You May Also be Interested in: 360Works Scribe
      While there was not a session for Scribe, we thought it was worth mentioning because of the overwhelming positive response we received for Scribe 4, which introduced OCR to easily scan documents in FileMaker and merge or splice PDF files in container fields. Scribe has always been able to automate processing of Microsoft Office and PDF files. It continues to allow you to populate forms, insert images, highlight changes between versions of documents, manipulate Microsoft Word and Excel tables, retrieve metadata from Excel table cells, and secure PDF files.
      Interested in more than one of the tools mentioned?
      Get the 360Works Portfolio Bundle Deal
      For less than the cost of 3 of our most popular plug-ins, you can get the entire 360Works product portfolio, all upgrades and most new releases for a year, and a single license key that works for all covered products.

      See exactly what is included in the Portfolio Bundle Deal.
      360Works Product List:
      Product Name Product Description Portfolio Bundle All of our FileMaker Plug-ins for one discounted price Charts Easy-to-use charting for FileMaker CloudMail Send and track email marketing campaigns Email Send and receive emails, plus send SMS/Text Messages from FileMaker 360Deploy Automated, simple deployment of new database versions FTPeek FTP upload and download from your FileMaker database JDBC SQL access with JDBC Compatibility MirrorSync Sync FileMaker Go and Pro on a device without network connection, sync FileMaker Servers, and sync FileMaker with SQL databases, WordPress, Salesforce, or AWS databases Plastic Payment processing plugin RemoteScripter Trigger scripts on FileMaker Pro from the web or IWP SafetyNet FileMaker Server offsite backup Scribe Read and write PDF, Word, and Excel files, plus OCR functionality and PDF Merge/Splice in Scribe 4! ScriptMaster Adds many new functions & create custom plug-ins SuperContainer A better container field Web Assistant Access internet resources, URLs, and fill out web forms Web Services Manager Publish FileMaker scripts as XML Web Services Zulu FileMaker calendar sync - integrate with Google Calendar, Apple Calendar, and Exchange Calendar You can get a free fully-functional demo, learn more, or buy the product that meets your needs at any of the links above.

      If you need AWS consulting, custom FileMaker Plug-ins, or have any questions. Please email us at infobox@360works.com
        Watch the Videos   Stay updated with 360Works


      Sincerely,

      360Works
      plugins@360Works.com
       
    • By 360Works
      Case Study: FileMaker Clinical Data Capture Software Uses 360Works MirrorSync
      Learn how MirrorSync is being used in the ConnEDCt app built with FileMaker for medical research and healthcare improvements across the world
      “We have used MirrorSync from versions 3 to 6 for FileMaker Pro/Go to FileMaker Server. In our experience it is the most reliable technology for syncing FileMaker databases over inconsistent and slow connections.”
      
- Caleb Ruth, Data Performance LLC
      At 360Works, we believe in making FileMaker plug-ins and tools with outside-of-the-box thinking to help FileMaker Developers build what works best by extending the functionality of their FileMaker solution to solve real-world problems.

      FileMaker Developer and Business Owner, Caleb Ruth, was generous enough to spend time sharing with us some of the aspects of his work with health information systems which leverage 360Works MirrorSync.
      Read the Full Story   Stay updated with 360Works
          
    • By 360Works
      Fireside FileMaker Podcast: 'Keeping it in Sync' featuring Jesse Barnum on MirrorSync and more!
      We have the perfect FileMaker podcast for your listening pleasure as you ease back into the office and start the new year!

      Fireside FileMaker is a podcast collaboration between John Mark Osborne and Michael Rocharde discussing FileMaker related topics in a friendly and informative space.

      Today, we wanted to share the 'Keeping it in Sync' episode of the podcast with you, featuring Jesse Barnum, President of 360Works, for a conversation on MirrorSync, plugins, FileMaker Cloud, Claris Connect and other interesting FileMaker subjects!
        LISTEN HERE   *Learn more about the NEW MirrorSync 6 and download here

      *Follow Fireside FileMaker on iTunes or Podbean
      Stay updated with 360Works
       
  • Who Viewed the Topic

    1 member has viewed this topic:
    Daemon 
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.