EasyDeploy: Why use segments instead of relationship?

[Joshua Willing Halpern]

Hi, 

I have a db ~27Mb. Using EasyDeploy As Is takes very long to complete (~20 minutes). Each segment takes around 10 seconds and there can be anywhere 90 to 300+ depending on the $segment_size.

I modified the EasyDeploy file by adding a table occurrence for the hosted file's EasyDeploy table and then replacing the script's loop with a simple Set Field that copies the B64 data from the host's field and decodes it locally into the container field. The new wait time is < 1 minute.

Soooo... My question is why use this looped Get (ScripResult) method instead of simply grabbing the data through a relationship and then closing the EasyDeploy file when done? I have a hunch it has to do with Tim wanting to abstract the process as much as possible but I'm not sure. Maybe a relationship has other consequences?

Josh

[siroos12]

Joshua, 

 

I have the same problem, my file is almost 100 mb. I know it is big, but I can not make it smaller at the moment. Can you please elaborate more clear how did you modify the script? or maybe post a sample file here?

[Joshua Willing Halpern]

42 minutes ago, siroos12 said:

Joshua, 

 

I have the same problem, my file is almost 100 mb. I know it is big, but I can not make it smaller at the moment. Can you please elaborate more clear how did you modify the script? or maybe post a sample file here?

Hi, yeah 100MB is going to sting using the segments method. I've uploaded a sample file.

To install, do the following in this order (in the sample file): 

1. Go to manage > external data sources and input the correct server/file info for your hosted solution.
2. Go to Manage > Database > Relationship Graph and correct the xJoined table occurrence to reference the EasyDeploy table in your hosted file.
3. Go to the Upgrade Solution script and fix the Set Field step on line 25. The calculation should be the Database_Container in your hosted EasyDeploy table.

Note: By default EasyDeploy saves the new database to your documents folder. If you want to save it to a custom folder location on FMPro, call the SetPath script from your client db and pass your the file's current directory location (you can do this with an fmp url). The upgrade database script will use this instead as the save location for the new file.

 

While this method is faster, I think it's imperfect for at least two reasons.

1. We're unable to quantify progress during the set field step, whereas segments will tell you how many have completed as they do.
2. The external file table occurrence, once accessed, will hold the connection to server open until the whole EasyDeploy File is closed. Therefore, we must automatically close it after update instead of showing a dialog and waiting for the user to exit.

Hope this info was useful.

EasyDeploy v2_SAMPLE.fmp12.zip

Edited February 29, 2016 by Joshua Willing Halpern

[Joshua Willing Halpern]

On 2/29/2016 at 1:00 PM, siroos12 said:

Joshua, 

 

I have the same problem, my file is almost 100 mb. I know it is big, but I can not make it smaller at the moment. Can you please elaborate more clear how did you modify the script? or maybe post a sample file here?

Did this speed things up at all? Or did you find a better solution?

[siroos12]

Hi,

 

Sorry, I supposed to get back to you sooner. Actually I am loaded with tasks these days. I did not have time to test it, I will test it later this week and defiantly update you here.

 

Thanks for your support.

 

[siroos12]

On 2/29/2016 at 11:39 PM, Joshua Willing Halpern said:

1. Go to Manage > Database > Relationship Graph and correct the xJoined table occurrence to reference the EasyDeploy table in your hosted file.

I did change the Table Occurrence to address EasyDeploy table on my Host file. But the relationship still says it is linked to an unknown field. Should I make any modification there?

[Joshua Willing Halpern]

1 minute ago, siroos12 said:

I did change the Table Occurrence to address EasyDeploy table on my Host file. But the relationship still says it is linked to an unknown field. Should I make any modification there?

Because it is a cartesian join, you can use any fields. It won't change the behavior. I had it hooked up to 'comments'.

[siroos12]

On 3/10/2016 at 11:28 AM, Joshua Willing Halpern said:

Because it is a cartesian join, you can use any fields. It won't change the behavior. I had it hooked up to 'comments'.

Joshua,

 

It is working and it is much faster, Thank you so much. The only issue is that when the upgrade process is finished and it tries to open the new file, it shows the logging in window and close it after 1 second, it does it for several times, and then you can not log in to the file at all, you have to kill the Filemaker go and restart it, after that it allows you to open the file. any thoughts?

[Joshua Willing Halpern]

2 hours ago, siroos12 said:

Joshua,

 

It is working and it is much faster, Thank you so much. The only issue is that when the upgrade process is finished and it tries to open the new file, it shows the logging in window and close it after 1 second, it does it for several times, and then you can not log in to the file at all, you have to kill the Filemaker go and restart it, after that it allows you to open the file. any thoughts?

Hmm that's interesting. I'm afraid I don't know off hand why that's happening, so I think some diagnostics are in order. First I'd check your onFirstWindowOpen scripts for the updated file for manual 'relogin' steps (looped attempts?).

Perhaps it is a subsequent step(or steps) after the 'export container [automatically open]' step in the updater script. Maybe the final 'pause' step? Try disabling steps and see if the problem is solved.

If not you could run a simple test. Create a new file with a container holding another password protected Filemaker file. Write a script that exports the container and automatically opens that file, then closes itself. In other words, see if you can replicate the problem in a new simplified file.

Maybe instead of using the Automatically Open option when exporting the file we can open the file with a separate fmp url step.

Try a few options and see if you can find the issue.

[siroos12]

On 3/12/2016 at 1:49 PM, Joshua Willing Halpern said:

Hmm that's interesting. I'm afraid I don't know off hand why that's happening, so I think some diagnostics are in order. First I'd check your onFirstWindowOpen scripts for the updated file for manual 'relogin' steps (looped attempts?).

Perhaps it is a subsequent step(or steps) after the 'export container [automatically open]' step in the updater script. Maybe the final 'pause' step? Try disabling steps and see if the problem is solved.

If not you could run a simple test. Create a new file with a container holding another password protected Filemaker file. Write a script that exports the container and automatically opens that file, then closes itself. In other words, see if you can replicate the problem in a new simplified file.

Maybe instead of using the Automatically Open option when exporting the file we can open the file with a separate fmp url step.

Try a few options and see if you can find the issue.

Joshua,

 

It works perfect and much faster, thanks mate.

 

Regards,

[Joshua Willing Halpern]

6 minutes ago, siroos12 said:

Joshua,

 

It works perfect and much faster, thanks mate.

 

Regards,

Glad to help. Stoked it works.

[tmr_slh]

I don't know why you copy the B64 field. You can save up to 30% of the data if you just copy the container field directly. I also make the target container field a global field to avoid the need for creating any relationship between the TOs. 

Upgrade time down from 20 minutes to 11 seconds.

[Joshua Willing Halpern]

@tmr_slh. You're right, I'm not sure what I was thinking--I was very young and impressionable back then (6 months ago). I love your idea about the global field, I must try that.

[bcooney]

Well, we finally got around to this mod since we're . In our scenario, we store version in the host's EasyDeploy table. So, we needed the cartesian relationship to the Host's EasyDeploy table to be sorted desc. by version so that we grab the latest version. (just in case this helps).

[Joshua Willing Halpern]

24 minutes ago, bcooney said:

Well, we finally got around to this mod since we're . In our scenario, we store version in the host's EasyDeploy table. So, we needed the cartesian relationship to the Host's EasyDeploy table to be sorted desc. by version so that we grab the latest version. (just in case this helps).

Good idea. Some people store only one new version at a time, but for those who keep multiple this is the way  to go. 

[Joshua Willing Halpern]

On 6/30/2016 at 10:02 AM, tmr_slh said:

 

I don't know why you copy the B64 field. You can save up to 30% of the data if you just copy the container field directly

 

I should mention to any newcomers that the example file I added in the third comment of this thread back in February actually just copies the field directly as tmr_slh suggested.

[iOllivier]

Just implemented new way of delivering the file to the mobile user.

Still it's based on the FMEasyDeploy framework.

Instead of downloading new version in segments .. or as recent proposal to copy via relationship all I am using is a DropBox link.

Server app on each new version saves copy to a dropbox folder. It remembers link and this link is only exchanged with clients.

Now all you have to do is download a version using insert from URL with dropbox link.

By my measures ~10 times faster .. and you get automatically a download progress bar at the bottom of the app. 

I recommend this method to those with a large initial data set.

200MB file takes <1 minutes on 4G and ~6minutes on 3G to complete the transfer.

Edited August 13, 2016 by iOllivier