Converting / importing .fp7 backup into fmp12 Runtime (urgent)

[devices99]

I have a v11 Runtime that we are upgrading to v14 - it's used by hundreds of users across the country.

They have a backup routine which exports all the data into FP7 files (we chose this format, as some of the tables have a small amount of user defined container data in older version, which we'll slowly phase out now), it saves each file as a .bak so it's easy for the user to know which files are the backups, mainly when the upgrade to a newer version.

This has been fine over many updates/new versions which we've written over the past few years, our restore function imports the data back smoothly. Until now...

Moving to v14 runtime is a massive headache due to the file format changes, as FM doesn't allow users to import data only from the older .fp7 file version (how thoughtful).

I've written a sophisticated backup conversion routine, which takes the users set of fp7 backup files (named .bak files), and converts them to fmp12, it then removes the old files and renames the converted ones to the original .bak filename - so no changes are needed to our restore procedures. Written all natively in Filemaker using tricks with container fields to check files exist, and rename by exporting out again.

This works perfectly in FMPA14, however it fails in the runtime with the error (each time trying when trying to convert an old backup file): "The file cannot be opened because it does not belong to this solution".

Obviously I can't bind the backup files to the solution, as each and every users will be different.

(It actually appears on a test, that the runtime doesn't even convert the files presumably because they 'don't belong to the solution')

If it hadn't been for older versions with containers, I'd have just used a csv backup files and this problem wouldn't exist - however it does, and I urgently need some help!

Any ideas and pointers gratefully received, 21 hours on it this weekend and counting!

 

Edited November 15, 2015 by devices99

[Lee Smith]

I deleted your replies to those other two threads, and agree that this should be your own question and have your own thread.

I was concerned about you waiting for the OPs of the others threads returning to answer your question. BTW, please do not multiple post your questions. Find a place you think it will get the answer you need and ask it there. 

 

p.s. if you feel another thread would be helpful to your own questions, you can always link them to your Thread.

Edited November 15, 2015 by Lee Smith
p.s.

[comment]

@devices99

FIY, it is extremely rude to mark your post as urgent. Everyone here donates their time and effort to the community. You have no right to rush anyone.

FWIW, I believe your question has already been answered by @bcooney in one of the other threads you have replied to.

[NLR]

I sympathise.  I can see your problem and imagine the pain and frustration at not having a clear way to get user's data out of the old and into the new.  I have had quite a lot of experience in providing updaters.  As I see it you have the worst possible situation in trying to jump two hurdles.  First the conversion from an fp7 file format to the new fmp12, and the second being that a Runtime application is absolutely restricted to only "knowing" its own bound files with that internal binding code.

Each case on its own presents challenges but both together... (a really tough one).  I know it's always nice to have a press-button answer whereby a user merely clicks and the whole process is fully automated.  In your situation this is hardly possible.  There are a few ways to go and it appears you've looked hard. Further to that, to consider a "best/least painful" way it would be helpful to know a bit more:

1) How many files?
2) How many tables within each file, and the record counts involved (just roughly).
3) Can you accept loss of container data, or are you prepared to look at workaround to save images.  (Such as another script to manage exporting of field contents looping over all records)
4) Global fields are far less of a problem because only one value per table (if you need to do it) is required.
5) If there are repeating fields, you have an extra problem to look at and solve. (It's possible but tricky)

 

If you are happy to confine data retrieval to all types except containers and repeating fields, one solution I'm thinking of is based on your saying that "This has been fine over many updates/new versions which we've written over the past few years, our restore function imports the data back smoothly. Until now…"   

I would therefore do it in 2 stages:
Stage1 means providing your users an (old format) updater to their solution as you've done in the past.  In it there would be a script (and button) which basically exports each table (of each file) in a format which is universally understood - such as XML.  That way you get the field name info along with the data.   The script would loop through each file, then each table, show all records, unsort them, and export.   A scheme for naming each export file is important so you can match later (knowing such names in advance).  For $Filepath I would name it after the BaseTable, for example "file:Contacts.xml".  Saved to local folder, (unless you can manage the extra step of providing them with a "SavedData" folder beforehand).   It would then be "file:/SavedData/Contacts.xml"
So then you have a stack of xml files such as "Contacts.xml",  "Invoices.xml", "LineItems.xml"  etc etc.  - All this for one file.  (If you have multiple files, include part of the file's name prior to each Table name).

Stage2.  At this point your new runtime system can take over and import from all those xml files.
I think this should work.  You would obviously need to rigorously test beforehand.
There are other aspects too.  Things like Value Lists, and Serial numbers...

Regards
Ralph

 

[devices99]

Ralph thank-you so much for such a helpful and considered reply in my hour of need (and not being offended by my use of the word "urgent") – it is much appreciated! I think a conversion app updater is the only way to go, I've exhausted all other possibilities.

I'm happy to dump the container data, as we've moved away from those facilities in the last version anyway.

Ralph, would you recommended using XML or a MER file as a backup/upgrade files from now on, as we never want to be caught out like this when the file formats change again.

We have 5 older versions stretching back 4-5 years with different binding keys (it's a world wide project with hundreds of customers), so that is the other issue.

Do you know if FM12 has the ability to import fp7 files without conversion (unlike FM14), as I was considering downgrading to this 12, but having only moved from 11 to 14, I didn't want to request the downgrade unless it was possible - and I've made use of many v14 features which possibly may not downgrade.

Thanks again Ralph.

[NLR]

Regarding XML versus MER, I think either would do.  I used to think of XML as safer because of its delimiters. Then again I've just now done some practice exports and re-imports back into Filemaker.  So far I haven't been able to trip up the MER format.  It seems to handle carriage returns, double quotes, tabs, and other characters just fine, and for ease of reading the raw file (if for no other reason), I'd now use that.  On checking FM Help I see that the character separating fields varies according to language, if that's a consideration.

Regarding the use of FM12 versus FM14 for importing fp7 format files… I know that FM12 behaves the same, that is both require conversion to fmp12 format before importing.

Regards
Ralph

Edited November 16, 2015 by Ralph Learmont