some fields don't import from .txt file

[stefangs]

i'm back with more importing weirdness...

i've exported a merge file from a .fp5 file, containing about 1800 records. i'm only exporting 5 fields from this file, because the rest contains irrelevant stuff. i verified that this file imports ok into the .fp5 file itself.

now i want to import that same .txt file file into a converted .fp7 file and something very bizarre happens: i receive my 1800 records, but only fields 3, 4, 5 import correctly. field 1 doesn't import at all and field 2 imports for 77 out of 1800 records only.

on closer examination, i found that only those 77 records contain data in the field 1. it is blank field for the rest of the records.

the data in field 1 (of which there are only 77) is not imported at all.

i've also clicked the >> button in the import field mapping preview to see what's imported and this looks perfectly fine!

how is this possible?

i realize this may be a bit hard to read - hope you can make it out. thanks for any help,

stefan

[stefangs]

i had a go at this one again today. i tried all kinds of options to no avail.

for testing purposes, i exported a single field only from the original .fp5 file as a merge file. the field i'm exporting is the record ID. the text file reads like this:

recordID

"ARC1000"

"ARC1001"

"ARC1002"

...

and so on. about 1800 records in total.

i go back to the new .fp7 file to import this. in the import dialog (by matching names), i click the >> symbol to verify that valid data appears next to the recordID field and import.

i end up with 1800 records that have no data! nothing is imported, only records are added. i tried both with and without auto-entry options.

i'm going nuts. does anybody have an idea what's going on?

thanks,

stefan

[comment]

It's not that I am particularly interested in doing your debugging - but if you don't attach your .fp5 file and the .mer file, this is going to be at most an exercise in wild guessing.

[stefangs]

It's not that I am particularly interested in doing your debugging - but if you don't attach your .fp5 file and the .mer file, this is going to be at most an exercise in wild guessing.

ok, i'll modify the destination file so that it can be opened without all the other related files it may ask for and post that. you're right, that's better than wild guessing.

thanks,

stefan

[stefangs]

@comment: i was just about to modify my file so it can be posted. i started out by saving as a clone, and for the hell of it attempted importing into the clone. what do you know... all records imported perfectly well!

i suppose that indicates that the original file is corrupt somehow?

thanks for steering me in the right direction!

stefan

[stefangs]

looks like i spoke too soon... the import works when done manually, but fails when scripted. i have stripped the file any fields, relationships, etc. and left only the bare essentials, but it behaves the exact same way as the real thing.

1. manual import: everything is imported ok.

2. scripted import: only 73 records are imported into the 'credit' field.

perhaps you'd like to check out the attached file and spot what's wrong with it.

i'd be very curious!

thanks in advance,

stefan

[IdealData]

Attached file?

[stefangs]

i think the board is buggy... i got stuck in a loop when i tried editing my post.

of course, i meant to say that i stripped the file from all 'unnecessary fields etc., not all.

so let's try again with the attachment.

thanks,

stefan

ImportProblem.zip

[comment]

I suggest you delete the Import Records [] step from your script, and insert a fresh one instead.

[LaRetta]

"2. scripted import: only 73 records are imported into the 'credit' field."

It works fine for me on PC FMPA 11.0v3. 1839 records imported and 1839 records have text (and not just a space) in the Credit field.

[LaRetta]

I would love to see data result from broken import (can you attach fp7 with the data in it?). I wonder if there is invalid character or character which breaks the field delimiter. But different via platform? Curious ...

... or different via FM version?

BTW, Stefan please list your OS and FM version in your profile. I see you put it in your signature and I appreciate your consideration but we all don't think of looking for it there. :^)

[comment]

I would love to see data result from broken import

I have been able to reproduce the result. AFAICT, there is some "residue" in the memorized field mapping. As soon as it's reset, the import works fine.

ImportProblem.zip

[LaRetta]

Yes, I tried it originally after looking at the map first and saying OK so I inadvertently reset it. I would have gotten an 'F' as Tester today LOL. I used Notepad++ and could find nothing at all wrong with the imported data itself. We know saying OK changes the import map internally because if we print the script first, it shows the import as:

Source field 1 import to Credits::comments

Source field 2 import to Credits::credit

Source field 12 import to Credits::name

Source field 17 import to Credits::recordID

After saying OK to the import dialog, source fields print out as source fields 1, 2, 3 and 4. Is this the 'residue' you mention or did you spot something else? Stefan, this file may have crashed and if it has, it is best to revert to prior good backup. It is never worth the risk (my opinion only). Thank you, Michael, for providing me a data-play file. :^)

[comment]

Source field 12 import to Credits::name

Source field 17 import to Credits::recordID

Yes, that's exactly what I meant.

[stefangs]

comment and LaRetta - you guys really went to town with that! i would not have thought of this in a million years. thanks so much for having a look.

so the good news is that yes, when i replaced the import script step, import worked fine. that bad news is that it only worked in my crippled file. when i tried it in the real file, i had the same results as before.

since i have no idea whether or not, when, how many times, etc. this file may have crashed, i probably have no other option than to rebuild it from scratch.

also a great tip with printing the script, i didn't realize you can see the map order.

@LaRetta: i didn't attach the file with the broken import data, because i think this is what comment posted already.

btw, i updated my profile too

stefan

[stefangs]

i forgot one thing... if i have to rebuild my file, am i allowed to import things from the old file (scripts, fields, etc) or do i need to start with 'new database' and retype everything?

thanks,

stefan

[LaRetta]

You are allowed to do anything you wish. But risk increases because every piece, every icon, every script you bring over might contain corruption. Your 'new' file could then crash and cause more corruption ... and on it goes. For example, if you hadn't found out this script was corrupt and you imported it into a new file, then what?

Ideally, you should back up after every few hours of designing and save a clean clone aside with the date time (I save all copies back through a project, sometimes several months' worth). I back up every 30 minutes when I'm working heavily in a file.

It may seem like overkill but if you've ever had to go back and re-design (like you are doing now), it doesn't take long to realize that frequent cloning is worth it. Do not back up or zip a file which is open. Save As Copy or close file first.

We feel your pain ... we have all been there and done it. I am sorry you must go through it but you are learning a hard lesson early and better early than later when it might mean you have lost MONTHS worth of work like I did once when I was first starting.

UPDATE: And if your file ever wonks out, crashes or even sneezes, ditch it and grab a good backup. Once your files are served, be sure you are notified if it ever crashes so you can migrate the data to a backup. I turn OFF auto-start on server. If it crashes I want to know it immediately.

[stefangs]

yep, wise words. i better get going ;-)

thanks again!

stefan