bogwort47

Thank you very much for your responses - both prompt and helpful.

Gentlemen - Thanks for getting my brain straightened out. I eventually found a free test version of File Splitter SE (also in Apple Store) which works like a dream. I've split the files into ~ 500,000 Mb chunks (quite quick), used Text Editor to change to csv (~6 hrs) and now I'm importing into FMP the 1st chunk (looking like about 5 hrs). For what it is worth Excel imports the original file just fine without converting to csv BUT the version I have has a max page size of ~65K lines - so with 30 x106 lines to import that would look like ~ 0.5 x 106 pages - unless there is a bigger version of Excel?? The question is now: "When will FMP be modified so that the 'field separator' can be selected by the operator before import OR better still determine for itself what the separator is after getting an answer to the question 'How many fields in original data?'?" Thanks again,

Sorry I'm a bit thick - Terminal Application?? OK - In Utilities I guess

Hi Lee Thanks for that - I've been a bit confused this week and I was really beginning to think I was loosing it - thank you for your patience - I've tried Wrangler with the larger file both copying across and using the insert file content function but failed because of insufficient memory. I guess because I am limited to 2G that even with the smaller file I'm still looking at a file splitter?

Hi txt = Text Editor, sorry I was being lazy. Character Count? The file lines I gave a sample of above are from a population of 28,194,115 lines - 1.02 GB on disk (1,022,402,995 bytes) The largest file has about 23.5 Million lines 2 to 3 times the character count - 2.16 GB on disk (2,155,500,248 bytes) This is the link to where the files are: http://data.gov.uk/dataset/anonymised_mot_test The 2 problem files are the 2013 files, bottom left. Sorry, FMP 8.0V1 not '7' which crops up in the file extn as 'fp7'. Is this OK? Thanks for coming back to me. PS I am confused - I cannot find my original post and your questions now - Heh Ho I just copied one of the files and pasted it into TextWrangler without any problems. I then did a find and replace | with t and it seemed to do what you want? Sheesh, Lee. I didn't reply here because I didn't want to ruin your work - and now you do? Please update your profile to Reflect your current version, here is a quick link for your convenience. MY PROFILEhttp://fmforums.com/forum/index.php?app=core&module=usercp&tab=core&area=profileinfo Hi comment, What did I do? I didm’t mean to screw things up. Uhm, you need to move these last ones (starting from #9) to the new thread you opened here? http://fmforums.com/forum/topic/91080-how-to-deal-with-large-text-imports/ Ok - I've done that - it now shows '8'. Sorry, I'm getting confused here - are you suggesting I get hold of Text Wrangler and try that please?

Hi Everyone I am new here - so please be gentle with me. This thread is the nearest thing I have found so far relating to my problem. I need to convert? and load into FMP on a Macbook txt files of size about 2.2Gb. Here are a handful of lines from a file: 17|530|A||A|| 22|548|A||B|| 22|548|A||A|| 44|7003|P|2||| 52|8553|A||A|| If I extract a sample and use txt to find and convert to csv, the data pops in OK. My problem is the size of the files which are too big to change in txt. I have tried a number of file splitters from the web without success and I am very much a beginner with code writing - the last time was with my BBC B Can anyone help me please? Many thanks