Jump to content

How to import records from XML file > 10 GB


This topic is 3642 days old. Please don't post here. Open a new topic instead.

Recommended Posts

Oh, it's Government Federal State Address database, that gives open source in XML, DBF and Cladr russian database format. Cladr impossible to use in FM, DBF has incorrect CP1251 coding in import of russian letter. And I have only a chance to import from XML.

Thank you. May be you will advice how and where I can do pre-processing outside of FM for easiest way?

Link to comment
Share on other sites

Pre-processing kinda depends on what you are comfortable with.  For instance you could create a VBscript / Powershell / AppleScript / shell script that fixes the CP1251 coding in the DBF format, or that parses the XML into a CSV for import into FM, or goes through the XML and deletes any nodes that you do not need but preserve the XML otherwise...

Something along those lines.

Link to comment
Share on other sites

Pre-processing kinda depends on what you are comfortable with.  For instance you could create a VBscript / Powershell / AppleScript / shell script that fixes the CP1251 coding in the DBF format, or that parses the XML into a CSV for import into FM, or goes through the XML and deletes any nodes that you do not need but preserve the XML otherwise...

Something along those lines.

I think now I need to learn applescript... I don't imagine how script fixes cp1251 coding. How I understanding to convert file from xml into csv better to use replace text function. Is it right?

Link to comment
Share on other sites

How I can import XML file that is about 12 GB.

I have an error after about 4 hours of parsing file, with no text in error dialog window.

 

It sounds like you have run out of memory. Your best way to proceed, IMHO, would be to split the file into smaller chunks. See for example:

http://fmforums.com/forum/topic/90048-import-plain-text-to-fm/?p=418221

 

Note, however, that splitting an XML file is not as trivial as splitting a .csv file, for example.

 

 

---

 

 

DBF has incorrect CP1251 coding in import of russian letter.

 

This would be a lot easier to answer if we knew what is the exact problem with the encoding. Did you manage to import the data? Perhaps all you need to do is a series of character substitutions (in Filemaker, after importing)?

Edited by comment
Link to comment
Share on other sites

This topic is 3642 days old. Please don't post here. Open a new topic instead.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.