I am struggling with a really strange bug.

I export from FileMaker as XML. As near as I can tell, FileMaker defaults to UTF8 in this circumstance, and offers no other choice.

I define a MySql database table as UTF8.

I write a PHP script which imports the XML from FileMaker and inserts it into the MySql database.

I then view the data through a web browser (on an HTML web page where the character encoding has again been declared as UTF8) but i get garbled accents on European names such as "Rhone".

I'm desperate for some help so I'm offering a $25 prize on this question.

I've posted more details and screenshots here:

http://www.filemakeremergency.com/question/show/id/9

There is a very long chain here, and it would be useful to check along the way and see where the problem originates. First stop: open the exported file on your Mac using the free TextWrangler text editor and see what encoding is reported (at the bottom of the window).

Also state your version (incl. OS version).

Edited December 13, 201015 yr by comment

There is a very long chain here, and it would be useful to check along the way and see where the problem originates. First stop: open the exported file on your Mac using the free TextWrangler text editor and see what encoding is reported (at the bottom of the window).

Also state your version (incl. OS version).

At that point, the text editor has no problem opening it as UTF8. Of course, that is a Mac created file being opened on a Mac, with Mac software, so that would be surprising if the text editor could not open it as UTF8.

Most text editors give me the option of opening the file in any character encoding I want. Not sure where Macs store that info, but clearly Mac software knows how to read the charset of a Mac file.

I write a PHP script which imports the XML from FileMaker and inserts it into the MySql database.

My guess is that something in the PHP isn't preserving the UTF-8 as intended. PHP 6 is expected to have native support for UTF-8, but it is touch and go with PHP 5 and earlier.

Certain PHP functions can mangle the UTF-8 string since they expect to see ASCII.

See http://www.phpwact.org/php/i18n/utf-8 for some specific examples and potential solutions.

At that point, the text editor has no problem opening it as UTF8.

I am not sure what you mean by "opening as". The text editor* opens a file in the file's encoding (you can force another encoding, but then you need to re-save the file because it has been modified). If the file opens as UTF-8, then it is UTF-8. Check that the accent characters are displayed correctly, and if they are - move to the next stop:

5.) I take the resulting XML file (3.2 gigs) and upload it to my dev server

Can you open the open the uploaded file in your text editor through FTP?

---

(*) I am of course referring to TextWrangler or its big brother BBEdit. If you are using something else...

Edited December 13, 201015 yr by comment