Import XML via XSLT using named xpath select when XSD is missing?

[xochi]

I'm trying to import XML data into FileMaker 9 (or 11, can use either) and I'm running into an issue. The XML file references an XSD file which doesn't exist (I've contacted the vendor and they are 'working on it').

If I don't use any named tags in the xpath selection commands and just hard-code things by position, e.g.

<xsl:for-each select="/*/*[position()=2]/*[position()=1]/*[position()=5]/*">

then it works. However, I think this is brittle as any change in the order of tags in the XML file will break it.

I'd much rather use named tags for selection, e.g.

<xsl:for-each select="soap:Envelope/soap:Body/abcxyz:Person/abcxyz:Address">

(the abcxyz is the vendor namespace with the missing XSD file).

However, when using these named tags, the import is failing. I'm pretty sure the missing XSD file is the problem.

Question: is there any way to tell FileMaker to ignore the missing XSD file and import anyway? Or does the xpath used in the "for-each select" require a valid XSD?)

I'm sure once the vendor gives me the XSD file it'll all work, but as usual this is already behind scheudle.

thx

[comment]

Hard to say anything without seeing the source XML document.

[xochi]

Hard to say anything without seeing the source XML document.

It's proprietary & sensitive so I can't provide the actual data, but here's a heavily sanitized version:

<?xml version="1.0" encoding="utf-8"?>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

  <soap:Header>

    <xxxx:HeaderInfo xmlns:cuns="http://REDACTED/REDACTED">

	  <xxxx:InterfaceTypeId>REDACTED</xxxx:InterfaceTypeId>

	  <xxxx:BusinessPartnerId>REDACTED</xxxx:BusinessPartnerId>

	  <xxxx:SchemaVersion>1.0</xxxx:SchemaVersion>

	  <xxxx:DateTime>2011-09-06T12:00:00</xxxx:DateTime>

    </xxxx:HeaderInfo>

  </soap:Header>

  <soap:Body>

    <abcxyz:DeductionRegister xmlns:abcxyz="http://REDACTED/REDACTED" xsi:schemaLocation="http://REDACTED/REDACTED REDACTED.xsd" xmlns:xxxx="http://REDACTED/REDACTED" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

	  <abcxyz:BusinessPartnerREDACTED>REDACTED</abcxyz:BusinessPartnerREDACTED>

	  <abcxyz:OrganizationName>REDACTED</abcxyz:OrganizationName>

	  <abcxyz:RollMonth>2011-09</abcxyz:RollMonth>

	  <abcxyz:WarrantIssuedDate>2011-10-01</abcxyz:WarrantIssuedDate>

	  <abcxyz:ParticipantInfo>

	    <abcxyz:ParticipantID>

		  <xxxx:REDACTEDId>REDACTED</xxxx:REDACTEDId>

	    </abcxyz:ParticipantID>

	    <abcxyz:ParticipantIDType>PID</abcxyz:ParticipantIDType>

	    <abcxyz:FirstName>REDACTED</abcxyz:FirstName>

	    <abcxyz:MiddleName>REDACTED</abcxyz:MiddleName>

	    <abcxyz:LastName>REDACTED</abcxyz:LastName>

	    <abcxyz:ParticipantDeductionInfo>

		  <abcxyz:PayeeAccountId>REDACTED</abcxyz:PayeeAccountId>

		  <abcxyz:ProgramTypeCode>REDACTED</abcxyz:ProgramTypeCode>

		  <abcxyz:DeductionDetail>

		    <abcxyz:DeductionTypeCode>REDACTED</abcxyz:DeductionTypeCode>

		    <abcxyz:ExceedGrossIndicator>REDACTED</abcxyz:ExceedGrossIndicator>

		    <abcxyz:DeductionFrequency>REG</abcxyz:DeductionFrequency>

		    <abcxyz:PayeeAmountInfo>

			  <abcxyz:AmountType>REDACTED</abcxyz:AmountType>

			  <abcxyz:Amount>REDACTED</abcxyz:Amount>

		    </abcxyz:PayeeAmountInfo>

		    <abcxyz:AgreementId>REDACTED</abcxyz:AgreementId>

		    <abcxyz:SponsorName>REDACTED</abcxyz:SponsorName>

		    <abcxyz:SponsorREDACTEDId>REDACTED</abcxyz:SponsorREDACTEDId>

		  </abcxyz:DeductionDetail>

	    </abcxyz:ParticipantDeductionInfo>

	  </abcxyz:ParticipantInfo>

    </abcxyz:DeductionRegister>

  </soap:Body>

</soap:Envelope>

[xochi]

Well, I found one solution:

Just use a text editor to search & replace all of the namespaces (the "abcxz:" and "xxxx:" in the above XML sample) as well as the xmlns: tags for the missing file. Not elegant, but it does work.

[comment]

<abcxyz:DeductionRegister xmlns:abcxyz="http://REDACTED/REDACTED" xsi:schemaLocation="http://REDACTED/REDACTED REDACTED.xsd"

I am afraid this is pretty much useless. If that's a real URL, I believe your vendor could simply place a dummy .xsd file there. Or, if you are comfortable with editing the document before importing it, you could just remove the reference to a schema.

[xochi]

The URLs that I redacted are real URLs, with files that don't exist (which is what started this problem). Vendor is $worlds_largest_something and thus is not terribly agile, and my schedule was such that I needed to work on this today.

To clarify my question : since it does seem that removing all traces of the namespace works, I'm curious if your idea of putting a dummy xsd file in place would work, too? I'm not clear on whether the filemaker XSLT processor actually parses the XSDs or what...anyone know?

[comment]

AFAIK, if the XML document references a schema, then Filemaker's XML parser will attempt to validate the document against the schema. However, this doesn't agree with what you said earlier about using a XSLT stylesheet with no element names. Perhaps the behavior is different if the schema reference is confined to a specific element (though I cannot see a good reason why this should be so). What is the exact error message you get?

[xochi]

Ok, the plot thickens a bit: turns out $worlds_largest_something actually did have the XSDs, I just downloaded the wrong but similarly-named zip file.

So now I've got the missing .xsd files on my hard drive (but they are still not hosted at $vendor's website).

The XML that vendor is providing has that covered, though:

http://www.w3.org/TR..._schemaLocation the purpose of the xsi:schemaLocation tag is to "to provide hints as to the physical location of schema documents". However, this seems to choke filemaker, and any attempt to select into the abcxyz: nodes fails. A simple workaround is to just change the xmlns so they are all local:

<abcxyz:DeductionRegister

xmlns:abcxyz="http://redacted/redacted/abcxyz"  

xsi:schemaLocation="http://redacted/redacted/abcxyz abcxyz.xsd"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

>

 

As per 

<abcxyz:DeductionRegister

xmlns:abcxyz="abcxyz.xsd"  

>

Then, FileMaker is happily able to find the local xsd files on the disk.

Looks to me like a bug in FM's handling of the xsi:schemaLocation attribute? Same issue seen in FM9 and FM11...

I've asked the vendor about this. The goal here is a fully automated process, so having to hand-edit the XML files before feeding them into filemaker is clearly not feasible in the long-term.

[comment]

However, this seems to choke filemaker, and any attempt to select into the abcxyz: nodes fails.

You put me to great disadvantage here - not only do I lack access to the actual files, you also do not report exactly what your attempt is and how it "fails".

Still, I wouldn't subscribe to your theory. I am afraid you are mixing between a namespace and a schema location. A namespace can be (and most often is) a non-existing URL. For example:

<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">

 

There is no document at this location, and that's perfectly fine - the only requirement for the URL is that it be unique among the namespaces involved in the current process.

 

A schema location, OTOH, should point to an actual .xsd document.

 

Now, when you change:

 

<abcxyz:DeductionRegister

xmlns:abcxyz="http://redacted/redacted/abcxyz"

xsi:schemaLocation="http://redacted/redacted/abcxyz abcxyz.xsd"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

>

 

to:

 

<abcxyz:DeductionRegister

xmlns:abcxyz="abcxyz.xsd"  

>

 

the effective change here is the removal of schema location - not "change the xmlns so they are all local". I am quite sure you will get the same result with:

 

<abcxyz:DeductionRegister

xmlns:abcxyz="http://redacted/redacted/abcxyz"

>

[xochi]

You put me to great disadvantage here - not only do I lack access to the actual files,

Yes, they are proprietary/sensitive. I hope you can understand that, and I appreciate your help nonetheless

you also do not report exactly what your attempt is and how it "fails".

Specifically, this line fails to select any nodes:

 

<xsl:for-each select="soap:Envelope/soap:Body/abcxyz:DeductionRegister/abcxyz:ParticipantInfo[position()=1]">

 

(The error message from FM is the "XML/XSL information is not enough to proceed with import  " error you get when no metadata is returned).  If I select only into the 2 soap nodes (soap:/Envelope/soap:Body) then I get an import (with the wrong data, of course, since its not at the right level).

 

Still, I wouldn't subscribe to your theory. I am afraid you are mixing between a namespace and a schema location. A namespace can be (and most often is) a non-existing URL. For example:

<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">

 

There is no document at this location, and that's perfectly fine - the only requirement for the URL is that it be unique among the namespaces involved in the current process.

Thank you, I believed that was the case, too, but it's good to hear that confirmed

A schema location, OTOH, should point to an actual .xsd document. Now, when you change:

<abcxyz:DeductionRegister

xmlns:abcxyz="http://redacted/redacted/abcxyz"

xsi:schemaLocation="http://redacted/redacted/abcxyz abcxyz.xsd"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

>

 

to:

 

<abcxyz:DeductionRegister

xmlns:abcxyz="abcxyz.xsd"  

>

 

the effective change here is the removal of schema location - not "change the xmlns so they are all local".

I agree with you, in the sense of "what the w3 specifications say" however what I'm reporting seems to be the actual way that FileMaker behaves. For example, it seems that in FM9 the XSD files must be in the same folder as the XML file being processed. In FM11, it seems that you can put the XSD file in the same folder as the XSLT being processed. So, clearly something has changed between 9 and 11 w.r.t. how it locates XSD files. However, in neither case does FM seem to "do the right thing" when there is a valid schemalocation.

I am quite sure you will get the same result with:

<abcxyz:DeductionRegister

xmlns:abcxyz="http://redacted/redacted/abcxyz"

>

Let me try that now.

Well, what do you know, it works! Let me digest this all and write a follow-up post.

[xochi]

Summary

After thinking the I understood things, now in further testing, I can't reproduce the issue I was seeing before.

It appears as if there is a big difference between FM9 and FM11 in handling of XSD files: I can easily do the XML import in FM11 even if the XSD files don't exist. In FM9, it fails unless the XSD file is in the same folder as the XML file being imported. Putting the XSD in the same folder as the XLST file (which would seem natural to me) doesn't work in FM9.

My earlier claim that there's a bug in FM's handling of xsi:schemaLocation ? I'm not seeing that now.

Since the import is now working, I'm not sure what's changed. (I did check, and the missing XSD files are not on the vendor's website this morning, either). Possibly I was just confused when switching back & forth between FM9 and FM11?

Thanks for your help -- if I can clarify just what was going wrong here, I'll definitely report back.

[comment]

It appears as if there is a big difference between FM9 and FM11 in handling of XSD files: I can easily do the XML import in FM11 even if the XSD files don't exist. In FM9, it fails unless the XSD file is in the same folder as the XML file being imported.

Yes, there seems to be difference between v.11 and all previous versions. In previous versions, if the XSD file is not where the XML file says it should be (not necessarily in the same folder as the XML file), you will get an error. However, the error is NOT what you have stated:

The error message from FM is the "XML/XSL information is not enough to proceed with import " error you get when no metadata is returned).

Instead, you should see:

An exception occured! Type:XMLPlaformException, Message:Could not open file: http://redacted/redacted/abcxyz abcxyz.xsd

and nothing will be imported.

If you see another type of error, then the reason is NOT a missing XSD document. Most likely, you are trying to reference an element that is defined in its own namespace, without having defined the same namespace in your stylesheet. That would also explain why it worked for you when you used a relative reference of the type:

<xsl:for-each select="/*/*[position()=2]/*[position()=1]/*[position()=5]/*">