DDR: Won't show which calcs contain a field?

[ĸupiэȶƺ]

Am I right in thinking that FMD 7's Database Design Report will not tell you what calculations reference a given field? IE, if I look up a field, I want to see a list of what calculations depend on it.

I really hope FileMaker doesn't expect me to lay out money for a third-party solution for this elemantary reporting. If so, it's the biggest blemish on this otherwise excellent upgrade.

Any suggestions? Am I missing something?

Thanks,

Mike

[Wim Decorte]

If I remember correctly the HTML report does not show you per field in what calcs they are used. But that info is in the DDR, FMI just didn't include it in the HTML report they produce out of the DDR.

But no, you don't have to spend money to get the info. Since it is in the DDR, you can can extract it in any number of ways (with an XSLT stylesheet or anything that can process XML).

[ĸupiэȶƺ]

But that info is in the DDR, FMI just didn't include it in the HTML report they produce out of the DDR.

OK, that's real interesting. I didn't realize the HTML DDR was just extracted from the XML version. Makes perfect sense though.

This has already been discussed in another thread, but I'm a brand-newbie to XML/XSLT, so before I reinvent the wheel, has anyone out there got an already-written XSLT stylesheet to mine more complete data out of the DDR?

Thanks a bunch for the response.

Mike

[ĸupiэȶƺ]

that info is in the DDR, FMI just didn't include it in the HTML report they produce out of the DDR.

I've just taken a good close look at the XML DDR, and it reaaaaaally looks like that information just isn't there. For instance, here's an entry for a calculation field in the DDR I just generated:

<Field id="95" name="BarCodePrint" dataType="Text" fieldType="Calculated">

				<Calculation>

					<![CDATA["*" & GetAsNumber(BarCode) & "*"]]>

				</Calculation>

				<DisplayCalculation>

					<Chunk type="NoRef">"*" &amp; </Chunk>

					<Chunk type="FunctionRef">GetAsNumber</Chunk>

					<Chunk type="NoRef">(</Chunk>

					<Chunk type="FieldRef">

						<Field id="29" name="BarCode" table="STUF Shot List"/>

					</Chunk>

					<Chunk type="NoRef">) &amp; "*"</Chunk>

				</DisplayCalculation>

				<Comment></Comment>

				<AutoEnter alwaysEvaluate="False"/>

				<Storage index="None" global="False" autoIndex="True" indexLanguage="English"

				maxRepetition="1" storeCalculationResults="True"/>

			</Field>

 and here is the entry for the field "BarCode", which is referenced in the above calculation: 

<Field id="29" name="BarCode" dataType="Number" fieldType="Normal">

				<Comment></Comment>

				<AutoEnter allowEditing="True">

					<Serial generate="OnCreation" increment="1" 

					nextValue="1552461"/>

				</AutoEnter>

				<Validation type="OnlyDuringDataEntry" 

				alwaysValidateCalculation="False">

					<NotEmpty value="False"/>

					<Unique value="True"/>

					<Existing value="False"/>

					<StrictValidation value="True"/>

					<ErrorMessage>Cannot have duplicate Bar Code 

					number!</ErrorMessage>

				</Validation>

				<Storage index="All" global="False" 

				maxRepetition="1"/>

			</Field>

...which doesn't mention that BarCode is used in a downstream calculation.

This was in the detail XML file for one of my database files. Am I looking in the wrong place?

Thanks,

Mike

[Edited: hard wraps inserted in some tags in code segments, because stupid UBB "code" tags were page width of some browsers past the edge of the window. ]

[QuinTech]

But you could deduce this information from other XML elements. You could go all-out and create an XSLT that will give you this result by clicking on the field name; i imagine that this is what some, if not all, of the third-party apps do. If you are going to be referring to this DDR frequently, maybe it would be a good idea to write such an XSLT; if you just need it sporadically, though, would a text search for the field name work out?

Incidentally, you could do the same thing by printing field definitions to a text file or PDF, then searching for the field name in the resultant file.

J

[ĸupiэȶƺ]

But you could deduce this information from other XML elements. You could go all-out and create an XSLT that will give you this result by clicking on the field name;

[...snip...]

would a text search for the field name work out?

Incidentally, you could do the same thing by printing field definitions to a text file or PDF, then searching for the field name in the resultant file.

I've got a 600-field table, in which I need to look for downstream issues with 50 or so fields, I really don't want to have to search manually... hence the need for a design report to tell me. -(

Unfortunately - don't mean to sound cranky, but here goes - "You could deduce the info yourself from the XML" or "You could write an XSLT style sheet to do it" or "You could do a search in a text editor" are just not the same thing as "The DDR does contain this information." :-( To sit and learn XSL (this week, at least) or search manually for what I need is so much less cost-effective than just plunking down the dough for a third party utility that it would sort of defeat the whole point of my asking ;-) With the amount of information needed, and my having no time right now to learn a new programming language, there's no "you could"s about it. I can't.

Sorry to sound crabby - it's late, and this just seems like such an egregious omission on FMI's part. It's such an elementary piece of data to want, and IIRC the previous DDR versions used to provide it, but getting it now requires significant after-market technical expertise or effort - be it from myself or a commercial developer - which I call: requiring third party software. Sometimes I don't understand what those people are thinking. Who thought it was an improvement to completely abandon the highly usable, thoroughly informative, easily customizable database-formatted DDR, in favor of only offering ungainly huge text files that require learning a whole new programming language to get basic information out of? This is clearly the best upgrade they've ever released, but still it seems like they just couldn't avoid throwing in a step or two backwards *somewhere* in there, just to remind you they're still FMI.

Awright, I'd better get off the soapbox now.... thanks again... despite the possibly prickly reply, the suggestions were appreciated.

Mike

[ĸupiэȶƺ]

But you could deduce this information from other XML elements. You could go all-out and create an XSLT that will give you this result by clicking on the field name;

[...snip...]

would a text search for the field name work out?

Incidentally, you could do the same thing by printing field definitions to a text file or PDF, then searching for the field name in the resultant file.

I've got a 600-field table, in which I need to look for downstream issues with 50 or so fields, I really don't want to have to search manually... hence the need for a design report to tell me. -(

Unfortunately - don't mean to sound cranky, but here goes - "You could deduce the info yourself from the XML" or "You could write an XSLT style sheet to do it" or "You could do a search in a text editor" are just not the same thing as "The DDR does contain this information." :-( To sit and learn XSL (this week, at least) or search manually for what I need is so much less cost-effective than just plunking down the dough for a third party utility that it would sort of defeat the whole point of my asking ;-) With the amount of information needed, and my having no time right now to learn a new programming language, there's no "you could"s about it. I can't.

Sorry to sound crabby - it's late, and this just seems like such an egregious omission on FMI's part. It's such an elementary piece of data to want, and IIRC the previous DDR versions used to provide it, but getting it now requires significant technical expertise or effort from a third party - be it myself or a commercial developer - which I call: requiring third party software. Sometimes I don't understand what those people are thinking. Who thought it was an improvement to completely abandon the highly usable, thoroughly informative, easily customizable database-formatted DDR, in favor of only offering ungainly huge text files that require learning a whole new programming language to get basic information out of? This is clearly the best upgrade they've ever released, but still it seems like they just couldn't avoid throwing in a step or two backwards *somewhere* in there, just to remind you they're still FMI.

Awright, I'd better get off the soapbox now.... thanks again... despite the possibly prickly reply, the suggestions were appreciated.

Mike

[Fenton]

Well, I certainly agree that the DDR is not as useful as it could be. It's kind of like the difference between exporting the data out your complex solution as an XML file, then saying it has the same functionality as the FileMaker solution. They know that, I'm sure. They just don't have the time (or want to spend the time) to make it as functional as 3rd party applications; which, while we'd prefer they did, is an engineering and business decision.

The data is there; it's just not where you want it.

In the entry for:

<Field id="95" name="BarCodePrint" dataType="Text" fieldType="Calculated">

you have:

<Chunk type="FieldRef">

<Field id="29" name="BarCode" table="STUF Shot List"/>

</Chunk>

It tells which earlier fields are referenced in the data of "downstream" calculations. It doesn't show the downstream calculations in the original base field. I wouldn't expect it to. That would be quite difficult, since that data is not IN the original field's definition. The 3rd party apps can show this; but it's because they run post-processing to gather this info. And it's one of the reasons they are slow.

You could do a text search for "BarCode" I suppose. In this case that would likely work. It wouldn't be much fun though.

[QuinTech]

The data is there; it's just not where you want it.

Exactly...

Kupietz, i hear you about the user-unfriendliness of the DDR report. There is a lot of work that you need to do to get the data you want from it. On the other hand, as Fenton says, the data is there (in the 3rd line of the code you posted).

Since i rarely use DDRs, maybe i'm talking out of turn here, but in general i really like the XML format because of its flexibility. I think i'll take the unusual step of respectfully disagreeging with Fenton -- i would prefer they not make the DDR a functional program. It's nothing but metadata, and as such, i think XML is a fine format for it. The fact that this type of code is even offered is a very good step. It comes close to allowing a developer to analyze his database in the same way he might look at C++ code or other compiled languages.

But if they offered the DDR as a database with the functions you described, well... i also can't imagine why they wouldn't offer an XML report in addition to that. Step backwards, indeed.

[ĸupiэȶƺ]

It doesn't show the downstream calculations in the original base field. I wouldn't expect it to. That would be quite difficult, since that data is not IN the original field's definition.

But it shows what layouts, relationships, scripts, and value lists a field is in, even though those aren't in the field's definition. What's the diff?

BTW, your point is definitely taken about it being an engineering & business decision. I've been reminded once or twice in the past by FM phone support that certain limitations had to be accepted because it is (was) a $200 program. I myself have always said I'd be happy to pay an extra $500 for a "FileMaker Pro Gold Edition" that took care of some of the more obvious feature omissions and limitations... One can dream. Nonetheless I really am chagrined that I have no real alternative to spending the next few days chasing down the complicated downstream calcs for 50 fields by hand.

Agh! You know, I just thought of something. My iBook's been down for a week with an ailing HD, but once I'm back on that instead of this infernal Windows machine, it strikes me that BBEdit's "Batch Find" might make text-searching the XML file feasable, since it'll produce a single list of all occurences of a text string within an entire folder of documents. Not the solution I wanted, certainly not ideal, but probably my best option for the time being.

[ĸupiэȶƺ]

But if they offered the DDR as a database with the functions you described, well... i also can't imagine why they wouldn't offer an XML report in addition to that.

I guess that's what it boils down to. On consideration, I see what you guys appreciate about having the XML report. It just should have been an additional option, instead of out-and-out replacing something that was both simpler to use and more informative.

Well, I'm sure at some point someone will make an XSLT stylesheet available that can list the downstream calculations for a field... I can't be the only person missing this functionality. I'll do it myself if I haven't seen one by the time I finally have some free time to sit & learn how to.

Speaking of which, can anyone recommend if there's a particularly good book or resource to get me up & running with XML & XSLT?

Thanks,

Mike

[QuinTech]

"FileMaker Pro 6 Developer's Guide to XML/XSL" by Beverly Voth. She is widely recognized as one of the tops in terms of FM & XML.

You can also get a good grounding from the sample files on the FMDeveloper CD, although (i must admit) i needed a lot of guidance because those aren't the easiest to follow. There's quite a bit of good and varied material on this site as well.

HTH,

J

[ĸupiэȶƺ]

[post deleted - see following post instead]

[ĸupiэȶƺ]

The data is there; it's just not where you want it.

I had a funny thought over the weekend... this is sort of like ordering a steak tartare in a restaurant, and having the waiter bring you a cow and a chainsaw. :-)

A few last thoughts, having played with this a few days...

1.) I spent a few hours in BBedit GREPping around with both the HTML and the XML to pull out the data I need... it can be done, but it's so much work that buying a third-party utility is much more cost-effective.

2.) I noticed that the XML report does not seem to actually contain any more detailed info on, say, which scripts or layouts use a particular field, than on which other fields reference it, but the HTML report does show that info. So some sort of post-processing is already being done by the XSLT, to coalesce that layout & script reference info on a per-field basis. So it's not immediately apparent why giving field dependencies per field, in addition, would represent some greater technical hurdle.

Interesting, all in all. The omissions certainly seem likely to have been a business decision, as someone speculated above... certainly not something based on fulfilling end-user needs, that's for sure. Think I'm going to send FMI some feedback about it.

Mike

[QuinTech]

this is sort of like ordering a steak tartare in a restaurant, and having the waiter bring you a cow and a chainsaw. :-)

LOL... i'll steal that one from you if you don't mind!

J

[ĸupiэȶƺ]

Well, I went and did it. I learned XSL, wrote the transforms I need to turn the DDR into what I call a "Dependency Report". Overall, I'm quite pleased with how it's working, it shows all the important cross-referenced dependency data I need, but now I find myself faced with another substantial roadblock.

The XML DDR files I need to transform are up to 25MB in size. Can anyone recommend a powerful, non-java OS X program to help me work on this system?

I've tried the popular java XML programs (Oxygen, Editix) and they are much too slow to use... when trying to reference the huge DDR output files, I gave up and force-quit after 13 hours (!!) of the "beach ball" wait cursor.

I've also been using the public domain "TestXSLT", which sometimes works but generally behaves like a public domain software: too many unexpected quits, features mysteriously refusing to function, etc., for me to be productive with it.

I know none of this is my XSLT, because when TestXSLT is in the mood to work properly, the transformation works fine, and takes under 90 seconds.

Any other Mac users have better suggestions? What program are you using to write/test your XSLT on?