Request for HUGE DDRs

[Dave Ramsey]

I'm looking for some DDRs that I can use to torture-test FMPerception.

 

--The biggest multi-file DDR that I have for testing is 2.12 gigabytes.

--The largest complete single-file DDR is 450 megabytes. (I have an incomplete one (missing scripts) that's 574.7 MB)

If you've got something larger, and you can allow me to use a copy for testing purposes, I would love to hear from you.  As larger DDRs are provided, I'll update this thread with the new numbers to beat. 🙂

 

Let the games begin!

 

[Dave Ramsey]

Biggest single file is now 303.8 MB.  Thanks!

[Wim Decorte]

Is most of that in the Base64 representations of graphics?  Or real elements?

[Fitch]

It is all too real. Just ran a new one and that one xml file is up to 310 MB.

[Dave Ramsey]

On August 4, 2016 at 8:27 AM, Wim Decorte said:

Is most of that in the Base64 representations of graphics?  Or real elements?

Wim,

As far as I can tell, the DDR doesn't contain many/any Base64 graphics representations anymore.  They appear to have disappeared around about FileMaker 12.  It's in the clipboard XML, but not in the DDR XML.  I wish it was still there.  I had some cool ideas for what to do with it.

I was curious about the answer to your question, though, so I wrote a little tool to analyze DDRs to determine where the size was coming from.  I analyzed half a dozen big files to determine what percentage of the total character count of the DDR was tied up in each major element, and found the following:

• In general, I'm seeing that DDRs are about 70-80% layouts, 10-20% scripts, and generally less than 5% "everything else together".  
• Within the Layouts XML, 60-70% of that is styling information (LocalCSS, character and paragraph styles, and number/date/time formatting).  That last one is possibly the worst, as it's verbose and present on basically everything (the DDR exports number/date/time formatting for web viewer objects…).  It accounts for effectively about a third of the size of the layouts.  Optimizing styles and such has no impact on it.
• If you could remove ALL local CSS, it's likely going to save you about 10-30% of your DDR size.

I wrote the code, but I can't come up with a really good argument to roll it into FMPerception.  It's interesting, but I haven't been able to come up with any particular use for the information (aside from triggering an OCD reaction from some developers).  If somebody has a good argument for why it might be legitimately useful, I'll consider adding it to FMPerception.  But the numbers that I have suggest that it isn't a reliable measure of the complexity of a system, or even of its performance.  I am willing to be swayed, though.

Wim, thanks for the awesome question.

[Lee Smith]

Hi Dave,

3 hours ago, Dave Ramsey said:

Within the Layouts XML, 60-70% of that is styling information (LocalCSS, character and paragraph styles, and number/date/time formatting).  That last one is possibly the worst, as it's verbose and present on basically everything (the DDR exports number/date/time formatting for web viewer objects…).  It accounts for effectively about a third of the size of the layouts.  Optimizing styles and such has no impact on it.

Would something like the file RCC_FM_Layout_optimize.fmp12 located here http://www.rcconsulting.com/downloads.html be of any help?

Lee

 

[Wim Decorte]

5 hours ago, Dave Ramsey said:

I wrote the code, but I can't come up with a really good argument to roll it into FMPerception.

It'd be interesting to know the number of layouts with local CSS and an indication of how many elements are handled within that CSS (is just the font or a whole slow of overrides).

There is some good value in this:

- low hanging fruit to optimize speed especially for WAN access

- one more indication of how sophisticated / clean the solution is designed, it'd be a good indicator of those upfront assessments.

[Dave Ramsey]

3 hours ago, Lee Smith said:

Would something like the file RCC_FM_Layout_optimize.fmp12 located here http://www.rcconsulting.com/downloads.html be of any help?

Lee,

That would definitely help you to reduce the local CSS of a specific problem layout.  My point was just that I'm not sure that looking at that at a file-level is optimal.  Is it worthwhile to spend the time to optimize every local CSS node out of every layout?  My sense is that it's not, but I don't have any math to back that up.  It feels like "premature optimization".  

2 hours ago, Wim Decorte said:

It'd be interesting to know the number of layouts with local CSS and an indication of how many elements are handled within that CSS (is just the font or a whole slow of overrides).

Definitely.  Node counts in local CSS I think are really useful, and I'd like to work on some code to allow me to display those higher in the hierarchy (at the layout level, rather than just at the layout object level like now).  That said, I don't want to kill the performance of a tool optimized for performance.  It's a tricky balance to maintain.

I'm less convinced of the value of the character count of the overridden nodes.  There's a line between optimizing the layout and optimizing the DDR itself... and that's probably over that line.

I also like the idea of looking into which nodes are most often overridden.  That could be helpful.

 

[Fitch]

Our interface file has nearly 500 layouts, many of them very complex. The system has been developed and in use since FileMaker 9, so of course nearly all layouts are Classic theme, which means basically every object has local theme overrides.