Piggy backing on a CF input parameter

[dual_mon]

First of all, hello. This is my first post on FMForums.

In designing a solution to a specific problem I came up with an interesting technique that I thought was worth sharing, but I want to see if anyone can think of a simpler way to do the same thing.

I was looking for a way to store and pass variable data in an array from within the logic of the CF. I wanted a tool similar to a script parameter or a global field. I toyed with creating "non-user" parameters that I could use to hold variables, but this seemed way too flakey. Basically as the subject line hints, I settled on piggy-backing additional variables onto one of the "user" input parameters in a CF.

In the example I show below, the piggy-backed parameter is a counter. The function loops through a list and either omits the first value or moves it to the end of the list. In order to know when the cycle is complete, the function needs to count down from the original number of values as it makes each pass.

In this example the CF takes a list of values and returns only those values that match a supplied prefix.

Example input values as returned from LayoutNames ( filename ):

• layout1
  layout2
  layout3
  printReport
  printInvoice
  printEstimate
  layout4
  layout5

Returned values from the CF when the prefix is "print":

• printReport
  printInvoice
  PrintEstimate

Here is the CF. Note that I declare a variable named Prefix1 which trims the input parameter, Prefix, to the left most value with no trailing pilcrow. From there I am free to parse any other values in Prefix as I please, leaving the original input parameter intact. Of course this assumes that the input parameter is only intended to be one line. Side note: because the function strips all the pilcrows from Prefix, the CF is immune to accidental trailing carriage returns in the input parameter (a common user mistake, in my experience).

The second declared variable is Iteration, which is simply parsed from the second value of Prefix unless that value is null, in which case it is set with ValueCount ( List ).

The third declared variable, Counter, is the decrement of Iteration. This is concatenated with Prefix1 at the conclusion of each pass until it reaches <= 1 which is the escape condition.

The second group of variables are the list handlers, and the nested If statements in the body of the function either make the recursive call, or return the result list with trailing pilcrow removed.

MatchPrefixAll ( List ; Prefix )

Let ( [

Prefix1 = Substitute ( LeftValues ( Prefix ; 1 ) ; "

[Ender]

Welcome Jeremiah,

I'm having trouble seeing the usefulness of this function from your example. I can think of easier ways to filter a list based on a prefix, but I don't think that was your point.

Do you have another example of this idea?

It looks like the main thing you're accomplishing here is eliminating the sub function that is usually needed with iterative functions. I suppose this makes it easier to copy and paste, but is there another advantage?

[comment]

I understand the problem, but not the solution.

First, let's do as Ender suggests and simplify the example: let's say we are given a number n, and our task is to produce a sequence of 10 numbers: n, n+1, n+2, n+3 ... n+9.

The number 10 is a fixed constant, therefore the user is not expected to fill in this parameter. Thus our function needs to be in the form of Function ( n ).

Our problem is that this is not possible without a sub-function. The subfunction can take the form of subFunction ( n ; m ), where n is fixed and m is n increased by 1 until m - n = 9. Or, m could be 10, and while n is increased, m is decreased until m = 1.

Now if you can suggest a way to solve this problem without a subfunction, I'd be VERY interested to hear it.

[dual_mon]

Ender,

Despite the risk of veering off-topic, I am interested in your simpler method of filtering a list based on a prefix. Perhaps in a new thread?

comment,

Sure. No problem. In fact, let's go one better and include an increment input variable so the list can be incremented 10 times by any number.

Iterate10 ( Start ; Increment )

Let ( [

List = Substitute ( LeftValues ( Start ; 1 ) ; "

[dual_mon]

PS: Sorry about all the white space in the Code. I tried to edit, but it looks normal in the input field.

[dual_mon]

Or to boil it right down to your given criteria,

Function ( n )

Let ( [

List = Substitute ( LeftValues ( n ; 1 ) ; "

[comment]

OK, now you're rolling. I think this is a great idea. Thanks.

I have taken the liberty to rewrite it in a simpler form that is easier for me to understand:

// Custom Function PiggyBack ( n )

Let ( [

//REAL PARAMETER AS PASSED BY THE USER

realParameter = Substitute ( LeftValues ( n ; 1 ) ;

[Ender]

I'm still not sure what the advantage is here. To me, it looks like it creates added complexity for the function design, and adds a limitation where line returns could not be used in the original parameter. I suppose alternative separators could be used, but I still think the use of a sub function is easier to write and debug.

Jeremiah, here is my version of MatchPrefixAll():

MatchPrefixAll ( List; Prefix ) =

Case( not IsEmpty(list) and not IsEmpty( Prefix );

  Case( Left( List; Length( Prefix )) = Prefix; LeftValues( List; 1 )) & 

  MatchPrefixAll( MiddleValues( List; 2; ValueCount( List ) - 1); Prefix )

)

[dual_mon]

Ender,

That MatchPrefixAll is sweet. Thanks. I was trying to get it to return the last value without the trailing pilcrow, but didn't come up with it yet. Any ideas?

I take your point that the single value in the original parameter is a drag, but it is not a limitation of the core concept. Also, the idea is not to eliminate subFunctions, but rather to eliminate "holder" parameters where they would be exposed at the "API" so to speak.

Let's consider an example that illustrates both points.

Check out this two-part custom function from Jed Verity:

http://www.briandunning.com/filemaker-custom-functions/detail.php?fn_id=230

http://www.briandunning.com/filemaker-custom-functions/detail.php?fn_id=231

The exposed (or input) function is GetPermutations ( values; counter; output )

The backend is GetPermutationsSub ( values ; counter ; output ; subcounter )

This function is totally sweet. Jed must hover about six inches off the floor if he can think up a tree like this, but the fact that the exposed (or input) function requires mysterious "non-user" parameters bugs me. If I am programming and I want to use this function, I don't want to have to go back and figure out what the hell all of those are for. All I care about is that I want to feed it a list and get my result.

So my goal is to pare this down to a single input parameter (which is a list) without making the function so complex that only Ray Cologon can understand it

Check it out:

GetPermutations ( list )

Let ( [

flag = If ( LeftWords ( LeftValues ( list ; 1 ) ; 1 ) = "SomethingNoOneWillEverUseAsAValue" ; 1 ; 0 ) ;

realList = If ( flag ; Substitute ( list ; LeftValues ( list ; 1 ) ; "" ) ; list ) ;

counter = If ( flag ; GetAsNumber ( RightWords ( LeftValues ( list ; 1 ) ; 1 ) ) ; 1 )

];

If ( counter = ValueCount ( realList ); RealList & "--

[Ender]

Hi Jeremiah,

Thank you for the example and the discussion.

I think in most cases, people's custom functions are (and should be) built so that you don't need to send those extra utility values from the initial function call. The GetPermutations function is kind of an oddball that recursively calls itself and its sub function.

I would still find it difficult to design and debug a function using your technique. Instead, if I wanted to simplify the initial function call to have only those parameters that are relevant, I would add another function that has those other initial values in it. For the GetPermutations example, I would add this simple function:

GetPermutationsExposed ( list ) =

//GetPermutationsExposed ( list )



//This is the function that should be used in calcs to initiate GetPermutations()

GetPermutations ( list; 1; "" )

My poor name choice aside, this would be an easy way to initiate the GetPermutations function without needing to know the initial 'counter' and 'output' parameters. I would imagine Jed may have thought of doing something like this, but omitted it because of the format of Brian Dunning's CF listing (this one-line function is not too impressive by itself.)

In fact, I used this initial function idea in my SmartRanges() functions too. They're not really needed, but they simplify what you need to know for the function call:

http://www.fmforums.com/threads/showflat.php?Cat=0&Number=124596

My bias aside, its clear you have developed a good technique for stacking parameters. It's kind of like stacking script parameters, eh? I will definately keep it in mind.

[dual_mon]

Ender,

I think I must concede the point. Using a "masking" function does resolve my peeve about having exposed utility parameters, and surely it is simpler to debug a function when you are not worrying about whether or not it is the parsing of the piggy-back parameter that is making it barf.

Your analogy with stacking script parameters is apt, as that is what got me thinking along these lines in the first place. The big difference, of course, is that in scripts you only have one parameter to work with, and therefore stacking or piggy-backing becomes much more interesting there. Further, from the context of the script, you have a much more direct means for parsing out the parameters.

I still think there might be a place for it in functions, I will let you know when I find it

[comment]

Maybe it's my short attention span, but I really hate working with sub-functions. I find de-bugging much more harder when the process is divided into two, especially when I cannot see the two at the same time. Not to mention how neat it is to have everything in the same function, and how easy it is to duplicate it for re-use.

I have just made an experiment and converted one of my text multiplication formulae from repeating field to CF. Piece of cake, and it will be much easier next time, now that I have the mechanism in place.

I have always complained about the lack of a real counter in the CF, so I for one am very grateful to you for coming up with this.

[dual_mon]

Ah, so maybe I conceded defeat too soon! Thanks Comment.

FWIW, I am still enjoying the technique for the same reasons you mention. Check out this routine I just dusted off and made into a CF for converting a base 10 number to a base 16 number. Note the additional use of Filter in the parsing of the piggy-backed values. This has the dual advantage in this context of stripping pilcrows, and ensuring that no illegal values are passed to the function:

DecToHex ( Decimal )

Let ( [

Dec = Filter ( MiddleValues ( Decimal ; 1 ; 1 ) ; "0123456789" ) ;

Rem = Mod ( Dec ; 16 ) ;

NewHex = Middle ( "0123456789ABCDEF" ; Rem + 1 ; 1 )

[comment]

This time I am not going to agree. There is no need to piggy-back a static result. You can simply put it aside:

Let ( [

bit = Mod ( decimal ; 16 ) ;

alpha = "0123456789ABCDEF" ;

char = Middle ( alpha ; bit + 1 ; 1 ) ;

next = Div ( decimal ; 16 )

] ;

Case (

next ;

DecToHex ( next )

)

& char

)

[dual_mon]

Doh! (nice)

[dual_mon]

Okay, so I hate it when people hijack a thread, but since I started it, I guess I can give myself permission...

Remember my MatchPrefixAll at the top of this thread that Ender rewrote using the same 'setting aside' technique that you used in your rewrite of DecToHex? (Okay, so Ender and comment have now drilled this simple concept into my thick skull. *******, one less thing I can do the hard way now!)

Anyway, I had made an aside that I was looking for a way to have it return it's result list with no trailing CR. Since Brian Dunning just added the ability to edit your CFs on his site, I decided to update it with Ender's better version.

As such I put my thinking cap on and figured out a way to rewrite Ender's rewrite with no trailing CR and no external function calls. I settled on using a second recursive call inside each iteration of the primary recursion to count the remaining prefix matches in each new input list. When it comes up with 0, it strips the CR off the value before setting it aside. At first I was worried that this would eat itself, but it works.

Check it out:

MatchPrefixAll ( List ; Prefix )

Case ( not IsEmpty ( List ) and not IsEmpty ( Prefix ) ;

Case ( Left ( List ; Length ( Prefix ) ) = Prefix ;

If ( ValueCount ( MatchPrefixAll ( MiddleValues ( List ; 2 ; ValueCount ( List ) - 1 ) ; Prefix ) ) > 0 ;

LeftValues ( List ; 1 ) ;

Substitute ( LeftValues ( List ; 1 ) ; "

[Ender]

Hi Jeremiah,

You should look for a method that won't require the additional recursive call. Having it in there greatly reduces the efficiency of the function (it goes from an O(n) function to an O(n^2) function, meaning instead of a 10 line List having 10 recursive calls, it would have on the order of 100 recursive calls.)

[dual_mon]

This is true, however, I can't think of a way to do it, barring external handling, such as trimming the trailing CR externally by wrapping it in something like this: Substitute ( Right ( MatchPrefixAll ( List ; Prefix ) ; 1 ; "

[comment]

I can't think of a way to do it

Why not use the piggy-backing method?

[Ender]

Well, after looking at it in more detail, it looks like the extra recursive call does not add that many recursions to the algorithm. Only n+1 recusions for n items. And since I can't see another way to get rid of that extra return, short of adding a parameter or a sub function, I'll concede that your algorithm is sound.

Good work.

[dual_mon]

Actually, I do not have the higher math (I will have to wait until I see my wife at home this evening so she can school me on the math to figure the exact numbers), however, I do believe that your instincts were correct about the non-linear increase in the number of function calls with the double nested version. After you asserted the n^2 increase, I did some quick Empiracal tests, as I alluded in my first reply by continually feeding it more values. As I mentioned, the function *does* seem to eat itself (hit the recursion limit) when you feed it somewhere between 10 and 20 values, depending on how many are matches. Either it is hitting the limit, or there is some other problem, because what happens, is it returns a "?" after a noticable delay while the processor spikes.

However, I came up with this simple modification on the same structure that adds zero additional calls.

Check it out:

MatchPrefixAll ( List ; Prefix )

Case ( not IsEmpty ( List ) and not IsEmpty ( Prefix ) ;

Let ( newlist = MatchPrefixAll ( MiddleValues ( List ; 2 ; ValueCount ( List ) - 1 ) ; Prefix ) ;

Case ( Left ( List ; Length ( Prefix ) ) = Prefix ;

If ( ValueCount ( newlist ) > 0 ;

LeftValues ( List ; 1 ) ;

Substitute ( LeftValues ( List ; 1 ) ; "