Parsing captured data

[LaRetta]

Scenario: Allow Staff to grab information via [Ctrl-C] (from within emails, websites, address lookup programs, letters or whatever) and insert accurate data into our database fields without the need to retype it (which allows typing errors) and without the need to jump back and forth copying & pasting bits and pieces of information (which might also insert leading/trailing spaces or garbage) as well as being a major pain.

This would usually be used with the Customers or Prospects DB but not necessarily. I want to attach this parser DB to several files as a separate file and, although we will be networked, I

[Ugo DI LUCA]

You can still use a SetField[GlobalValue,:Capture:TextField] and then the other way around without the copy /paste process.

The solution breaks even if an empty line remains between 2 so that you'd need to change your first step to

Substitute(Substitute(gTextField & "

[DanBrill]

Hey LaRetta,

This is a cool concept, and it looks like you are well on your way. As for the 10 line limit, increase the size of the portal to, say, 100 lines. Of course this will cause the portal to run on past the length of the form and look awful. So cover that portion with a solid rectangle, no border, same color as the background. Define the rectangle as a button and have it run a script, 'exit', which has 1 step, exit script. (This will prevent the user from selecting portal rows that are past the bottom of the form, which would be really confusing since they wouldn't realize that they had selected anything at all, but it would suddenly be in the clipboard.) As lines get used up, the portal moves up the hidden lines.

As for the insertion of the break character, can't they just go into the field, position the cursor, and hit enter? You could script the insertion of the paragraph charcter too, but why bother? They still have to position the cursor in the text box, so once they've done that its easier to just hit enter than to click another button.

I can't think of a way to ensure that the state will be only two characters. (Well, at least the right two characters! ) You could set up a long case statement that would substitute state names (like Oregon) with the proper abbreviation (OR). But again, is it worth the trouble? Maybe you could just have the state field be a drop-down of the states and have that much entered manually.

I've played with it a bit to automatically suggest fields based on labels. I've set up first name, last name, phone, and e-mail.

See what you think.

Dan

WebParser.zip

[LaRetta]

Hi Ugo! Hi Dan!

Thanks for helping me with this. The reason I decided to use Copy/Paste instead of Set Field, Ugo, was because using Set Field bypasses Field Validation. And Dan, that's how I can validate State and all the other fields. I would prefer not writing each validation within the script, when I have already established many of the 'rules' at the field level for all other data-entry layouts in each Main database.

I am unsure why it breaks with more than 10 lines. And will covering the portal really work? I think anything placed over a portal disappears and the portal will still display. And I don't want Users to be able to scroll down the page and the page scroll bar would still show.

As for insertion of the break character, yes, Users can now insert their cursor and hit enter - but I was thinking of allow break characters, then the parser make several determinations: 1) Split by break character, 2) Look for Labels and either remove them (?) or try to match to existing field names, 3) Search for web & email addresses and automatically take care of those fields, and 4) Clean up any garbage characters, trim, etc. And only then, present the User with the remaining lines to parse (or ReParse) as needed.

Also, is there a way to determine if a User's clipboard contains anything? Thanks for the calc, Ugo! I'll incorporate that into my solution. I'm still trying to pull together the best ideas at this point! Now I just need to accomplish it. Please keep the ideas coming! If you were a User, what would you like or dislike about it? As a Developer, what can I do to protect Users from messing it up? How can I fix my current problems in it? All ideas welcome.

LaRetta

[Ugo DI LUCA]

Hi LaRetta,

I stopped Copy/Paste in all my solutions since one day, about 2 years ago, my brother was playing with the Internet while a loop script was running backwards, using copy/paste.

All what he copied from the web were obviously pasted by FM as well, and we ended up in a total mess, and a few nights of debugging.

I mentionned this was my brother, because, if not, this user would just not be on that earth anymore (or would it be the developer that used this f...Copy/Paste )

This sure won't apply here. Just kidding before week-end starts with this bad "souvenir"

To make sure the user's clipboard contains nothing, just force it be such with a blank field and a copy step to that field

[DanBrill]

Hey,

No, covering the portal seems to work fine. In the demo I reposted I set the portal to to 100 lines and put the box over it. No problems. The scroll bars are another issue, and without better window control native to FM, I can't think of a way around it. Is it really a problem if they just scroll down to see a blank page?

To determine if a users clipboard contains anything (text anyway), go to a temporary global field and paste. Then if the field <>"" there's something in the clipboard.

Let us know what you come up with. This is a neat idea.

Dan

[DanBrill]

Hmmm. I wonder if the hacker broke into the system and deleted a few day's worth of data just so we could have Ugo's Santa picture back....

[LaRetta]

Yeah, normally I don't use copy/paste because I've pasted huge multi-lines into a Quark document or something before I noticed it and was able to Undo. Major bummer. But, in this case, it is the User which uses the clipboard and not a script running it, so I feel pretty good about it.

Uh, Dan, thanks for the file. Truthfully, I hadn't looked yet but am eager to ... we are dealing with PeachTree crashing problems today (a great way to end the week) I think, as Ugo says, it shouldn't be a problem with the portal adding new records with my script. I just don't understand why it won't!! I would like to understand it and fix that problem if possible, before changing it. Who knows! Depending upon how I end up designing this thing, I may dump the portal entirely and just use the Parsing db to create all the lines first and 'do its thing', and then display them on a list view or something.

I just like the 'point' and 'point' methodology. Anyone come up with alternate methods that I might consider?

LaRetta

[Ugo DI LUCA]

Well, I've looked back to the script and the 'Next Field' step won't help you create a new row as it is related to what is on the Layout. So won't help the GoTo Portal Row [Next] as it doesn't exist. In fact, when you've reached the 10th row, it will go to the next field in your Tab order, which actually is the Text Field itself !

IMO, the NextField could be useful in a SelfJoin portal, but not with this kind of relationship you have (I may be wrong, didn't tested it...).

I think you'd need an external script to create a new record and base a relationship on it, if you want to keep the Portal to 10 rows without playing with Dan's idea (which works though and shouldn't be troublesome)

[Ugo DI LUCA]

So everyday brings its lot of new stuff. You can create new records if the navigation bar is on. You can't if it itsn't in place.

[Ugo DI LUCA]

DanBrill said:

Hmmm. I wonder if the hacker broke into the system and deleted a few day's worth of data just so we could have Ugo's Santa picture back....

Well, I had my helmet back, but can't find it anymore.

You'd have to live with this. I wish XMas would last all year, na ?

[Lee Smith]

Ugo,

I seem to remember that you promised to post your actual picture as a Xmas present to the rest of us.

Forget the Helmet, lets see your face buddy.

Lee

[LaRetta]

Bob Weaver! You're a text parsing master! What would you do?

Thanks so much for the input Ugo. And Dan? I appreciate the file - it has helped a lot! I've decided I will probably handle the entire parse in the related db first, before presenting to the User. In this way, I can let the parsing script do its thing, create however many records needed as best it can, identify labels (and set a secondary field with its recommended field name match), search for email and web addresses and remove some junk first. Portal length will no longer be an issue then.

I want the PatternCount() match from Labels to Field Names to be generic; based upon Similarity % instead of hard-coded and messages to recommend (as Dan indicated). In this way, it can be easily used with other DBs and script won't need to be hardcoded. And the same with the Paste (I muttered under my breath while having to create each Paste to specific Field). And, as it stands, I couldn't loop through the DB fields using Status(FieldName) with GetField because (if those db fields were in the tab order), the loop to create the portal records would also jump to those fields (and again, I would lose field validation).

BTW, address validation will be via related Address DBs (and members of Value Lists) supplied by the Post Office (with portals that display if more than one city matches the zip or area code for manual click selection). I don't want to override those Validations with scripts (that ignore field validation) and I don't want to hard-code the validations AGAIN within my scripts.

I also want to account for (and remove) multiple spaces, multiple '....' (but not the period from St.) so I need to list out several scenarios here. And what happens if they grab a graphic or And once the parser is finished, the User can then be presented with a similar view as the one currently displayed (possibly). I still really like the point-click-point theory.

I think I'll go back to pen and paper on it. Additional suggestions, or observation of possible problems, would be much appreciated before I rewrite this thing.

And yeah, Lee, I second your vote to see the 'real' Ugo!!!

LaRetta

[LaRetta]

Ugo said...You can create new records if the navigation bar is on. You can't if it itsn't in place.

Uh, you mean the Status Bar Ugo? I allowed the Status Bar and pasted six more consecutive lines and it broke while parsing. It would be interesting to learn whether some things can be done with the Status Bar showing that can't otherwise! Or are you talking about something else here? I'm intrigued - how did you get it to work?

LaRetta

[Ugo DI LUCA]

No, I meant the Portal Scroll.

If you disallow it, you can't go up to the 10th row with the "Next Field" step, while if you have it shown, it works like a charm.

It would still be invisible as you formated the portal to be so. Then, just allow the scroll and the job is done.

[LaRetta]

Huh??? I had originally tried that and it didn't work. I just tried it again and it still doesn't work?! Oh! I want to figure out what the difference is! Can you send me the file with it working via portal scroll showing?

LaRetta

[Ugo DI LUCA]

Sure

WebParser2.zip

[LaRetta]

Hi Ugo,

I wonder if it's a platform issue. Because I took your file (didn't touch the scroll), doubled the lines so it ended up with 14 (with no carriage return at the end or no spaces between) in gDemoText. Clicked Reset Demo and reran the Parse. And it breaks. It creates 8 records and the first three are blank and so is the last one (see Parsing file). Drat!

See attached! Scroll is active.

Edited: Demo removed

LaRetta

[Ugo DI LUCA]

Hi LaRetta,

Not a Cross Platform issue at all.

Your script does

SetField["gCapture", "Left(gTextField, Position( gTextField , "

[LaRetta]

Hi Ugo! Hi Everyone!

I removed the 'breaking' copy of the Parser. It was provided only to get assistance and clarification from Ugo on why he said it would work with scroll displayed. Ugo, my comment about cross platform was only because nothing else would make sense on why the scroll would work for you and not for me. It appears it wouldn't. And gCapture doesn't create 7 lines, but is just a holding field.

I again can not understand why, since the gCounter should control it and keep looping according to the count and the count is based upon

[Ugo DI LUCA]

LaRetta said:

And gCapture doesn't create 7 lines, but is just a holding field.

I know that. Its a capture that changes at each incrementaion of the counter.

I again can not understand why, since the gCounter should control it and keep looping according to the count and the count is based upon

[LaRetta]

Because, once gCapture has been "inserted" into a portal row, your next step was to substitute gCapture from the gText.

As soon as you said this, I thought I understood, because it removes both lines (which are duplicates). But I tried it with no duplications whatsoever and it still breaks.

I'm too tired, I think. I'll set this aside and attack it after some rest in a few days. And, truthfully, I had realized I didn't even need gCapture and I didn't even need to delete the text as I progressed. Maybe Staff would want to use a line again for another field also. I'm sure everything you said will make sense later after some sleep.

This file was a first concept and provided to give a vision of what I wanted to accomplish. I knew it wouldn't be the end-product by any means. Yep, this idea will be reworked. Thanks for all your help (and patience).

Oh dear. I didn't have the scroll turned on. Where did I get this 'unscrolled' version from? Once turned on and without duplicate lines, it works right! But I'll still use Replace() instead and still rework the file. Duh! I told I was brain dead.

LaRetta