Not sure if the problem is with my Cleansing Package or my Data Cleanse configuration. Just using base Data Cleanse and a simple Custom Cleansing Package that happens to have multiword Standard Forms and Variations.
Consider the following two CP Entries:
Variation (ONE TWO THREE) to Standard Form (One Two)
Variation (THREE) to Standard Form (Three Four)
The Standardized results I get when the input is “ONE TWO THREE” is “One Two Three Four”. When the input is “THREE”, I get the correct results of “Three Four”.
BTW, I have removed the auto-generated “Phrase Words” leaving only Standards and Variations. It would seem that the parser is still matching individual words and concatenating multiple Standard Forms based on a single word in the input.
Is there a technique to force the rule to only be applied when the full input string matches the full Variation? Either in the CPB or in the Data Cleanse?
It looks at each word individually, so even though you define a 3 word entry and a standardized form … it will never hit that definition. You would have to define classifications for each word and a rule that ties all 3 together. (which is less than ideal)
An easier solution is to use a search and replace on the multi-word fields, stripping the white space out of them and entering them in the Data Dictionary as all one word. Again … less than ideal, but would work.
Well, I examined the rules using Advanced Mode and found that the first auto-generated rule was concatenating the primary attribute to itself ‘MyString + MyString’. I said “that can’t be right”. So I modified the rule to just be ‘MyString’, and the cleansing is done properly now. But, now, it squaks about AutoGeneratedRule1 not being found, since it is now a user defined rule.
Still seems to work OK, though the behavior in Design mode is different, in that it no longer generates the base Variation (that matches the Std Form), and then once I add the value as a Variation, I can only add more using Advanced mode. I can’t seem to find any decent documentation.
(BOB member since 2009-10-27)