SAS Functions: SOUNDEX, COMPGED, and Their Alternatives
Introduction
In SAS, the SOUNDEX and COMPGED functions are powerful tools for text comparison, particularly when dealing with names or textual data that may have variations. In addition to these, SAS offers other functions like DIFFERENCE and SPEDIS that provide additional ways to measure similarity and distance between strings. This article explores these functions, provides examples, and compares their uses.
The SOUNDEX Function
The SOUNDEX function converts a character string into a phonetic code. This helps in matching names that sound similar but may be spelled differently. The function generates a four-character code based on pronunciation.
Syntax
SOUNDEX(string)Where string is the character string you want to encode.
Example
data names;
input name $20.;
soundex_code = soundex(name);
datalines;
John
Jon
Smith
Smythe
;
run;
proc print data=names;
run;In this example, “John” and “Jon” have the same SOUNDEX code, reflecting their similar pronunciation, while “Smith” and “Smythe” have different codes.
The COMPGED Function
The COMPGED function measures the similarity between two strings using the Generalized Edit Distance algorithm. This function is useful for fuzzy matching, especially when dealing with misspelled or slightly varied text.
Syntax
COMPGED(string1, string2)Where string1 and string2 are the strings to compare.
Example
data comparisons;
string1 = 'John';
string2 = 'Jon';
distance = compged(string1, string2);
datalines;
;
run;
proc print data=comparisons;
run;The COMPGED function returns a numerical value representing the edit distance between “John” and “Jon”. Lower values indicate higher similarity.
Alternative Functions
The DIFFERENCE Function
The DIFFERENCE function returns the difference between the SOUNDEX values of two strings. This function is useful for comparing the phonetic similarity of two strings directly.
Syntax
DIFFERENCE(string1, string2)Where string1 and string2 are the strings to compare.
Example
data soundex_comparison;
input name1 $20. name2 $20.;
diff = difference(name1, name2);
datalines;
John Jon
Smith Smythe
;
run;
proc print data=soundex_comparison;
run;In this example, the DIFFERENCE function compares the SOUNDEX values of “John” and “Jon”, and “Smith” and “Smythe”. Lower values indicate more similar phonetic representations.
The SPEDIS Function
The SPEDIS function measures the similarity between two strings based on the Soundex encoding and a variant of the Generalized Edit Distance. This function is useful for matching names with variations in spelling.
Syntax
SPEDIS(string1, string2)Where string1 and string2 are the strings to compare.
Example
data spedisp_comparison;
string1 = 'John';
string2 = 'Jon';
spedis_score = spedis(string1, string2);
datalines;
;
run;
proc print data=spedisp_comparison;
run;The SPEDIS function returns a score reflecting the similarity between “John” and “Jon”. A lower score indicates higher similarity, similar to COMPGED, but with a different approach to similarity measurement.
Comparison of Functions
Here’s a quick comparison of these functions:
- SOUNDEX: Encodes a string into a phonetic code. Useful for phonetic matching, but limited to sounds and does not consider spelling variations.
- COMPGED: Uses the Generalized Edit Distance algorithm to measure string similarity. Suitable for fuzzy matching with spelling variations.
- DIFFERENCE: Compares the phonetic similarity of two strings based on their
SOUNDEXvalues. Provides a direct measure of phonetic similarity. - SPEDIS: Measures similarity using a combination of Soundex and Edit Distance. Useful for matching names with spelling variations and phonetic differences.
Conclusion
The SOUNDEX and COMPGED functions are valuable tools for text comparison in SAS. By understanding their characteristics and how they compare to other functions like DIFFERENCE and SPEDIS, you can choose the most appropriate method for your specific text matching needs. Each function offers unique advantages depending on the nature of the text data and the type of comparison required.