IBM 15 Manual Do Utilizador
Chapter
6
6
6
6
Handling Missing Values
Overview of Missing Values
During the Data Preparation phase of data mining, you will often want to replace missing values
in the data. Missing values are values in the data set that are unknown, uncollected, or incorrectly
entered. Usually, such values are invalid for their fields. For example, the field Sex should contain
the values M and F. If you discover the values Y or Z in the field, you can safely assume that such
values are invalid and should therefore be interpreted as blanks. Likewise, a negative value for the
field Age is meaningless and should also be interpreted as a blank. Frequently, such obviously
wrong values are purposely entered, or fields left blank, during a questionnaire to indicate a
nonresponse. At times, you may want to examine these blanks more closely to determine whether
a nonresponse, such as the refusal to give one’s age, is a factor in predicting a specific outcome.
in the data. Missing values are values in the data set that are unknown, uncollected, or incorrectly
entered. Usually, such values are invalid for their fields. For example, the field Sex should contain
the values M and F. If you discover the values Y or Z in the field, you can safely assume that such
values are invalid and should therefore be interpreted as blanks. Likewise, a negative value for the
field Age is meaningless and should also be interpreted as a blank. Frequently, such obviously
wrong values are purposely entered, or fields left blank, during a questionnaire to indicate a
nonresponse. At times, you may want to examine these blanks more closely to determine whether
a nonresponse, such as the refusal to give one’s age, is a factor in predicting a specific outcome.
Some modeling techniques handle missing data better than others. For example, C5.0 and
Apriori cope well with values that are explicitly declared as “missing” in a Type node. Other
modeling techniques have trouble dealing with missing values and experience longer training
times, resulting in less-accurate models.
modeling techniques have trouble dealing with missing values and experience longer training
times, resulting in less-accurate models.
There are several types of missing values recognized by IBM® SPSS® Modeler:
Null or system-missing values.
These are nonstring values that have been left blank in the
database or source file and have not been specifically defined as “missing” in a source or
Type node. System-missing values are displayed as
Type node. System-missing values are displayed as
$null$
. Note that empty strings are not
considered nulls in SPSS Modeler, although they may be treated as nulls by certain databases.
Empty strings and white space.
Empty string values and white space (strings with no visible
characters) are treated as distinct from null values. Empty strings are treated as equivalent to
white space for most purposes. For example, if you select the option to treat white space as
blanks in a source or Type node, this setting applies to empty strings as well.
white space for most purposes. For example, if you select the option to treat white space as
blanks in a source or Type node, this setting applies to empty strings as well.
Blank or user-defined missing values.
These are values such as
unknown, 99, or –1 that are
explicitly defined in a source node or Type node as missing. Optionally, you can also choose
to treat nulls and white space as blanks, which allows them to be flagged for special treatment
and to be excluded from most calculations. For example, you can use the
to treat nulls and white space as blanks, which allows them to be flagged for special treatment
and to be excluded from most calculations. For example, you can use the
@BLANK function to
treat these values, along with other types of missing values, as blanks.
© Copyright IBM Corporation 1994, 2012.
99