| This post was kindly contributed by SAS and R - go there to comment and to read the full post. |
We’re both users of multiple imputation for missing data. We believe it is the most practical principled method for incorporating the most information into data analysis. In fact, one of our more successful collaborations is a review of software for multiple imputation.
But, for me at least, there are times when a simpler form of imputation may be useful. For example, it may be desirable to calculate the mean of the observed values and substitute it for any missing values. Typically it would be unwise to attempt to use a data set completed in this way for formal inference, but it could be convenient under deadline pressure or for a very informal overview of the data.
Nick disagrees. He finds it hard to imagine any setting in which he would ever use such a primitive approach. He passes on to the reader the sage advice he received in graduate school: that making up data in such an ad-hoc fashion might be construed as dereliction or even misconduct. Use of single imputation approaches (which yield bias in many settings and attenuate estimates of variance) seems hard to justify in 2014. But one of the hallmarks of our partnership is that we can agree to disagree on an absolute ban, while jointly advising the reader to proceed with great caution.
SAS
In SAS, it would possible to approach this using proc means to find the means and then add them back into the data set in a data step. But there is a simpler way, using proc standard.
proc standard data=indata out=outdata replace;
run;
This will replace the values of all missing numeric variables in the indata data set with the mean of the observed values, and save the result in a new data set, outdata. To restrict the operation to specific variables, use a var statement.
R
There may be a function designed to do this in R, but it’s simple enough using the features of the language. We provide an option using the bracket ([) extractor operator and another using the ifelse() function. The latter may be more approachable for those less familiar with R.
df = data.frame(x = 1:20, y = c(1:10,rep(NA,10)))
df$y[is.na(df$y)] = mean(df$y, na.rm=TRUE)
# alternative
df = transform(df, y = ifelse(is.na(y), mean(y, na.rm=TRUE), y))
In the first example, we identify elements of y that are NA, and replace them with the mean, if so. In the second, we test each element of y; if it is NA, we replace with the mean, otherwise we replace with the original value.
An unrelated note about aggregators:We love aggregators! Aggregators collect blogs that have similar coverage for the convenience of readers, and for blog authors they offer a way to reach new audiences. SAS and R is aggregated by R-bloggers, PROC-X, and statsblogs with our permission, and by at least 2 other aggregating services which have never contacted us. If you read this on an aggregator that does not credit the blogs it incorporates, please come visit us at SAS and R. We answer comments there and offer direct subscriptions if you like our content. In addition, no one is allowed to profit by this work under our license; if you see advertisements on this page, the aggregator is violating the terms by which we publish our work.
| This post was kindly contributed by SAS and R - go there to comment and to read the full post. |