As my colleague Margaret Crevar recently wrote, it is useful to know how long SAS programs take to run. Margaret and others have written about how to use the SAS FULLSTIMER option to monitor the performance of the SAS system. In fact, SAS distributes a macro that enables you to

The post Read more »

Tags: Efficiency, Getting Started, Uncategorized

Posted in SAS | Comments Off on Compare the performance of algorithms in SAS

Imagine that you have one million rows of numerical data and you want to determine if a particular "target" value occurs. How might you find where the value occurs? For univariate data, this is an easy problem. In the SAS DATA step you can use a WHERE clause or a

The post Read more »

Tags: data analysis, Efficiency, Uncategorized

Posted in SAS | Comments Off on Finding observations that match a target value

How do you count the number of unique rows in a matrix? The simplest algorithm is to sort the data and then iterate down the rows, comparing each row with the previous row. However, this algorithm has two shortcomings: it physically sorts the data (which means that the original locations

Read more »

Tags: Efficiency, Statistical Programming, Uncategorized

Posted in SAS | Comments Off on Count the number of unique rows in a matrix

The power of a statistical test measures the test's ability to detect a specific alternate hypothesis. For example, educational researchers might want to compare the mean scores of boys and girls on a standardized test. They plan to use the well-known two-sample t test. The null hypothesis is that the

Read more »

Tags: Efficiency, Sampling and Simulation, Statistical Programming, Uncategorized

Posted in SAS | Comments Off on Using simulation to estimate the power of a statistical test

Last week someone posted an interesting question to the SAS/IML Support Community. The problem involved four nested DO loops and took hours to run. By transforming several nested DO loops into an equivalent matrix operation, I was able to reduce the run time to about one second. The process of

Read more »

Tags: Efficiency, Uncategorized, vectorization

Posted in SAS | Comments Off on How to vectorize computations in a matrix language

The SAS/IML language secretly creates temporary variables. Most of the time programmers aren't even aware that the language does this. However, there is one situation where if you don't think carefully about temporary variables, your program will silently produce an error. And as every programmer knows, silent wrong numbers are

Read more »

Tags: Efficiency, Getting Started, Tips and Techniques, Uncategorized

Posted in SAS | Comments Off on Oh, those pesky temporary variables!

A while ago I saw a blog post on how to simulate Bernoulli outcomes when the probability of generating a 1 (success) varies from observation to observation. I've done this often in SAS, both in the DATA step and in the SAS/IML language. For example, when simulating data that satisfied

Read more »

Tags: Efficiency, Sampling and Simulation, Statistical Programming, Tips and Techniques, Uncategorized

Posted in SAS | Comments Off on Generate binary outcomes with varying probability

In a recent article on efficient simulation from a truncated distribution, I wrote some SAS/IML code that used the LOC function to find and exclude observations that satisfy some criterion. Some readers came up with an alternative algorithm that uses the REMOVE function instead of subscripts. I remarked in a

Read more »

Tags: data analysis, Efficiency, Tips and Techniques, Uncategorized

Posted in SAS | Comments Off on Remove or keep: Which is faster?