Tracking progress in your program with SAS Enterprise Guide: another trick

I’ve got a new trick that you can use to track progress in a long-running SAS program, while using SAS Enterprise Guide.

I’ve previously written about the SYSECHO statement and how you can use it to update the Task Status window with custom messages. SYSECHO is a “global” statement in the SAS language, which works great within the boundaries of a PROC step to annotate your Task Status messages. For certain interactive PROCs (like SQL), you can pepper in SYSECHO statements to give more frequent and detailed updates.

But the DATA step, which is often used for long-running operations, cannot use multiple SYSECHO statements. When SAS compiles the DATA step it processes all global statements up front, so that only one use of the SYSECHO statement takes effect. This means that if you want to use multiple SYSECHO calls to report which observation or values are being processed as the DATA step runs, you can’t.

That is, you couldn’t, until now. With SAS 9.3 Maintenance 2 you can use a new function, named DOSUBL, to process any SAS statements (including global statements) “just in time” as your DATA step runs. DOSUBL accepts one or more SAS statements and executes them immediately, inline with the currently running DATA step. The statements must represent a “complete” segment of a SAS program: no partial PROCs or partial DATA steps. But it can execute any PROC step (bounded by RUN or QUIT) or a DATA step (bounded by RUN) or — key for our example — a SAS global statement.

NOTE: DOSUBL is an experimental function in SAS 9.3, and you really need the M2 revision or later for it to work well. It’s on target for a production release with SAS 9.4. It’s also the topic of a paper by Rick Langston at SAS Global Forum 2013 (here’s a preview from Jason’s paper last year).

If you have SAS 9.3M2, try running this from your SAS Enterprise Guide session. Note how I put a condition around the DOSUBL call; I don’t want to process the statement for every record, as that might get too chatty in the status window. Instead, I call it once for every 5 records.

/* Tracking where you are in the DATA step iterations */
data _null_;
 set sashelp.class;
 if mod(_n_,5)=0 then
  rc = dosubl(cats('SYSECHO "OBS N=',_n_,'";'));
 s = sleep(1); /* contrived delay - 1 second on Windows */
run;

Here’s another example that reports on progress, but instead of triggering the update based on record count, I’ve triggered it based on the current BY value in process:

proc sort data=sashelp.cars out=sortedcars;
by make;
run;
 
/* Tracking which "category" value is being processed */
data _null_;
 set sortedcars;
 by make;
 if first.make then do;
  rc = dosubl(cats('SYSECHO "OBS N=',_n_,' MAKE=',make,'";'));
 end;
run;

With a little imagination, you can devise some sophisticated SYSECHO statements that report on other aspects of the DATA step processing: the timings, the program data vector, and so on. It might become a useful debugging technique (in lieu of the DATA step debugger, which can’t run in a client application like SAS Enterprise Guide).

Other references:

Tracking progress in SAS programs with SAS Enterprise Guide
Executing a PROC from a DATA step