| This post was kindly contributed by SAS Users - go there to comment and to read the full post. |
Set it and forget it. That’s how SAS Viya Jobs and Flows should work, right? Well, just like most other things in life, it’s not that simple. What if you have multiple Jobs set to run in a Flow and one fails? Do you exit the process or continue with subsequent Jobs? This post provides some tips on how to schedule flows in the SAS Viya Environment Manager to solve these types of situations.
What is a Job?
A SAS Viya job consists of a program and its definition (metadata related to the program). The job definition includes information such as the job name, the author and when it was created.
What is a Flow?
A flow is defined as a container of jobs. Flows must contain at least one job or at least one flow, which is called a subflow. You can schedule a flow to run now or in the future.
You can catch up on the basics of Jobs and Flows in the SAS Viya Administration documentation.
The Problem
Customer A has two jobs in a flow. Job 2 is dependent on Job 1 completing with any exit code. If Job 1 fails, then Customer A still wants his flow to complete successfully.
Customer B has two jobs in a flow. Job 2 is dependent on Job 1 completing with any exit code. If Job 1 fails, then Customer B wants his flow to fail.
How do you satisfy the requirements from both Customer A and Customer B?
The Solution
The Flow Completion Criteria setting, in the SAS Viya Environment Manager, controls the outcome of the flow. The setting controls the result of the flow based on the results of the jobs within. To configure the criteria, go to the Jobs and Flows page. Then, on the Scheduling tab, select New ►Job Flow. On the right-hand side, click the Attributes icon, to get to the Flow Completion Criteria. Below is a screenshot of selections for the configuration.
A Deeper Look into Flow Completion Criteria
Scenario 1
In this example, I have two jobs, Bad Program and Good Program, in a flow labelled Test Flow. Bad Program completes with an exit code. Good Program is dependent on Bad Program completing with any exit code. Good Program completes successfully.
I use the default Flow Completion Criteria setting:
and looking at the results of the run, the flow completes successfully:
Scenario 2
What if I want the flow to fail when any jobs fail? I’ll need to modify the Flow Completion Criteria. This time I’ll select Only selected items end. I then have a choice of Any or All jobs. I’ll choose Any and the two jobs relevant for the criteria.
In this instance the flow fails:
Now, I know what you are thinking. What about Good Program? It did not run because I set the flow to end if any of the jobs ended. Since Good Program is dependent on Bad Program, and Bad Program ended, Test Flow ended too.
Scenario 3
In the next scenario, I’ll choose the All jobs option.
With this setting, no matter which job fails or succeeds, the flow should complete only when all jobs end with an exit code.
Scenario 4
Here is one more example. I want to add an AND gate to the flow.
Bad Program and Good Program kick off a flow called Ursula_s Import Program. The job dependency of Bad Program and Good Program ends with any exit code. I’ll run the scenario using the default settings:
And I get the following results:
Here is what happened: Good Program and Bad Program executed, but Ursula_s Import Program did not. I really want Ursula_s Import Program to execute (it has my name in the job name, so, of course, I want the job to execute). So let’s adjust the configuration to get the desired flow outcome.
I’ll update the criteria to Only selected items end and Any:
And here are the results:
The flow completed successfully as did Ursula_s Import Program, despite Bad Program failing. What did I do? I modified how I wanted to complete the flow. I wanted two things: (1) the flow to complete successfully and (2) Ursula_s Import Program to complete. I forced the flow to complete because I selected Any and I chose the last job of the flow sequence to run. I told the flow to complete after running Ursula_s Import Program.
Conclusion
As any fashion expert knows, one size does not fit all, and that especially applies to flows. Luckily, you can manipulate the flows to fit your business needs. All you need to do is to understand the logic and modify the Flow Completion Criteria.
Learn More
Go with the Job Flow in SAS Viya 3.5
SAS Help Center: Jobs and Flows: How To (Job Flows)
Tips for Scheduling Flows with SAS Viya Environment Manager was published on SAS Users.
| This post was kindly contributed by SAS Users - go there to comment and to read the full post. |