| This post was kindly contributed by SAS Users - go there to comment and to read the full post. |
Just getting started with this series? Make sure to explore Part 1 and Part 2.
There are different ways you can use these two tools to accelerate model building, deployment and monitoring. Figure 1 summarizes best practices for conducting ModelOps using SAS Model Manager and Azure Machine Learning.
Figure 1: Five best practices to optimize the ModelOps life cycle on the Microsoft Cloud.
Best practice 1: Create models with repeatable machine learning pipelines using SAS® Model Manager
When data scientists create models in SAS Viya and register in SAS Model Manager, they can innovate and build faster. SAS Viya not only creates pipelines of reusable machine learning components but also allows users to easily import reusable components from any language. SAS Viya can also convert score code from one language to another, helping developers avoid recoding efforts. For example, SAS Viya can import R or Python packages, or even a Jupyter notebook, and instantly convert them to run in SAS code (and vice versa).
Central to these SAS capabilities is the modeling metadata repository that SAS uses for drift monitoring and oversight functions. The actual code repository can be in GitHub, Microsoft’s library for prebuilt code. Integrated SAS capabilities enable you to store and keep track of repeatable components in GitHub and make it easy for others to find and reuse them. In this way, SAS supports an ecosystem for model development. SAS Model Manager also provides this central modeling metadata repository and simultaneously enhances Azure MLOps processes.
As you build models, you can test them using SAS Model Manager before putting them into production. Whether you create in Python, R, SAS or a Jupyter Notebook, the software enables you to efficiently validate the model logic using precise methodology that records each test. As a result, you always know models will run seamlessly in your deployed environment.
Best practice 2: Apply automation to your ModelOps rollout and audit trail
SAS Model Manager also facilitates more effective governance by enabling users to:
-
• Collect all models, projects and pipelines in a searchable, secure and versioned repository.
• Keep track of different versions for each model so you can track model history and ownership changes in a single, readily available user interface.
• Compare models from different languages and libraries side by side (for instance, SAS, TensorFlow, scikit-learn and more).
• Maintain model version control and history tracking in one place.
• Access models and model-score artifacts using open REST APIs.
Best practice 3: Use Azure Machine Learning to deploy models into the Microsoft Cloud
Now that you have validated your model, the analytic scoring code will execute correctly, and you can deploy your model anywhere in the Microsoft Cloud. You can call your model from Microsoft Dynamics 365, Azure Synapse Analytics and Microsoft Power Platform, including Power BI, which empowers you to bring your models to more places.
You can also deploy your model to the OCI (Open Container Initiative) compliant Docker containers in the Microsoft Cloud as a lightweight image in SAS Container Runtime (SCR). SCR images provide a modern, bulletproof, reliable, nonproprietary system for transferring and executing runtime code. This greatly simplifies efforts to move code from your development system to your production system. Simply register your SAS containerized model to your favorite Microsoft Azure or docker location and create an app or use REST APIs to score new data against your published model with SAS Container Runtime.
Best practice 4: Use SAS® Model Manager to monitor performance and drift
Models begin to degrade the moment you put them in production as conditions in the market, customer behavior and data change over time. SAS Model Manager includes performance monitoring to easily track and view performance against the KPIs that have been predetermined with IT and line-of-business experts.
SAS Model Manager also supplements the ModelOps process by empowering line-of-business experts to take ownership of the monitoring process. For example, SAS Model Manager provides visual analytics and reporting from which you can centrally monitor model performance within Azure Machine Learning and detect model drift. Use it to:
-
• Generate out-of-the-box performance reports using a wizard.
• Easily access data to build your own reports and KPIs.
• Schedule jobs to detect model drift and automatically retrain models in case of poor performance or bias detection.
Data visualizations make it easy for teammates in the line of business to see data score changes, monitor changes in accuracy over time and see which models are drifting and need retraining.
Best practice 5: Use SAS® Model Manager to retrain, score and redeploy models faster
If model performance degrades, your organization should embark on one of three approaches:
-
• Retrain the existing model on new data.
• Revise the model with new techniques (such as feature engineering or new data elements).
• Replace the model entirely with a better model.
But regardless of your choice, the critical questions remain the same:
-
• How do you know when you need to retrain the model?
• Once the model is retrained, how do you determine when to redeploy the model?
• Can you predict how long the model will be relevant?
SAS Model Manager helps you answer these questions and take timely action. If it observes model drift, it proactively alerts you with a built-in feedback mechanism and vastly simplifies, optimizes and accelerates model retraining and redeployment. In fact, SAS allows you to revise a model with new techniques (such as feature engineering or new data elements) or replace an entire model with a better one.
Learn more
There are many advantages to using SAS Model Manager with Azure Machine Learning. Most notably, you gain a no-code/low-code environment in which business analysts can build and deploy SAS and open-source models, continuously monitor for performance and drift, and retrain models – all within a single, intuitive SAS user interface. This frees you to establish collaborative, cross-functional teams that ensure models solve real business problems today, and continue to do so in the future. Issues get detected earlier – and retraining and redeployment happen faster – because all models are hosted, collected and tracked in one application. And the efficiencies you gain from automation and real-time monitoring, reporting, testing and more means you can scale your use of ML models in your Microsoft environment with confidence.
Next time, you’ll see how SAS Model Manager, Azure Machine Learning, and Power BI work in practice, by learning how to call a SAS model in Power BI.
To learn more about ModelOps and our partnership with Microsoft, see our whitepaper: ModelOps with SAS Viya on Azure.
5 best practices for optimizing ModelOps with SAS® Model Manager and Azure Machine Learning, part 3 was published on SAS Users.
| This post was kindly contributed by SAS Users - go there to comment and to read the full post. |