The machine learning lifecycle encompasses every stage of machine learning model development, deployment, and performance monitoring. This includes the initial conception of the model as an answer to an organisation’s problem, to the ongoing optimisation that’s required to keep a model accurate and effective. Machine learning models can degrade over time due to a range of factors, such as the external context of the data shifting. A model should be regularly reworked and optimised to resolve detected model shift or bias, achieving a continuous cycle of improvement.
Machine learning development is a complex process, but the journey doesn’t finish once the model is deployed. The full machine learning model lifecycle goes beyond the initial stages of model research and machine learning model deployment. It should include monitoring the model’s health and performance once it is deployed in a live environment too. Other considerations include steps to embed the model in the wider organisation, and important elements such as model governance and management.
This guide explores the basics of the machine learning model lifecycle, explaining the different stages and what they mean.
6 stages of the machine learning model lifecycle
Getting a machine learning model effectively embedded within the organisation is a complex task. The lifecycle will need to involve many varied stakeholders from across the organisation. The development and deployment of the model will need data science specialists, but other stages will involve stakeholders who may not have data science backgrounds or knowledge. As machine learning models become more and more common across different settings and sectors, a holistic view of a model’s lifecycle becomes more important.
The 6 main stages of the machine learning model lifecycle moves from initial planning through to training and deployment. There should also be a major focus on achieving a cycle of monitoring, optimisation and maintenance to ensure the model stays as effective as possible.
The 6 stages of the machine learning model lifecycle are:
- Define objectives of the project
- Plan for machine learning model lifecycle management
- Collect and prepare data
- Train and evaluate the model
- Deploy the model to a live environment
- Monitor and optimise model performance
Define objectives of the project
The early stage of the machine learning model lifecycle should consist of defining and planning the focus and scope of the project. Clearly defining the problem a machine learning model will help to solve should be the first step. Models are increasingly being levered in a range of environments to solve business and organisational needs. Clearly defined objectives will ensure machine learning is the best solution for the problem. Otherwise, the problem may be solved through less resource-intensive means.
At this stage it will be useful to understand the system environment a model will be embedded in, and the data that’s accessible within the organisation. As with any system and software, a machine learning model will need to be mapped within the organisation’s network to understand any cybersecurity issues and dependencies. As machine learning is so data dependent, the source and type of data should be clearly defined too. The overall aim of the project and the type of data available will impact the type of machine learning model that is selected and deployed.
All decisions should be clearly documented so that the risks and rewards of developing a machine learning model are understood across the organisation. Clearly defining the objectives and aims of the project at an early stage will keep the project on track, and help to define model success once deployed.
Steps that should be achieved at this stage include:
- Confirming the operational need for a machine learning model over other options and solutions.
- Clearly defining the aims and goals of the model.
- Highlighting problems a model will need to solve and the metrics for success once deployed.
- Identifying the individuals and teams responsible for machine learning model development, deployment, monitoring, and governance.
Plan for machine learning model lifecycle management
Once the scope and aims of the project are defined, working out policies for the machine learning model lifecycle management is important. Model development and deployment is a complex process, so a clear management process should be defined at this early stage. Identifying the key stakeholders will help to streamline the decision-making process throughout the machine learning model lifecycle.
An important element is embedding machine learning model governance processes in the organisation, which include policies on version control and change management. This should feature in any information security policies enacted by the organisation too. Focusing on explainability at an early stage is also important, so that elements of machine learning model management can be performed by non-technical stakeholders. Explainability in machine learning is the process of understanding and interpreting how and why a model makes a decision. Explainability is crucial if a model is used to make decisions in a regulated industry such as machine learning in finance. It’s also an important part of the machine learning model management process, as decisions can be clearly understood and documented across the organisation.
The machine learning model lifecycle management stage should include:
- Clearly defined model governance policies which fit in the wider system security policies.
- Identification of project owners for each step of building, development and monitoring.
- Delegation of responsibility for the project and ongoing maintenance and optimisation of the model.
- Embedding a process for scrutinising model performance to identify bias and model drift.
Collect and prepare data
High quality data is an integral part of a successful machine learning model, regardless of the type of machine learning model selected. Models rely on huge arrays of high quality data to learn and train from. Data sets are used to both train and validate the effectiveness of a model, so high quality data is vital. The first step is ensuring a reliable source of data is available. The type of data available should have been identified in an earlier stage, as this has a direct impact on the type of machine learning algorithm required.
The level of preparation required will be relative to the type of machine learning algorithm chosen. For example, supervised machine learning models require labelled datasets to learn the relationship between input and output data. Labelled data must usually be prepared by a data scientist, which is a labour-intensive process. Unsupervised machine learning on the other hand is often used for identifying trends and patterns in datasets. It will learn from unlabelled data sets which will usually require less initial preparation compared to labelled data.
Whatever the type of machine learning model selected, initial exploratory analysis of the data should be performed by a data specialist. Exploratory techniques can help data scientists understand basic features of the data, such as its basic features or groupings. This informs both the preparation of the data and the configuration of human-controlled elements like hyperparameters.
Across all types of machine learning, the quality of the data is important. Poor quality data might mean outliers or bias within the training datasets. This will lead directly to an ineffectual and inaccurate model. Collected training data should be prepared and cleaned, with techniques in machine learning anomaly detection a key part of this phase.
The data collection and preparation phase should include:
- Ensuring a secure and reliable source of high quality data of the right type and format.
- Exploratory data analysis by a data scientist to understand the data’s basic groupings and features.
- Preparing and cleaning of training data through anomaly and outlier detection and analysis.
- The splitting of data into training and testing datasets ahead of training and evaluation.
Train and evaluate the model
Machine learning models learn from training data, usually in an offline or local environment. Different machine learning algorithms will have different training processes. Unsupervised machine learning sees the model learn from unlabelled data, usually to cluster data or identify patterns. Supervised machine learning will see a model learn from a labelled data set prepared by a data scientist, with labelled input and output data. The available data will usually be split into training and testing datasets. The model will be trained on the larger data set, and evaluated on the other unseen data. This is the process of cross validation.
Cross validation is a key part of evaluating a model’s ability to generalise ahead of machine learning deployment. Generalisation is a model’s ability to function with new and unseen data, and is a core aim of the training process. Although a model may be achieving high levels of accuracy on training data, the model may not have the same levels of accuracy when deployed to a live environment with new and unseen data. The model may be overfit on the training data, so unable to discern patterns within the new and unseen data.
In the worst cases, overfitting can cause the model to fail completely once deployed. This is why evaluation is a key stage before final deployment. There are a range of cross validation techniques which can be used to evaluate a model’s performance with unseen data.
Deploy the model to a live environment
The environment for model deployment is a key consideration, as the model can usually be scaled to process less or more data depending on the resources attributed to it. Containers have become a popular way to deploy machine learning models for this reason. A containerised approach can provide a consistent and scalable environment for the model, even though the containers may be drawing resources from a range of settings and systems. The approach also makes updating distinct parts of the model more straightforward. Container orchestration platforms like Kubernetes help to automate the management of containerised machine learning models, including monitoring, scaling and maintaining the containers.
Another consideration for deployment is ensuring the model is properly embedded in the organisation. This could mean deploying an effective communications campaign to update the wider organisation ahead of the deployment. Another example could be a series of training sessions to prepare non-technical colleagues. A thought-out deployment plan will ensure the model is utilised to its full potential across the organisation.
The final considerations mirror the deployment of any piece of software. Model deployment will often be a different team from model development, so code needs to be explained with a clear ‘read me’ file to aid deployment too. The code should also be cleaned and tested too before live deployment, to make sure it’s legible outside of a training environment.
The deployment phase should include:
- Preparing an environment for deployment.
- Testing code quality ahead of deployment.
- Creating explanatory ‘readme’ files for the deployment team.
- Communicating with the wider organisation to properly embed the model.
Monitor and optimise model performance
The machine learning model lifecycle doesn’t stop once the model has been deployed. The model should be continuously monitored for signs that it is degraded over time, to ensure ongoing model accuracy. Machine learning monitoring is the trigger for intervention when a model may be underperforming. Once issues like model drift or bias are detected, a model can be retrained or refitted to improve accuracy. Like any other system or software in an organisation’s network, a machine learning model should also be monitored for system health. This could include monitoring model output or resource use.
Machine learning optimisation is a process for improving the accuracy of a model, often by tweaking the human-controlled elements of the model called hyperparameters. The effectiveness of a model is usually measured through a loss function, the difference between the actual value and predicted value of data. A core aim of machine learning is to minimise this through constant optimisation. This means a cycle of monitoring, optimisation and deployment to iteratively improve the model.
The final phase should include:
- Monitoring the ongoing health of machine learning models, such as use of GPU resources or data flow.
- Detecting model drift and any drop in model accuracy.
- Identifying any bias in output caused from the training data being unrepresentative.
Operationalise Machine Learning for Every Organisation
Seldon moves machine learning from POC to production to scale, reducing time-to-value so models can get to work up to 85% quicker. In this rapidly changing environment, Seldon can give you the edge you need to supercharge your performance.
With Seldon Deploy, your business can efficiently manage and monitor machine learning, minimise risk, and understand how machine learning models impact decisions and business processes. Meaning you know your team has done its due diligence in creating a more equitable system while boosting performance.
Deploy machine learning in your organisations effectively and efficiently. Talk to our team about machine learning solutions today.