Last week, the Seldon team spoke at and attended this year’s Nordic Data Science Machine Learning (NDSML) Summit, which brought together members of the data science and machine learning community from across the Nordics to explore developments in advanced analytics, data science, and machine learning (ML).
Across a talk and three roundtables, the Seldon team drove conversation around the state of productionising ML models, how 5G is enabling network-wide machine learning, what financial services organisations can gain from ML while abiding by compliance expectations, and how ML may facilitate a recovery in retail in 2021.
Production Machine Learning: Orchestration, Monitoring, Explainability, and Governance at Scale
Amid the widespread adoption of ML by organisations across all sectors, we are becoming increasingly aware of the risks associated with sub-par ML deployments – cybersecurity breaches, software outages, misuse of personal data, and algorithmic bias.
Seldon’s CEO and founder Alex Housley spoke to NDSML about how organisations can avoid these pitfalls, and gave an outline on how to approach productionising machine learning deployments so as to make them more secure, reliable, responsible, and ethical. To achieve this, teams need to be able to orchestrate, monitor, explain, and handle governance for ML models at scale.
Alex discussed the importance of a nuanced approach to orchestration when deploying ML at scale. While at the R&D stage a simple and straightforward deployment on Flask may be great for teams, a scaled production-level deployment usually requires multiple optimised model servers, metrics and tracing, lineage and audibility, and other features that Flask simply wasn’t built to accommodate.
Instead, Alex explained why more complex architecture leveraging the capabilities of Kubernetes is more suitable for scaled ML deployments. Through containerising ML models, teams can abstract away complex ML concepts into standardised infrastructural components. Alex introduced Seldon Core, Seldon’s open-source solution, which allows teams to construct complex inference graphs that enable multiple models to be deployed and run concurrently.
Next, Alex discussed the importance of monitoring at scale. He talked through the importance of measurements such as service metrics, statistical importance, and outlier detection, and addressed why each of them matters to delivering a robust monitoring solution and how organisations should approach these issues.
Alex then covered why explainability is so important for scaled ML deployment, so as to bolster trust in ML, deliver insights for data scientists, avoid damage to organisational reputations, and in meeting regulatory and ethical requirements. The lack of explainability and many models serving as impenetrable “black boxes” of decision-making is one of the biggest blockers to production ML, and so Alex talked through how teams can deploy explainability tools and the range of options on the table.
Finally, Alex talked through the issue of governance and complying with regulatory and ethical norms. Alex talked through regulatory developments such as the EU’s pending AI regulations, and the eight principles for trusted AI from the Linux Foundation’s AI & Data Group.
5G: The enabler of network-wide machine learning
Seldon’s Scott Bowler hosted a roundtable discussing how telcos can deploy ML models to optimise network performance while building out and monetising 5G networks.
5G potentially opens up many opportunities for telco, but it also comes with challenges such as a vast proliferation of devices and cell tower sites; thankfully, ML is perfectly suited to helping to optimise such burgeoning and complex networks. Scott and the roundtable explored how ML can help telco providers in the age of 5G, working to integrate equipment from a variety of vendors, and drive new revenue streams. The roundtable also discussed how use-cases for ML may change amid the infrastructural change presented by 5G.
What innovation is necessary for retail’s recovery in 2021?
Seldon’s Joel Hodgson joined this roundtable to discuss how organisations can improve their online shopping experience through ML deployment, the technical infrastructure that’s needed to optimise supply chains, and what retailers should know in order to navigate the compliance expectations that come with scaled ML.
With 2020 and 2021 both having been transformative years for retail, the mass move to online shopping has seen greater strain placed on the supply chains of retailers and also in online platforms being able to drive new leads. Joel and the roundtable reflected on the key opportunities they see with ML in 2021, challenges facing retailers with ML adoption, and how retailers can manage ML at scale.
How can financial service organisations gain value from ML while navigating increasing compliance expectations?
Finally, Alex Housley joined a roundtable to discuss how financial services organisations can benefit from ML amid growing compliance requirements. In an already highly regulated industry, financial services organisations face even greater challenges in meeting data and compliance expectations, at both the internal and external level.
As Alex covered in his earlier talks, the tide is shifting towards more stringent regulation of how banks use their data, with the industry facing inbound regulation such as the EU’s upcoming AI bill and requests for information regarding the use of AI and ML tools by US regulators. The roundtable explored their biggest challenges when it comes to compliance and model deployment, the techniques they find essential in ensuring compliance, and how AI principles are determined outside of regulatory requirements.