Scalable machine learning with Apache Ignite, Python, and Julia: from prototype to production

Abstract

Apache Ignite incorporates a scalable, efficient machine learning (ML) framework that enables data transformation, ML model training, and inference on Apache Ignite nodes (without data leaving the cluster). However, sometimes scalability in regard to data size and data-transfer cost is not as important as scalability in regard to the number of parallel requests and the ability to use the external, more sophisticated models that are implemented in Python and Julia. In this talk, Peter Gagarinov shares his experience with integrating Apache Ignite with external machine frameworks. Using an ML-based, automated issue-management system (Alliedium) as an example, Peter shows how the business task of building a distributed and scalable service can be efficiently performed by relying on Apache Ignite and ML frameworks from Python and Julia ecosystems. In the second part of his talk, Peter presents a lightweight Apache Ignite data migration tool that was developed in-house for the needs of Alliedium: https://github.com/Alliedium/ignite-migration-tool

Date
May 25, 2021
Location
Virtual Event
Peter Gagarinov
Peter Gagarinov
Software Engineering Architect and Consultant

Software engineering architect and an open source enthusiast with a focus on machine learning, trading system, applied financial and mathematical modelling, distributed systems and technical computing.

Related