PROWLER.io's AI Platform

engineering generalised decision-making

back to our blogs

PROWLER.io's AI Platform

engineering generalised decision-making

Neil Ferguson

By Neil Ferguson (Software Engineering Team Lead)

At PROWLER.io, we’ve set our sights firmly on using AI to solve real-world problems. That means having a platform that ensures a principled & multi-faceted approach to decision-making. With its comprehensive set of AI tools and techniques, our platform makes lots of quick, effective decisions in high-dimensional, dynamic systems like financial markets, smart cities, ridesharing, massively multiplayer online games and logistics.

While other AI companies choose to focus on perception, we want machines to learn to do more than merely distinguish between different types of data. To this end, we have developed a software platform that specifically focuses on making intelligent decisions under uncertainty. Our platform has two central components: a decision-making system that is responsible for choosing the correct action to take in different situations, and a learning system that continuously learns new predictive models and policies in order to direct those actions.

PPROWLER.io's AI Platform Architecture

PROWLER.io's platform architecture.

These two components can be deployed on different machines or even in different data centres. For example, if a customer’s priority is to get decisions in the shortest time possible, we can deploy the decision-making system in the customer’s data centre and host fully-managed learning on PROWLER.io's hardware in a separate data centre. When a customer has less strict latency requirements, we can offer a fully-managed service, where both decision-making and learning take place in our own data centre.

 

How we make decisions

To teach machines how to make intelligent decisions, we have taken a three-pronged approach.

The platform we have developed has a well-tested set of baseline algorithms for Reinforcement Learning (RL). It also has a modular set of components (or ‘building blocks’) which can be used to construct completely new RL algorithms that learn from experience using our Policy Learner. Our Tuneable AI tool, for example, was developed by using the platform’s building blocks to extend some state-of-the-art RL algorithms. We were then able to compare it against our benchmark algorithms.

In order to make more accurate, data-efficient decisions, the platform can be used to build probabilistic models from multiple sources of environment data using our Model Learner. The platform then incorporates these probabilistic models into its decision-making process. In our work on supply and demand, for instance, we build models that predict demand for resources in a city, in order to make better resource allocation decisions.

We believe that the best decisions often come out of Multi-Agent Systems (MAS). Therefore we have chosen to design our platform to support many decision-making agents, by default. In Decentralised Learning in Systems with Many, Many Strategic Agents, we describe how we simulate thousands of agents on the platform, interacting with each other and learning continuously.

 

A single platform

PROWLER.io is a research-led company, so it is important for us to ensure that research and production are well aligned. Since much of our cutting-edge research is destined to be applied in real-world complex systems, we use exactly the same core platform for powering research experiments as we do for solving our customers’ problems.

Building a single platform to meet the different requirements of research and production is one of PROWLER.io’s major challenges. But it allows us to use the same high-quality software components for research experiments that we use for customer deployment.

 

For researchers

This means researchers from our different research groups can efficiently collaborate with each other, reproduce previous results and develop new techniques by building quickly on previous work from other researchers using common, well-tested software.

We believe proper experiment automation is key to useful, efficient machine learning research. Therefore we provide different ways for researchers to scale their experiments: from fully asynchronous learning across multiple processes to large-scale hyper-parameter searches.

We know it's also vital for our researchers to be able to analyse the results of an experiment easily. Our platform enables them to monitor what's happening in their experiments in real-time, and also to perform post-hoc data analysis.

Finally, since we must be able to reproduce any experiment, our tools record the specific conditions necessary for accurate reproduction.

 

For customers

These features also prove vital in customers’ production environments. The platform can distribute customers’ workloads across many CPUs and GPUs. It can also elastically scale to meet changes in demand from customers’ systems, scaling to many thousands of decisions per second.

In production environments, high availability and fault tolerance are essential. The platform supports decision-making and learning that will continue to operate during software and hardware failures, enabling customers’ systems to carry on working without interruption.

Our customers’ systems interact with the platform via a flexible Environment API. For example, a massively multiplayer online game server could continuously send information about what is happening in the game’s world to our platform via its Environment API, and ask us to make decisions about what actions non-player characters should take. Similarly, a taxi company could update our platform with details of the whereabouts of all its taxis and ask us to make decisions about the best routes its drivers should take.

 

Looking to the future

We believe that the PROWLER.io platform will enable machines to make more intelligent decisions quicker than ever before, and accelerate the time it takes to push insights gained from cutting-edge research into production. In creating a platform that solves real-world problems by drawing on the ability of machines to make complex decisions very rapidly, we have opened up new opportunities for both research and business to capitalise on the potential of AI.