Relational Forward Models for Multi-Agent Learning

December 14, 2018 - 1 minute read -
Paper Summary Machine Learning Graph Networks

This paper focuses on using a graph representation for a multi-agent trajectory prediction setting. @dmillard may be interested in this for trajectory prediction.

The input is a social graph of multiple agents, and the tasks require group coordination and knowledge sharing to complete. The output is the actions of each agent.

For example, the game Stag Hunt has 4 agents navigate an arena with 3 red stags and 12 green apples. Agents can either collect Apples for +5 reward by stepping on the apple. Alternatively, 2 agents can capture a stag for +10 reward but this requires two agents to step on the stag.

Therefore, it is useful for agents to predict what other agents will do to act accordingly. The graph network represents agents and objects in the environment has nodes. The node attributes are position, node type, capture status, and what their last action was.

They connect edges between agents and objects, as well as agents and agents. Then they do the standard graph neural network updates on edges, vertices, and attributes, and the respective aggregation functions as well.

They test this architecture on 3 different experiments, Cooperative Navigation, Coin Toss, and Stag Hunt. They show that their model outperforms baselines. They also note that LSTMs did not perform well. This suggests most of the predictive power comes from the relationship between objects instead of the state of the objects themselves.