Ed's Blog
  • About
  • The Termination Critic

    TLDR: This paper induces learned options to terminate in small distribution of states. This makes it easier to transition and plan.

    February 28, 2019 - 2 minute read -
    Paper Summary Hierarchical Reinforcement Learning Options Critic
  • Near-Optimal Representation Learning for Hierarchical Reinforcement Learning

    This paper presents a theoretically-backed representation objective for HRL that extends Data Efficient HRL by Nachum et al and outperforms other HRL methods. I liked the formulation and theoretical analysis of the formulation to derive the objective. The representation objective suggests a connection with maximizing mutual information of future state given current policy and state, which is task reward agnostic and seems intuitive. The experiments are simple AntWalker experiments but are in line with previous HRL baselines, which they compare against.

    February 10, 2019 - 2 minute read -
    Paper Summary Representation Learning Hierarchical Reinforcement Learning
  • Relational Forward Models for Multi-Agent Learning

    This paper focuses on using a graph representation for a multi-agent trajectory prediction setting. @dmillard may be interested in this for trajectory prediction.

    December 14, 2018 - 1 minute read -
    Paper Summary Machine Learning Graph Networks
  • How to Train Your MAML

    This paper presents empirical problems of MAML and proposes some fixes to address this. Overall, I did not find this paper to be too novel in its solutions but the proposed fixes are simple and seem effective. When I was training MetaGAN, I would run into these problems a lot. A good read for anyone wanting to apply MAML.

    November 24, 2018 - 2 minute read -
    Paper Summary Machine Learning Meta Learning
  • Hybrid Reward Architecture

    This paper focuses on improving RL algorithms by decomposing the scalar reward function into a composition of multiple reward functions. The key idea is that challenging domains’ reward functions cannot be easily reduced into a low dimensional representation so decomposition can alleviate this.

    October 19, 2018 - 3 minute read -
    Paper Summary Machine Learning Reinforcement Learning
  • Hierarchical Reinforcement Learning: Options-Critic and SECTAR

    Sectar jointly trains a state decoder and policy decoder

    September 7, 2018 - 3 minute read -
    Paper Summary Machine Learning Reinforcement Learning Options Critic SECTAR
  • Gradient of a Matrix Matrix multiplication

    This is just matrix multiplication.

    July 28, 2018 - 3 minute read -
    Machine Learning Backpropagation Matrix Calculus
  • Building an Image Editor with Canvas API

    Zoom Pan

    July 6, 2018 - 6 minute read -
    web angular hammerjs
  • Curiosity driven exploration by Self-supervised Prediction (Paper Summary)

    May 31, 2018 - 4 minute read -
    Paper Summary Machine Learning Intrinsic Reward Sparse Reward Exploration
  • Breaking Jazz Tracks into Beats with "Unsupervised" Clustering

    What we want! Each phrase is a different color.

    Spoiler: I end up just writing one while loop.

    March 7, 2018 - 5 minute read -
    Machine Learning Music Unsupervised Clustering
  • Deep Symphony - Classical Music Generation with Neural Networks

    January 10, 2018 - 101 minute read -
    Machine Learning Music Deep Learning GAN LSTM

My thoughts on machine learning, code, and math.