Sanaz Bahargam

Sanaz Bahargam

Machine Learning Engineer @Twitter

NAACL 2018, Summary of talks

less than 1 minute read

Published:

There was so much happening at NAACL; so many interesting works on all sorts of (old and new) NLP problems. Lots of papers focused on how to generalize the models beyond the conditions during training. In addition, there was workshop on “New Forms of Generalization in Deep Learning and Natural Language Processing”. In that workshop, Yejin Choi pointed out that natural language understanding (NLU) does not generalize to natural language generation (NLG). Another focus of the conference/workshops were on dialogue systems and chatbots. Lots of talks focused on using a knowledge graph in chatbots to have deeper conversations without staying on the topic for the whole conversations.

NAACL-2018

See here

You May Also Enjoy

Naacl2021

9 minute read

Published:

Here are the summary of some of talks from NAACL 2021. So far, I only summarized the following papers, I will be summarizing more. So, I will either append it here or a new blog post. Feel free to check back soon.

ICLR 2021

8 minute read

Published:

ICLR 2021

Edge Probing

3 minute read

Published:

In the past couple of years, Transformers has acheived state of art results in a variety of natural language tasks. In order to better understand Transformers and what they are learning in practice, researchers have done layer-wise analysis of Transformer’s hidden states to understand what the Transformer is learning in each layer. A wave of recent work has started to “prob” the state of the art Tranformers to inspect the structure of the network to assess whether there exist localizable regions associated with distinct types of linguistic decisions, both syntactic and semantic information. Researchers examine the hidden states between encoder layers directly and use those hidden states in a linear layer + softmax to predict what kind of information in encoded in each hidden state.

Transformers 2

5 minute read

Published:

This blog post is the continuation of my previous blog post, Transformers. In my previous blog post, I explained original Transformer paper, BERT, GPT, XLNet, RoBERTa, ALBERT, BART, and AMBER. In this blog post, I will explain MARGE, ConveRT, Generalization through Memorization, AdapterHub, and T5. Images and content used in this blogpost, otherwise mentioned, are all taken from the papers on each model.