RNN

Word Embeddings

https://bobondemon.github.io/2017/09/07/Word-Embeddings-and-Encoder-Decoder-Neural-Net/

CBOW、Skip-Gram

https://zhuanlan.zhihu.com/p/27234078

LSTM

https://colah.github.io/posts/2015-08-Understanding-LSTMs/

ELMo

GRU

refer to https://blog.csdn.net/sinat_28015305/article/details/109539891

Transformer

Refer to the tutorials http://nlp.seas.harvard.edu/2018/04/03/attention.html, http://jalammar.github.io/illustrated-transformer/, https://blog.csdn.net/zhong_ddbb/article/details/107564421 for a detailed introduction.

OpenAI GPT

pre-training(unsupervised learning) + fine-tuning(supervised learning)

refer to https://zhuanlan.zhihu.com/p/59286975

GPT-3

few shot learning are invloved in GPT-3; for more about few shot learning refer to https://zhuanlan.zhihu.com/p/165882989, https://www.zmonster.me/2019/12/08/few-shot-learning.html

BERT

refer to https://unclestrong.github.io/DeepLearning_LHY21_Notes/Notes_html/18_BERT_P1.html, https://zhuanlan.zhihu.com/p/103226488

PanGu-α

ERNIE

https://github.com/PaddlePaddle/ERNIE

https://zhuanlan.zhihu.com/p/87008569

system

Google

Cloud NLP, Microsoft Azure Text Analytics, IBM Watson Nat

ural Language Understanding and Amazon AWS Comprehend

Datasets

reading comprehension

CMRC 2018

Chinese Machine Reading Comprehension

website: https://hfl-rc.com/cmrc2018/

github: https://github.com/ymcui/cmrc2018

Baseline: BERT

related: https://zhuanlan.zhihu.com/p/73831398

more

refer to https://zhuanlan.zhihu.com/p/70355773