期刊名称：Computational and Structural Biotechnology Journal
印刷版ISSN：2001-0370
出版年度：2021
卷号：19
页码：1750-1758
DOI：10.1016/j.csbj.2021.03.022
出版社：Computational and Structural Biotechnology Journal
摘要：Natural language processing (NLP) is a field of computer science concerned with automated text and language analysis. In recent years, following a series of breakthroughs in deep and machine learning, NLP methods have shown overwhelming progress. Here, we review the success, promise and pitfalls of applying NLP algorithms to the study of proteins. Proteins, which can be represented as strings of amino-acid letters, are a natural fit to many NLP methods. We explore the conceptual similarities and differences between proteins and language, and review a range of protein-related tasks amenable to machine learning. We present methods for encoding the information of proteins as text and analyzing it with NLP methods, reviewing classic concepts such as bag-of-words, k-mers/n-grams and text search, as well as modern techniques such as word embedding, contextualized embedding, deep learning and neural language models. In particular, we focus on recent innovations such as masked language modeling, self-supervised learning and attention-based models. Finally, we discuss trends and challenges in the intersection of NLP and protein research.
关键词：Natural language processing ; Deep learning ; Language models ; BERT ; Bag of words ; Tokenization ; Word embedding ; Contextualized embedding ; Transformer ; Artificial neural networks ; Word2vec ; Bioinformatics
