Questions

1. RoBERTa uses a byte-level byte-pair encoding tokenizer. (True/False)
2. A trained Hugging Face tokenizer produces merges.txt and vocab.json. (True/False)
3. RoBERTa does not use token type IDs. (True/False)
4. DistilBERT has 6 layers and 12 heads. (True/False)
5. A transformer model with 80 million parameters is enormous. (True/False)
6. We cannot train a tokenizer. (True/False)
7. A BERT-like model has 6 decoder layers. (True/False)
8. Masked language modeling predicts a word contained in a mask token in a sentence. (True/False)
9. A BERT-like model has no self-attention sub-layers. (True/False)
10. Data collators are helpful for backpropagation. (True/False)