Check out the post I co-authored with Chris McCormick on BERT Word Embeddings here. We take an in-depth look at the word embeddings produced by BERT, show you how to create your own in a Google Colab notebook, and tips on how to implement and use these embeddings in your production pipeline.

This was created when BERT was pretty new and exciting, but the tooling for it was quite bad. Huggingface hosted the model, but documentation was very poor. As a result of that, we decided to create an in-depth tutorial.

It was one of the very first comprehensive BERT tutorials and became incredibly popular, so I’m proud to have helped introduce language models to so many others.