Reading List for Learning Natural Language Processing (NLP) with Python

A former colleague recently reached out and asked for advice on how best to learn natural language processing (NLP) from scratch.

"EXCELLENT!" I thought; I had already answered the exact same question for another former colleague so it should have been as simple as dusting off the previous reply, right?

Well, it could have been except for one tiny detail: I have absolutely no idea where I saved the list of resources.

To that end, to avoid violating the DRY principle (i.e., "Don't Repeat Yourself ") yet again, I wrote this post so that if/when I get another request, I can just forward them the link.

What follows is my highly opinionated take. Feel free to disagree and leave links to your recommended curriculum.

Disclaimer #1

I'm assuming you have an understanding of linear algebra, statistics, and linguistics.

If not, stop. Just stop. Why? Three reasons:

1. To parse sentences, identify parts of speech, and any number of other tasks, you need to have a solid understanding of linguistics so consider taking a course such as this one which will give you an overview.
2. Statistical significance, correlations, distributions, and probabilities are essential concepts for machine learning. To fill this gap pick yourself a good book on stats such as Practical Statistics for Data Science and work your way through it.
3. As for linear algebra, as the articles below deftly explain, it is fundamental for deep learning and for computing the similarity of texts:

To that end, do yourself a favor and check out this amazing series of videos from 3Brown1Blue

Disclaimer #2

The path outlined below is the one I wish I would have taken.

"What path did you take?"

Well, in a nutshell, I spent years (yep, years) trying to figure out how to use coding as a tool to do the projects I wanted.

Sounds fine, right?

Well, the problem was I didn't learn the fundamentals of Python so when my code threw an error or, God forbid, I tried to repurpose someone else's code from StackOverflow and/or Github and it didn't run right out of the box, I was lost as to what to do.

Why?

Simply put, I wasn't proficient at Python and so, I was unable to adapt to unfamiliar situations.

To that end, if you are looking to learn NLP specifically or machine learning in general the very first thing you need to do is learn to code to the intermediate level.

Why the intermediate level?

Because nearly every Python for NLP resource you come across will recommend that you are "familiar", "have an understanding", "are proficient" or some other incredibly vague criteria for getting the most from their product or platform.

In a nutshell, the better you are at coding the easier it will be for you to put into practice the concepts you learn along the journey.

So how do I reach "intermediate level" Python?

Thank you for asking 😄

Python Basics: A Practical Introduction to Python 3

This book by the team at Real Python is solid gold with the first 10 chapters covering everything from installing Python on your machine, running scripts (not notebooks), data types, functions, debugging, and object-oriented programming (OOP) by defining classes.

As for the second half of the book, it is a collection of coding problems like scraping and parsing text from websites (chapter 16) and creating a GUI for your program (chapter 18).

Consequently, I recommend working your way through each and every page of the first ten chapters and doing any of the remaining chapters if they strike your fancy.

You can pick up a copy on their website.

Python for Data Analysis

When you're munging data, you can NEVER be too proficient at Numpy or Pandas so every moment you spend working your way through Wes Mckinney's insanely accessible book should be seen as a favor to your future self.

Again, do yourself a favor and use chapter three to review the built-in data structures in Python before doing deep dives on NumPy and pandas while getting a good understanding of matplotlib. However, the most useful section of this book is chapter 14 which contains five worked examples.

Be sure to visit the book's repo on GitHub to leave a star, see the notebooks for each chapter, and find where to pick up a copy.

Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow

Now that your Python is up to scratch and you can manipulate data, it's time to learn about machine learning and there is no better book for learning the fundamentals of machine learning than this gem by Aurélien Geron. While the book covers everything from logistic regression to neural networks (including a discussion on attention for NLP) the real strength is the author's discussion of why different techniques are used in addition to simply explaining how they are done. As such, Chapter 2's End-to-End Machine Learning project, as well as Appendix B's Machine Learning Project Checklist, are worth the cost of the book.

To learn more about this text, as well as to run the accompanying notebooks without downloading or installing anything, visit the author's repo.

Practical Natural Language Processing

The first out-and-out NLP book I would read is this gem. What makes it so good? This book takes a task-based approach in that it presents an issue and then asks how it can be solved using NLP.

Additionally, it presents applications in disparate fields such as social media, finance, and law. Additionally, nearly every issue is solved with multiple approaches so the reader is introduced to major NLP libraries such as Spacy, NLTK, Textblob, Gensim, and more.

Visit the book's website to learn more.

Blueprints for Text Analytics Using Python

Now that you have a grasp of NLP, let's expand on what you can do with it. One way of learning about what is possible is by looking at how people have used NLP to find solutions to business problems.

And that is exactly the role this book serves; it contains ~ 100 common use cases with blueprints.

But what are 'blueprints?"

The authors state that a "blueprint […] is a best-practice solution for a common problem." In essence, they are templates that, since you are proficient in Python, you can minimally adapt to fit your specific use case.

Be sure to head over to the book's repo to leave a star and see worked examples.

Transformers for Natural Language Processing

Finally, now that you have a solid Python base along with an understanding of what problems you can solve with NLP and how to solve them, it's time to learn about the state-of-the-art architecture: transformers.

What are transformers?

Sorry, but that's beyond the scope of this article; however, if you really want to know, I recommend reading the seminal work Attention is All You Need and/or The Illustrated Transformer to get the gist of the idea.

Once you have that, be sure to check out Denis Rothman's page-turning text which covers topics like fine-tuning and pretraining models along with use cases. To learn more about this text, click here.

Books to Fill in the Gaps

While I consider the texts above to be core, you can never know too much about how your tools work.

As such, the following books provide deep dives into some useful tools.

Mastering spaCy by Duygu Altınok should be your go-to guide for learning all about spaCy.

Work your way through Approaching (Almost) Any Machine Learning Problem by Abhishek Thakur which is similar to Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow but uses PyTorch.

Getting Started with Google BERT by Sudharsan Ravichandiran is an excellent text for wrapping your head around the BERT architecture.

Conclusion

While I'm a huge fan of all the books listed above, I have no doubt that I've omitted some which should be included.

To that end, please leave a comment with the book(s) you'd recommend for someone who wants to start their journey into NLP in the new year.

Happy Coding!