I noticed my follower count keeps dropping, but I cannot figure out who unfollows me. Is there a simple way to check this without going through the whole list?

I had the same problem and it was very tiring to check everything manually. After some time I found a tool that helps to discover unfollowers in a clear way. It shows who has stopped following, so you do not need to guess or compare numbers all the time. I find it useful because it gives a simple overview and makes following changes easier to understand