Although technically true, it is also misleading. You are probably describing just the tip of the power curve here.
In my younger years, I was a die-hard fan of the most popular band in our country. The band went on a tour and they could easily fill up every venue with thousands of people. Yet there were only a handful of us who were following them closely, from concert to concert, collecting autographed merch, etc. The members of the band knew us by our names.
Yet, the survival of the band and the success of their tour wasn't up to us alone. They needed a mass of those who were just casual fans, an even larger mass of those who just like a song or two, and a smaller crowd who come to the concert because of the hype and booze. Yet all of them bought the same ticket, didn't they?
I see it is the same here. All of your 20k readers will check up on you from time to time, albeit not all of them with equal fervour, and certainly not equally often. But you need those too, right?