Weapons of Math Destruction -- Take 2

I finished Weapons of Math Destruction a while ago and didn't realize that I hadn't written about it.  Unfortunately, due to the delay and my terrible memory, the finer points are a bit fuzzy.  But my general thought follows:

This book was horrifying.

First, let's start with her definition of a model which becomes "weapon of math destruction." She describes three components:
1. Opacity - is it clear to an outsider what goes into the statistical model, or is it opaque?  Or, worse yet, is it completely invisible to the people being modeled?  Are people made aware of the results or "scores" they get?
2. Scale - how large of an impact does this model have?  Is its use widespread?  Is it easily scalable to cover ever-larger groups of people?
3. Damage - does the model (at least have the potential to) negatively impact the lives of the people it's modeling?

Statistical models aren't inherently, or at least don't have to be, destructive.  Not every statistical model becomes a WMD.  She uses the now-well-known idea promoted by the book and movie Moneyball as an example of when a model works well and is not destructive.

But many statistical models -- even if they didn't start out that way -- do eventually become WMDs.  They are opaque, widespread, and destructive.  The author illustrates many of them in her book; there are chapters on the U.S. News & World Report college rankings, online advertising, applying for jobs, getting credit or insurance, and others.

Though she does attempt a hopeful ending, I just don't have faith that we collectively will get to a place anytime soon where we are prioritizing people (and their privacy) over profits.  That seems unlikely.  Statistically.