As far as I can tell, the people against market rate housing are really just using low-income housing as justification of their NIMBYism.

Kind of like how domestic espionage is justified with calls of “think of the children”. They try to make you seem heartless to oppose their position.