Whether Matt is nice or not is not the point.
The point is, he associates with women who have unequal footing with him, be they druggies or babies, 1) so he can have some control and 2) because he is attracted to messes. And 16 year olds that will fuck 22 year olds are messes.