While investors have slowed their purchasing, critics are still concerned they're beating out smaller, individual homebuyers.