Banks are tapping the brakes on jumbo mortgages, which they have typically viewed as a low-risk way to attract wealthy customers.