The levels in D&D represent adventuring levels, so your average person wouldn’t have any adventuring abilities unless they’ve done something to earn experience. And magic will be as rare as the deities in your setting want it to be since they can basically just give magic to anyone. Hell, some powerful beings even grant magic and powers to their subjects against their wills.