The Liberal Democrats overturned a Conservative majority of 19,213 in the by-election.