I think if you introduced another layer of complexity with the sun oscillating in the r-direction (closer to the arctic during the north hemisphere summer and closer to the antarctic during the south hemisphere summer) it would make physical sense in that regard

of course it’d still make no sense in several other aspects, like the fact that we have discrete sunsets/rises instead of the sun just gradually fading into darkness, and also the fact that the sun’s path is an arc (these are actually the same point kinda)