There isn't just "one" angle of refraction. The angle continues to shift as the wave travels through denser media.
Waves also do not travel in lines per say, they travel in arcs, like a pebble making ripples in a pond. The LINES that you see are RADII VECTORS of CIRCLES. But the radii of ever-changing circles.
This may make more sense: Imagine you put a bunch of floating sensors on the beach in a straight line, out into the ocean. As a wave passed the points where your sensors were, you'd be able to measure the waves speed.
This is does not mean that the ocean only waves where you put your sensors
We can tell wether or not we're listening to the same wave because they make the same sound, and follow the laws of sound conductance. Seismographs are basically ultra-sensitive listening devices, and every earthquake sounds way-defferent, like the difference between two people speaking.
Add in the, the speed of sound, and that things sound quiter the farther away you get from them, and match the graphs with a computer, and you have yourself a recognized wave :)
The angle of refraction is calculated using physics. However, since the media is constantly changing from dense to more dense, the way only way you can calculate the defraction angle is by using calculus