You should try it yourself. Get a calculator (your computer should have one) which can do square roots, and try to compare, say, sqrt(1.002) with the value 1+ 0.002/2 .
The two get closer the smaller x is; it won't work as well for sqrt(1+1) and 1+(1/2) because then x=1 is "too large" for this approximation to work.
What's the relevance of this? You can use it to see things such as, for example, why you write the famous Einstein's equation as http://mathurl.com/6y5ol8m.png (from which you get E=mc^2 which p=0), and when you write it as a square root around x = p/mc^2 you get that term for the kinetic energy that you learn from high school, p^2 / 2m (or the more familiar form mv^2 / 2) ... It's not "magic" -- it comes from calculus.
The other subject you should also learn is called Linear Algebra.
Basically it has to do with matrices and vectors. You learn that a "vector" is really just an ordered sequence of numbers (e.g. (1, 2.6, -2, 5.51) ). And matrices are just a way to transform a vector from one representation to another. For example, you can describe the rotation of a vector in 3-D space as a matrix involving terms which are the angles of rotation. Or you learn which systems of equations are solvable, and how to solve linear systems of equations -- you write them as matrices and you "look" at the terms of the matrix. It's "another way of doing it", and it borrows itself pretty well for computation (that's how symbolic algebra systems work out systems of equations). You learn also that a four-dimensional space is no big deal -- it's just a 3-D space with an extra component. It could be a 5-D space just as easily -- it's more work, but it's no conceptual jump in the void.
So, as eri explained:
You should also master calculus (integral calculus, differential equations, limits of functions, sums, etc.), linear algebra, geometry, etc. Oh and probability and statistics too; you may not need it right away if you go the special/general relativity way, but when you get to cosmology you'll be needing statistics (particularly thermodynamics) besides knowing mechanics (motion). But the calculus+linear algebra is kinda like the "boot camp" of modern physics. Sure, you can grab a pointy stick and run into the battle field -- but it'll get scary very fast. It's not impossible, but it gets harder.