Let's assume we talk about the 3D case.
A rotation matrix is an ortho-normal 3x3 matrix. R.' * R is the unity matrix, or in other words: rotating in one direction at first and then backwards in the opposite direction does not change the data.
A general rotation in 3D is specified by a unit vector (called u here) to rotate around and the angle of rotation (called alpha). Then we get this general rotation matrix (see FEX: RotationMatrix): s = sin(alpha);
c = cos(alpha);
x = u(1);
y = u(2);
z = u(3);
mc = 1 - c;
R = [c + x * x * mc, x * y * mc - z * s, x * z * mc + y * s; ...
x * y * mc + z * s, c + y * y * mc, y * z * mc - x * s; ...
x * z * mc - y * s, y * z * mc + x * s, c + z * z .* mc];
There are different conventions depending on the point of view: Do you rotate the coordinate system or the object. So maybe R.' is matching in your case.
This is called "direction cosine matrix" also, because it contains the projections of a vector into the coordinate system. You can interpret this matrix as rotated coordinate system also and its 3 vectors (rows or columns) build an orthonormal tripod.
While this is the general matrix, which performs the operation of rotating around a vector, there are a number of different methods, to split this matrix into 3 different rotations around axes fixed to the coordinate system or to the body. The Euler angles or Euler-Cardan (also called Tait–Bryan) angles: The matrix R is split into 3 matrices of the style:
R1 = [ c1, -s1, 0;
s1, c1, 0;
0, 0, 1];
R2 = [ c2, 0, s2;
0, 1, 0;
-s2, 0, c2];
R3 = [ 1, 0, 0;
0, c3, -s3;
0, s3, c3];
Here "s1" means: sin(alpha1) etc.
Now you can decide for a certain convention, e.g. R3*R2*R1, or R3*R2*R3 (yes, the index can be repeated), etc. The angles alpha1, alpha2, alpha3 are called the Euler angles. Their definition requires a decision for a specific order. Different fields of science use typical conventions for order, e.g. analysis of human motion and aerospace engineering.
if you have the two vectors a and b and want to get the rotation matrix needed to transform one into the other, you can follow these steps:
- The axis of rotation is the normalized cross-product:
u = cross(a, b) / norm(cross(a, b))
- The rotational angle is defined by:
alpha = atan2(norm(cross(a, b)), dot(a, b))
- Use the above formula to create R.
An equivalent solution:
na = a / norm(a);
nb = b / norm(b);
v = cross(na, nb);
skew = [0, -v(3), v(2); v(3), 0, -v(1); -v(2), v(1), 0];
R = eye(3) + skew + skew ^ 2 * (1 - dot(na, nb)) / (norm(v))^2;