I am going to answer questions (2) and (3), and leave question (1) as exercise, since you'll have everything you need to construct the 8x8 matrix on your own.
- Or can you tell me another way how to derive this state?
The controlled-$R_2$ gate obviously does not affect, or depend on, qubits $3$, $4$, ..., $n$, so let's look just at qubits $1$ and $2$.
The result of a Hadamard gate $H_1$ can be written, for $j_1 = 0,1$,
$$
H_1|j_1 j_2\rangle = \frac{1}{\sqrt{2}}\left( |0_1\rangle + e^{i\pi j_1}|1_1\rangle \right)|j_2\rangle = \frac{1}{\sqrt{2}}\left( |0_1\rangle + e^{2\pi i \frac{j_1}{2}}|1_1\rangle \right)|j_2\rangle
$$
The action of the controlled-$R_2$ on the computational basis states reads
$$
CR_2|0_1 0_2\rangle = |0_1 0_2\rangle \\
CR_2|1_1 0_2\rangle = |1_1 0_2\rangle \\
CR_2|0_1 1_2\rangle = |0_1 1_2\rangle \\
CR_2|1_1 1_2\rangle = e^{\frac{2\pi i}{2^2}}|1_1 1_2\rangle
$$
or, in $j_1$, $j_2$ notation,
$$
CR_2|j_1 j_2\rangle = e^{2\pi i\frac{j_1 j_2}{2^2}}|j_1 j_2\rangle
$$
Applying the controlled-$R_2$ to the result of the Hadamard gate gives
$$
CR_2 \frac{1}{\sqrt{2}}\left( |0_1\rangle + e^{2\pi i \frac{j_1}{2}}|1_1\rangle \right)|j_2\rangle = \frac{1}{\sqrt{2}}\left(e^{2\pi i\frac{0 \cdot j_2}{2^2}}|0_1 j_2\rangle + e^{2\pi i\frac{1 \cdot j_2}{2^2}}e^{2\pi i \frac{j_1}{2}}|1_1 j_2\rangle \right) =\\
= \frac{1}{\sqrt{2}}\left(|0_1 j_2\rangle + e^{2\pi i\left( \frac{j_1}{2} + \frac{j_2}{2^2}\right)}|1_1 j_2\rangle \right) = \frac{1}{\sqrt{2}}\left(|0_1\rangle + e^{2\pi i 0.j_1 j_2}|1_1\rangle \right) |j_2\rangle
$$
where $0.j_1 j_2 = \frac{j_1}{2} + \frac{j_2}{2^2}$ uses binary notation in analogy to the decimal notation $0.n_1 n_2 ... = \frac{n_1}{10} + \frac{n_2}{10^2} + ...$. If we bring in the other qubit states, we retrieve the final result.
- 0.j1j2 is a binary fraction. But the matrix representation of Rk is in decimal. So in the matrix representation of controlled-Rk in the basis of n qubits, should it be written in binary representation or decimal representation?
See above. The $0.j_1j_2...j_k$ notation for $\frac{j_1}{2} + \frac{j_2}{2^2} + ... + \frac{j_k}{2^k}$ gives a convenient and beautiful number-theory meaning to an otherwise cumbersome sum of fractions: it is a fractional number in binary representation. The "decimal" point is in this case a "binary" point, indicating that the powers of $2$ associated with each $j_i$ in the sequence are negative, $2^{-i}$.