What is the motivation for the definition of concurrence in quantum information? On the surface, the definition looks pretty ad hoc.
The definition is often given for the case of 2 qubits only. What is the generalization to higher dimensional spaces?
To the best of my knowledge, the initial motivation for defining the concurrence involves the fact that for pure bipartite entangled states, the reduced density matrices of each subsystem are not pure. For example, consider the Bell state $$|\Phi\rangle_{AB}=\frac{1}{\sqrt{2}}(|00\rangle+|11\rangle).$$ It satisfies $\rho_A=\text{Tr}_B(|\Phi\rangle_{AB}\langle\Phi|_{AB})=\mathbb{I}_A/2$, where $\mathbb{I}_A$ is the identity operator on $\mathcal{H}_A$. The reduced state $\rho_A$ is not pure and it purity is $\text{Tr}(\rho_A^2)=1/2$. This leads to the definition of concurrence for a pure state $|\psi\rangle$ as $$C(\psi)=\sqrt{2[1-\text{Tr}(\rho_A^2)]}.$$ For a mixed state $\rho$, one defines the concurrence as $$C(\rho)=\min_{|\psi_i\rangle}\sum_ip_iC(\psi_i)$$ where the minimization is over all possible decompositions $\rho=\sum_ip_i|\psi_i\rangle\langle\psi_i|$ of $\rho$. The closed form of the concurrence for two qubits $$C(\rho)=\max\{0,\lambda_1-\lambda_2-\lambda_3-\lambda_4\}$$ is just the solution to the minimization problem.
The motivation is as follows, the initial motivation for the definition is that concurrence can be used to calculate entanglement of formation. For pure bipartite state $|\Psi_{AB}\rangle$, we know that entanglement entropy $E_f(\psi_{AB})=S(\rho_A)=S(\rho_B)$ measures the entanglement between $A$ and $B$. And the development of quantum information tells us that this quantity coincides with the entanglement cost $E_C$ and distillable entanglement $E_C$. This quantity plays a crucial role in various entanglement-based quantum information protocols. The generalization to mixed state is straightforward $$E_f(\rho_{AB})=\inf_{\{p_i,\psi^i_{AB} \} } \sum_i p_i E_f(\psi^i_{AB})$$ where the inf is taken over all ensemble decomposition of $\rho_{AB}=\sum_i p_i |\psi^i_{AB}\rangle \langle \psi^i_{AB}|$.
It turns out that $E_f(\rho)$ can be calculated using concurrence $C(\rho)$: $$E_f(\rho)=H(\frac{1+\sqrt{1-C^2}}{2}).$$ The formula first appears in the famous paper by Bennett et al., Mixed State Entanglement and Quantum Error Correction. Later, it was carefully studied by Hill and Wootters, Entanglement of a Pair of Quantum Bits; And by Wootters, Entanglement of Formation of an Arbitrary State of Two Qubits.
To understand the physics behind the definition of concurrence, it may be helpful to consider the case for a pure two-qubit state case. By choosing the basis as four Bell states (which has maximal entanglement): \begin{aligned} &\left|e_{1}\right\rangle=\frac{1}{2}(|\uparrow \uparrow\rangle+|\downarrow \downarrow\rangle) \\ &\left|e_{2}\right\rangle=\frac{1}{2} i(|\uparrow \uparrow\rangle-|\downarrow \downarrow\rangle) \\ &\left|e_{3}\right\rangle=\frac{1}{2} i(|\uparrow \downarrow\rangle+|\downarrow \uparrow\rangle) \\ &\left|e_{4}\right\rangle=\frac{1}{2}(|\uparrow \downarrow\rangle-|\downarrow \uparrow\rangle) \end{aligned} A state can be expressed as $|\psi\rangle =\sum_i \alpha |e_i\rangle$. We can take complex conjugation over these basis, $|\psi^*\rangle =\sum_i \alpha^* |e_i\rangle$, the concurrence is simply the overlap between these two states $$C(\psi)=|\langle \psi^*|\psi\rangle|=|\sum_i \alpha_i^2|.$$ From this we can infer some physical intuition, that a state can be projected to four maximally entanglement basis state, the concurrence is to characterize the total overlap of the given state with these four maximally entangled basis states.
The generalization of the above definition to mixed state is straightforward $$C(\rho_{AB})=\inf_{\{p_i,\psi_{AB}^i\}}\sum_i p_i C(\psi_{AB}^i)$$ where the inf is taken over all ensemble decomposition of $\rho_{AB}=\sum_i p_i |\psi^i_{AB}\rangle \langle \psi^i_{AB}|$. If we transfer to the standard basis $|00\rangle,|01\rangle, |10\rangle, |11\rangle$, we will obtain the usually seen definition, $\tilde{\rho}=(\sigma_y\otimes \sigma_y )\rho^* (\sigma_y\otimes \sigma_y )$ (this corresponds to taking complex conjugation in Bell basisi), then we can calculate the eigenvalues of $\sqrt{\sqrt{\rho}\tilde{\rho}\sqrt{\rho}}$ with $\lambda_1\geq \lambda_2\geq \lambda_3\geq \lambda_4$, $$C(\rho)=\max\{\lambda_1-\lambda_2-\lambda_3-\lambda_4,0\}.$$ Two definitions are equivalent, but this later one looks very ad hoc.
The generalizations to the higher dimensions are the $\Theta$-currence and $I$-concurrence. The $\Theta$-concurrence is more similar to the original definition of concurrence, it involves an operation introduced by Uhlmann, called $\Theta$-conjugation, but the definition of $I$-concurrence is more different in spirit. See the review, measures and dynamics of entangled states, for more details.
The basic motivation behind defining "Concurrence" is its faithful quantification of quantum entanglement and hence also is useful as a strong separability criterion. "Faithful" means that Concurrence = 0 <=> Unentangled/Separable state, and if Concurrence is non-zero then it would "quantify" how much far these states are from achieving separability well. This has been extended to pure states of arbitrary dimensions (aka multi-qudit pure states) in a very intuitive way by looking at the geometry of the tensor products with increasing dimensionality. A pedagogic explanation to reach the generalization from the basic notion of separability may be found in our recent article here: arXiv:1607.00164 (2016).
