In general, kets represent states of maximal information. Without more context, it's a completely abstract notion.
For example, suppose that you have a system with just three possible mutually exclusive measurement outcomes. Let $P_k$ be the proposition "the outcome will be $k$" ($k=1,2,3$), and consider building other propositions from them through the logical operators or and and. You will quickly see that this forms a Boolean algebra that's just like the algebra of projection operators to linear subspaces spanned by subsets of some set of mutually orthogonal vectors $\{\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3\}$ in any inner product space, e.g.,
$$P_1\,\mathrm{or}\,P_2 \longleftrightarrow \Pi_{12} = \Pi_{\mathrm{span}\{\mathbf{e}_1,\mathbf{e}_2\}}\text{,}$$
with the logical and represented by straight multiplication.
Because each $\Pi_k$ is a projection operator a one-dimensional subspace, it can be written as an outer product of a vector with itself: $\Pi_k = |k\rangle\langle k|$. Given any discrete probability distribution over our three mutually exclusive example outcomes, with probability $p_k$ for outcome $k$, we can represent it with the operator
$$\hat{\rho} = \sum_k p_k|k\rangle\langle k|\text{,}$$
which we'll call a (mixed) state (also density matrix).
If a state is in the form $\hat\rho = |\psi\rangle\langle\psi|$, and therefore is itself a projection to a one-dimensional subspace, then we call it a pure state. In that case, we can just take $|\psi\rangle$ by itself without loss of generality.
Slightly less trivial example: suppose we have a particle in one dimension, and all we're interested in is its position. Then our mutually exclusive outcomes are $\Pi_x = |x\rangle\langle x|$, and in terms of kets, mutual orthogonality means $\langle x|x'\rangle = \delta(x-x')$.
But that the amplitude is more fundamental for some reason.
Morally speaking, the reason for probability amplitudes is very simple: since probability distributions in general are represented by density matrices, and in a special case (pure state) they can be written as vector "squared" (outer product with itself), so of course when working in a basis on that vector, you'll have to square the components.
I wouldn't call it "fundamental". Probability amplitude is a component of a pure state written in some particular basis, e.g., $\psi(x) \equiv \langle x|\psi\rangle$ is the probability amplitude of outcome $x$. I'd rather consider the states as more fundamental than any particular basis representation.
Probability amplitudes are general feature of representing outcomes by linear subspaces and then considering probability distributions over them. What's special about quantum probability specifically is that your observables can involve projection operators to any linear subspace you like, so that superposition of orthogonal states like ${\hat\rho}_1 = |\psi\rangle\langle\psi|$ with $|\psi\rangle = \alpha|1\rangle + \beta|2\rangle$ is measurably different from ${\hat\rho}_2 = |\alpha|^2|1\rangle\langle 1| + |\beta|^2|2\rangle\langle 2|$, whereas classically, they are indistinguishable.