In macroscopic thermodynamics, the value of internal energy $U$, or entropy $S$ or other derived quantities (the Gibbs energy $G$, the free energy $F$) is often thought of as depending on the arbitrary choice of the reference state where that particular quantity has some prescribed value, such as zero. Which state is chosen to be such reference state does not matter. This arbitrariness poses no real problem as all experimentally relevant results of the macroscopic theory can be formulated in terms of changes $\Delta U,\Delta S, \Delta F$, etc.
Chemical potential $\mu$ for a simple system obeys the relation
$$
dU = TdS -pdV+\mu dN,
$$
thus we can use as its definition
$$
\mu =\frac{\partial U(S,V,N)}{\partial N}.
$$
Since
$$
dF = -SdT -pdV+\mu dN,
$$
it also obeys the relation
$$
\mu =\frac{\partial F(T,V,N)}{\partial N}.
$$
It is an intensive quantity like absolute temperature $T$ and pressure $p$, and just like with those, the reference state where its value is zero is arbitrary, because values of $U$ and $F$ are arbitrary; and this arbitrariness is also in dependence on $N$, so it allows for different values of $\mu$.
However, by convention $T$ is absolute temperature, so no longer arbitrary, $p$ is absolute pressure, so no longer arbitrary. With $\mu$ it is less clear, since it depends on how we define energy $U$ and entropy $S$ (and these are additional conventions). If we change definition of $U$ to $U+U_0$ (say, we want to take into account rest energy $mc^2$) $\mu$ will have different value than if we do not; similarly with $S$.
In quantum statistical physics which Kittel teaches in his book, we analyze some model of the thermodynamic system, and it is usual to adopt a specific definition of energy based on eigenvalues of a specific Hamiltonian operator and of entropy in terms of some definite number of quantum states, or in terms of partition sum over all quantum states. This leads to definite energy and entropy values for any microstate. In case of energy, the macrostate which implies quantum states with the smallest possible eigenvalues of the standard Hamiltonian $\sum_k \frac{p_k^2}{2m} +V$ (where $V$ is the standard pair-wise potential energy function, with no rest energy terms) is used as the one with zero or the smallest possible value of internal energy; and in case of entropy, the macrostate which has zero or the smallest possible number of allowed microstates (in simple cases, the ground state) is used as the reference state with zero or the lowest entropy. Thus the quantum models and calculations adopt specific definitions of energy and entropy, which remove this arbitrariness from the resulting thermodynamic quantities. But this is because we've chosen definite definition of energy out of infinity of the possible ones (the number of equivalent Hamiltonians is infinite, all differing by a constant shift), and we've chosen a specific definition of entropy, such as (microcanonical definition) entropy being log of number of states with energy lower than $U$, or (canonical definition) entropy being a specific derivative of the partition function, even though there is no real need to adopt such definition, except history and convenience.
This is mostly done for convenience, to simplify the formulae, verbal descriptions and the whole exposition. There is no real necessity to prefer one Hamiltonian to base definition of energy on and there is no real necessity to select one specific definition of entropy. It is possible to formulate the quantum statistics allowing for the arbitrariness in definition of energy and entropy and then it becomes clear the values of these thermodynamic quantities are arbitrary, depending on our conventions.
