There are several definitions of entropy in physics, based on the work of Claussius, Gibbs, Boltzmann, and others. Of particular interest here is Boltzmann's definition, which relates entropy to (the log of) the number of equivalent "microstates" for a given system state. Boltzmann used the equation $S = k_B\ln \Omega$, where $\Omega$ is the number of possible microstates. in the system. Gibbs developed this further to come up with the equation $S = -k_B\sum_i p_i \ln(p_i)$, where $p_i$ is the probability that the system will be in microstate $i$.

In computer science, Claude Shannon was working in the 1940's on on quantifying the amount of "information" available in symbols in a sequence. He eventually worked out that for a random variable $X$, the amount of information received from that random variable was given by the formula $H(X) = -\sum_{x\in X}p(x)\log p(x)$, where $p(x)$ is the probability that the random variable will give $x$.

The two formulas I gave above are: $S = -k_B\sum_i p_i\ln p_i$ and $H(X) = -\sum_x p(x)\log p(x)$. With the exception of the constant multiplier and the choice of logarithmic base, both these formulas are essentially the same.

Both Gibbs Entropy and "Shannon Entropy" are similarly defined, and serve similar purposes in their respective fields. The smaller the entropy of a system, the easier it is to predict what microstate it is in, regardless of if you are talking about a box of gas or a stream of text over a wire.

Because entropy is a measure (in both statistical dynamics and information science) of the (theoretical) unpredictability of a system, it gets talked about in the context of random number generators because you want random number generators to be unpredictable, to have "high entropy". A pseudo-random number generator will have internal variables that can have a collectively high number of states. Generating a random number both gives you the number, but also modifies those internal variables in a deterministic way. But only a fraction of the possible states would give you the random number you generated, so there are only a fraction of the total number of states that the RNG could be in after generating that number. The entropy of the RNG has reduced. In theory, after extracting enough random variables, the RNG becomes completely predictable; there is only one state it could possibly be in. It has no entropy.

There are several definitions of entropy in physics, based on the work of Claussius, Gibbs, Boltzmann, and others. Of particular interest here is Boltzmann's definition, which relates entropy to (the log of) the number of equivalent "microstates" for a given system state. Boltzmann used the equation $S = k_B\ln \Omega$, where $\Omega$ is the number of possible microstates. in the system. Gibbs developed this further to come up with the equation $S = -k_B\sum_i p_i \ln(p_i)$, where $p_i$ is the probability that the system will be in microstate $i$.

In computer science, Claude Shannon was working in the 1940's on on quantifying the amount of "information" available in symbols in a sequence. He eventually worked out that for a random variable $X$, the amount of information received from that random variable was given by the formula $H(X) = -\sum_{x\in X}p(x)\log p(x)$, where $p(x)$ is the probability that the random variable will give $x$.

The two formulas I gave above are: $S = -k_B\sum_i p_i\ln p_i$ and $H(X) = -\sum_x p(x)\log p(x)$. With the exception of the constant multiplier and the choice of logarithmic base, both these formulas are essentially the same. ruakh points out in the comments that a change of logarithmic base is the same as multiplying by a constant. With that in mind, both equations can be simplified to $S \propto -\sum_i p_i\log p_i$ and $H \propto -\sum_x p_x\log p_x$, which differ only in the choice of dummy summation variable.

Both Gibbs Entropy and "Shannon Entropy" are similarly defined, and serve similar purposes in their respective fields. The smaller the entropy of a system, the easier it is to predict what microstate it is in, regardless of if you are talking about a box of gas or a stream of text over a wire.

Because entropy is a measure (in both statistical dynamics and information science) of the (theoretical) unpredictability of a system, it gets talked about in the context of random number generators because you want random number generators to be unpredictable, to have "high entropy". A pseudo-random number generator will have internal variables that can have a collectively high number of states. Generating a random number both gives you the number, but also modifies those internal variables in a deterministic way. But only a fraction of the possible states would give you the random number you generated, so there are only a fraction of the total number of states that the RNG could be in after generating that number. The entropy of the RNG has reduced. In theory, after extracting enough random variables, the RNG becomes completely predictable; there is only one state it could possibly be in. It has no entropy.