Shannon Entropy Calculator

Enter category counts or probabilities and compute Shannon entropy H = -Σ pᵢ·log₂(pᵢ) in bits, along with maximum entropy and normalized entropy (efficiency).

Enter one value per category, separated by commas, spaces, or new lines. Use raw counts (e.g. 10, 20, 30) or probabilities that sum to 1 — the calculator normalizes automatically. Zeros are ignored (0·log 0 = 0).

Quick Facts

Formula
H = -Σ pᵢ · log₂(pᵢ)
Average information content per outcome; maximized (log₂ n) when all categories are equally likely, and 0 when one outcome is certain.

Your Results

Calculated
Shannon entropy (H)
-
Average information per outcome
Maximum entropy
-
log(n) for n categories
Normalized entropy
-
Efficiency: H / H_max
Categories used
-
Non-zero categories (n)

Ready

Enter your category counts or probabilities and press Calculate.

What Shannon entropy measures

Shannon entropy quantifies the average uncertainty — or information content — of a random variable. Introduced by Claude Shannon in his 1948 paper A Mathematical Theory of Communication, it answers a concrete question: on average, how many yes/no questions do you need to identify an outcome drawn from a given probability distribution? A fair coin (two equally likely outcomes) has an entropy of exactly 1 bit; a loaded coin that almost always lands heads carries much less than 1 bit because the outcome is nearly certain.

The formula

For a distribution with probabilities p₁, p₂, …, pₙ over n categories, the Shannon entropy is:

H = -Σ pᵢ · log₂(pᵢ)

The sum runs over every category. Each term pᵢ·log₂(pᵢ) is negative (because probabilities are ≤ 1 and their logarithm is ≤ 0), so the leading minus sign makes H positive. By convention, any category with probability 0 contributes 0, since the limit of p·log(p) as p→0 is 0. When the logarithm is taken in base 2, entropy is in bits; base e gives nats and base 10 gives dits (also called hartleys). This calculator accepts raw counts and converts them to probabilities for you by dividing each count by the total.

Why the maximum is log₂(n)

Entropy is largest when uncertainty is greatest, which happens when every category is equally likely (pᵢ = 1/n for all i). Substituting into the formula gives H_max = log₂(n). So two equally likely outcomes give 1 bit, four give 2 bits, eight give 3 bits, and 256 give 8 bits — which is exactly why a single byte can represent 256 equally likely symbols. At the other extreme, if one category has probability 1 and the rest are 0, entropy is 0: there is no uncertainty at all.

Normalized entropy (efficiency)

Dividing the observed entropy by its maximum, H / log₂(n), gives a value between 0 and 1 sometimes called the entropy efficiency or evenness. It lets you compare how uniform two distributions are even when they have different numbers of categories. A normalized entropy of 1 means perfectly uniform; a value near 0 means the distribution is dominated by a single outcome. Ecologists use the same quantity (as Pielou's evenness) to compare species diversity across communities of different sizes.

Common reference values

  • Fair coin (0.5, 0.5): H = 1 bit.
  • Fair six-sided die (uniform over 6): H = log₂(6) ≈ 2.585 bits.
  • Uniform English alphabet (26 letters): log₂(26) ≈ 4.70 bits per letter; actual printed English is closer to ~1–1.5 bits per letter because letters are far from uniform and highly correlated.
  • A distribution of {10, 20, 30, 40}: probabilities 0.1, 0.2, 0.3, 0.4 give H ≈ 1.846 bits, against a maximum of log₂(4) = 2 bits (efficiency ≈ 92.3%).

Frequently Asked Questions

What is Shannon entropy measured in?
It depends on the logarithm base. Base 2 gives bits — the average number of yes/no questions needed to pin down an outcome. Base e gives nats and base 10 gives dits (hartleys). To convert, note that 1 nat ≈ 1.4427 bits and 1 dit ≈ 3.3219 bits. This calculator defaults to bits but lets you switch the base.
Do I have to enter probabilities that sum to 1?
No. You can enter raw counts (for example 10, 20, 30, 40) and the calculator normalizes them by dividing each by the total, so the probabilities automatically sum to 1. Entering probabilities directly (0.1, 0.2, 0.3, 0.4) gives the same result. If your probabilities don't quite sum to 1 because of rounding, normalization corrects for it.
Why does a category with zero probability not break the formula?
The term p·log(p) approaches 0 as p approaches 0, so empty categories contribute nothing to the entropy. This calculator simply skips zero-probability categories, which is mathematically identical to the standard convention 0·log 0 = 0. Empty categories also don't count toward n when computing the maximum entropy.
Is higher entropy good or bad?
Neither by itself — entropy just measures uncertainty. In cryptography and password strength, high entropy is desirable because it means outcomes are hard to guess. In data compression, low entropy is good because predictable data compresses well. In machine learning, entropy is used in decision-tree splits and cross-entropy loss precisely to measure and reduce uncertainty. Interpret the number in the context of your goal.

Related calculators