The Three Extreme Value Distributions: An Introductory Review

The statistical distribution of the largest value drawn from a sample of a given size has only three possible shapes: it is either a Weibull, a Fréchet or a Gumbel extreme value distributions. I describe in this short review how to relate the statistical distribution followed by the numbers in the sample to the associate extreme value distribution followed by the largest value within the sample. Nothing I present here is new. However, from experience, I have found that a simple, short and compact guide on this matter written for the physics community is missing.

1

1Introduction

Extreme value statistics offers a powerful tool box for the theoretical physicist. But it is the kind of tool box that is not missed before one has been introduced to it—perhaps a little like the smart phone. It concerns the statistics of extreme events and it aims to answer questions like “if the strongest signal I have observed over the last hour had the value x, what would the strongest signal expected to be if measured over hundred hours?” Furthermore, if I divide up this hundred-hour interval into a hundred 1-h intervals, what would be the statistical distribution of strongest signal in each 1-h interval?

It is the latter question which is the focus of this mini-review.

There is no lack of literature on extreme value statistics, see e.g., [1–5] or simply google the term. We find it used in connection with spin glasses and disordered systems [6], in connection with $1/f$ noise [7], in connection with optics [8], in connection with fracture [9] or the fiber bundle model [10], in diffusion processes [11] etc. There are plenty of examples from diverse fields of physics.

So, there is no lack of material for the novice that has seen a need for this tool. The problem is that it is not so easy to penetrate the literature, which is often cast in a rather mathematical language which takes work to penetrate. The aim of this mini-review is to present the theory behind and the main results concerning the extreme value distributions in a simple and compact way. We will present nothing new. For a longer, wider and more detailed review of extreme value statistics, Fortin and Clusel [12] or Majumdar et al. present exactly that [13].We have a statistical distribution $p\left(x\right)$ and its associated cumulative probability

$P\left(x\right)={\displaystyle \underset{-\infty }{\overset{x}{\int }}p\left(x^{\prime }\right)d{x}^{\prime }},(1)$

which is the probability to find a number smaller than or equal to x. We draw N numbers from this distribution and record the largest of the N numbers. We repeat this procedure M times and thereby obtain M largest numbers, one for each sequence. What is the distribution of these M largest numbers in the limit when $M\to \infty$, which then defines the extreme value distribution?

It turns out that depending on $p\left(x\right)$, the extreme value distribution will have one of three functional forms:

• The Weibull cumulative probability

$\text{Φ}\left(u\right)=\{\begin{array}{cc}{e}^{-{\left(-u\right)}^{\alpha }}& \text{for}u<0,\\ 1& \text{for}u\ge 0,\end{array}(2)$

where we assume $\alpha >0$. Note that $\text{Φ}\left(-\infty \right)=0$. The corresponding Weibull extreme value distribution is

$\varphi \left(u\right)=\{\begin{array}{cc}\alpha {\left(-u\right)}^{\alpha -1}{e}^{-{\left(-u\right)}^{\alpha }}& \text{for}u<0,\\ 0& \text{for}u\ge 0.\end{array}(3)$

• The Fréchet cumulative probability

$\text{Φ}\left(u\right)=\{\begin{array}{cc}0& \text{for}u\le 0,\\ e^{-u^{-\alpha }}& \text{for}u>0.\end{array}(4)$

Also here we assume $\alpha >0$. Note that $\text{Φ}\left(\infty \right)=1$. The Fréchet extreme value distribution is

$\varphi \left(u\right)=\{\begin{array}{cc}0& \text{for}u\le 0,\\ \alpha u^{-\alpha -1}{e}^{-u^{-\alpha }}& \text{for}u>0.\end{array}(5)$

• The Gumbel cumulative probability

$\text{Φ}\left(u\right)=e^{-e^{-u}},(6)$

where $-\infty <u<\infty$, so that $\text{Φ}\left(-\infty \right)=0$ and $\text{Φ}\left(\infty \right)=1$. The corresponding Gumbel extreme value distribution is given by

$\varphi \left(u\right)=e^{-u-e^{-u}}.(7)$

The questions are 1. which classes of distributions $p\left(x\right)$ lead to which of the three extreme value distributions and 2. what is the connection between x and u in each case? It turns out that.

• distributions where $p\left(x\right)=0$ for $x>x_0$ and $p\left(x\right)\sim {\left(x_0-x\right)}^{\alpha -1}$ as $x\to x_0^{-}$, see Eq. 10, lead to the Weibull extreme value distribution,

• distributions where $p\left(x\right)\sim x^{-\alpha -1}$ as $x\to \infty$, see Eq. 24 lead to the Fréchet extreme value distribution,

• and distributions where $p\left(x\right)$ falls of faster than any power law as $x\to \infty$, see Eq. 53 lead to the Gumbel extreme value distribution.

Furthermore, we will find that.

• for the Weibull extreme value distribution, u is given in terms of x in Eq. 13,

• for the Fréchet extreme value distribution, u given in terms of x in Eq. 27,

• for the Gumbel extreme value distribution, u is given in terms of x in Eqs 51 and 43.

We summarize these results in Table I.

TABLE 1

TABLE 1. Summary of main results.

The discussion that will now follow, will be built on the following relation. We draw N numbers from the probability distribution $p\left(x\right)$: $x_1,x_2,\cdots ,x_N$. The probability that all the N numbers are smaller than or equal to a value x is

$\text{Prob}\left[x_1\le x,x_2\le x,\cdots ,x_N\le x\right]={\left[{\displaystyle \underset{-\infty }{\overset{x}{\int }}p\left(x^{\prime }\right)d{x}^{\prime }}\right]}^N=P{\left(x\right)}^N,(8)$

where $P\left(x\right)$ is the cumulative probability 1. Our task is to figure out the limit $\text{Prob}\left[x_1\le x,x_2\le x,\cdots ,x_N\le x\right]=P{\left(x\right)}^N\to \text{Φ}\left(u\right)$ as $N\to \infty$, and what is $u=u\left(x\right)$ as we approach this limit.

Rather than the conventional approach (see e.g., [10]) to this subject based on the Fréchet, Fisher and Tippett stability criterion [1], I will base the entire discussion on the relation

$\underset{N\to \infty }{\text{lim}}\left(1+\frac{x}{N}\right)=e^x.(9)$

I believe this to be the simpler and more intuitive way.

2

2Weibull Class

We consider here probability distributions $p\left(x\right)$ having the form

$p\left(x\right)=\{\begin{array}{cc}b\alpha {\left(x_0-x\right)}^{\alpha -1}& \text{for}x\to x_0^{-},\\ 0& \text{for}x>x_0,\end{array}(10)$

where b is positive. We note that $0<\alpha <1$ leads to a diverging probability density as $x\to x_0^{-}$. We furthermore note that $\alpha =1$ implies that $p\left(x\right)$ approach a constant when $x\to x_0^{-}$ — which for example is the case when the distribution is uniform. The corresponding cumulative probability is given by

$P\left(x\right)=\{\begin{array}{cc}1& \text{for}x\ge x_0,\\ 1-b{\left(x_0-x\right)}^{\alpha }& \text{for}x\to x_0^{-}.\end{array}(11)$

The extreme value cumulative probability for N samplings is given by

$P{\left(x\right)}^N={\left[1-b{\left(x_0-x\right)}^{\alpha }\right]}^N,(12)$

for $x\to x_0^{-}$. We introduce the variable change

$x-x_0=,\frac{u}{{\left(bN\right)}^{1/\alpha }}(13)$

where the reader should note that b is defined by the original distribution 10. Equation 12 then becomes

$P{\left(x\right)}^N={\left[1-\frac{{\left(-u\right)}^{\alpha }}{N}\right]}^N.(14)$

In the limit of $N\to \infty$, this becomes

$\text{Φ}\left(u\right)=\underset{N\to \infty }{\text{lim}}P{\left(x\right)}^N=e^{-{\left(-u\right)}^{\alpha }},(15)$

for negative u. Hence, we have that

$\text{Φ}\left(u\right)=\{\begin{array}{cc}{e}^{-{\left(-u\right)}^{\alpha }}& \text{for}u<0,\\ 1& \text{for}u\ge 0,\end{array}(16)$

which is the Weibull cumulative probability, valid for all values of u even though we only know the behavior of $p\left(x\right)$ close to $x_0$. The Weibull probability density is given by

$\varphi \left(u\right)=\frac{d\text{Φ}\left(u\right)}{du}=\{\begin{array}{cc}\alpha {\left(-u\right)}^{\alpha -1}{e}^{-{\left(-u\right)}^{\alpha }}& \text{for}u<0,\\ 0& \text{for}u\ge 0.\end{array}(17)$

We note that the Weibull distribution resembles a stretched exponential. This is correct for $\alpha <1$. However, $\alpha \ge 1$ is much more common in the wild.

We express the Weibull cumulative probability in terms of the original variable x using Eq. 13,

$\text{Φ}\left(u\right)=\text{Φ}\left({\left(bN\right)}^{1/\alpha }\left(x-x_0\right)\right)=e^{-Nb{\left(x_0-x\right)}^{\alpha }}=\widetilde{\text{Φ}}\left(x\right).(18)$

Hence, in terms of the original variable x, the Weibull extreme value distribution becomes

$\tilde{\varphi }\left(x\right)=\frac{d\widetilde{\text{Φ}}\left(x\right)}{dx}=Nb\alpha {\left(-x\right)}^{\alpha -1}{e}^{-Nb{\left(x_0-x\right)}^{\alpha }}.(19)$

2.1

2.1Weibull: An Example

We now work out a concrete example. Let us assume that $p\left(x\right)$ is given by

$p\left(x\right)=\{\begin{array}{cc}0& \text{for}x<0,\\ \alpha {\left(1-x\right)}^{\alpha -1}& \text{for}0\le x\le 1,\\ 0& \text{for}x>1,\end{array}(20)$

i.e., $b=1$ and $x_0=1$ in Eq. 10. The cumulative probability is then

$P\left(x\right)=\{\begin{array}{cc}0& \text{for}x<0,\\ 1-{\left(1-x\right)}^{\alpha }& \text{for}0\le x\le 1,\\ 1& \text{for}x>1.\end{array}(21)$

From Eq. 19 and we have that

$\tilde{\varphi }\left(x\right)=N\alpha {\left(1-x\right)}^{\alpha -1}{e}^{-N{\left(1-x\right)}^{\alpha }}.(22)$

We show the distribution 20 with $\alpha =3$ together with the corresponding extreme value distributions for $N=100$ and $N=\mathrm{1,000}$, Eq. 19 in Figure 1A.

FIGURE 1

FIGURE 1. (A) The curve that has its maximum at $x=0$ is the probability distribution 20 with $\alpha =3$. The curve that has its maximum in the middle is $\tilde{\varphi }\left(x\right)$, Eq. 22 with $N=100$ and the curve that has its maximum to the right is $\tilde{\varphi }\left(x\right)$ with $N=\mathrm{1,000}$. (B) The histograms shown here are based on data according to the probability distribution 20 with $\alpha =3$. The histogram having its maximum to the left shows all the generated data. The histogram having its maximum in the middle shows the largest number among each sequence of numbers of length 100, and the histogram having its maximum to the right shows the largest number among each sequence of numbers of length 1,000. We generated ${10}^7$ sequences for both cases.

Using a random number generator producing IID numbers¹r uniformly distributed on the unit interval, we may stochastically generate numbers that are distributed according to the probability density $p\left(x\right)$ given in 20. We do this by inverting the expression $P\left(x\right)=r$, where the cumulative probability is given by 21. Hence, we have

$x=1-r^{1/\alpha },(23)$

where we have also used that r may be substituted for $1-r$ in 21. We generate a sequence of sequences of numbers using this algorithm, each sequence having length N. We then identify the largest value within each sequence. We chose $N=100$ and $N=\mathrm{1,000}$, in each case generating ${10}^7$ such sequences. The histograms based on the random numbers themselves, and of the extreme values for each sequence of length either 100 or 1,000 we show in Figure 1B. This figure should be compared to Figure 1A.

The Weibull distribution, Eq. 17 is much used in connection with material strength [15]. This is no coincidence. Consider a chain. Each link in the chain can sustain a load up to a certain value, above which it fails. This maximum value is distributed according to some probability distribution. When the chain is loaded, it will be the link with the smallest failure threshold that will break first causing the chain as a whole to fail. Hence, the strength distribution of an ensemble of chains is an extreme value distribution, but with respect to the smallest rather than the largest value. The link strength must a positive number. Hence, the link strength distribution is cut off at zero or some positive value. The distribution close to this cutoff value must behave as a power law in the distance to the cutoff, e.g., due to a Taylor expansion around the cutoff. The corresponding extreme value distribution, which is the chain strength distribution, must then be a Weibull distribution.

3

3Fréchet Class

We now assume that the probability distribution $p\left(x\right)$ behaves as

$p\left(x\right)=b\alpha x^{-\alpha -1}\text{for}x\to \infty ,(24)$

and the corresponding cumulative probability behaves as

$P\left(x\right)=1-b{x}^{-\alpha }\text{for}x\to \infty .(25)$

The extreme value cumulative probability for N samplings is given by

$P{\left(x\right)}^N={[1-b{x}^{-\alpha }]}^N,(26)$

for $x\to \infty$. We introduce the variable change

$x={\left(bN\right)}^{1/\alpha }u,(27)$

where b comes from the original distribution 24. We now plug this change of variables into Eq. 26 to find

$P{\left(x\right)}^N={\left[1-b{\left({\left(bN\right)}^{1/\alpha }u\right)}^{-\alpha }\right]}^N={\left[1-\frac{u^{-\alpha }}{N}\right]}^N.(28)$

In the limit of $N\to \infty$, this becomes

$\text{Φ}\left(u\right)=\underset{N\to \infty }{\text{lim}}P{\left(x\right)}^N=e^{-u^{-\alpha }},(29)$

where $u\ge 0$ is given by Eq. 27. We see that $\text{Φ}\left(u\right)\to 0$ as $u\to 0^{+}$. Furthermore, for $u<0$, the function is no longer real. Hence, we define $\text{Φ}\left(u\right)=0$ for $u<0$. The ensuing extreme value cumulative probability is then given by

$\text{Φ}\left(u\right)=\{\begin{array}{cc}0& \text{for}u\le 0,\\ e^{-u^{-\alpha }}& \text{for}u>0,\end{array}(30)$

which is the Fréchet cumulative probability. The Fréchet probability density is given by

$\varphi \left(u\right)=\frac{d\text{Φ}\left(u\right)}{du}=\{\begin{array}{cc}0& \text{for}u\le 0.\\ \alpha u^{-\alpha -1}{e}^{-u^{-\alpha }}& \text{for}u>0.\end{array}(31)$

We express the Fréchet cumulative probability in terms of the original variable x using Eq. 27,

$\text{Φ}\left(u\right)=\text{Φ}\left(\frac{x}{{\left(bN\right)}^{1/\alpha }}\right)=e^{-N{x}^{-\alpha }}=\widetilde{\text{Φ}}\left(x\right).(32)$

Hence, in terms of the original variable x, the Fréchet extreme value distribution becomes

$\tilde{\varphi }\left(x\right)=\frac{d\widetilde{\text{Φ}}\left(x\right)}{dx}=N\alpha x^{-\alpha -1}{e}^{-N{x}^{-\alpha }}.(33)$

3.1

3.1Fréchet: An Example

We consider the distribution

$p\left(x\right)=\{\begin{array}{cc}0& \text{for}x\le 1,\\ \alpha x^{-\alpha -1}& \text{for}x>1,\end{array}(34)$

The corresponding cumulative probability is given by

$P\left(x\right)=\{\begin{array}{cc}0& \text{for}x\le 1,\\ 1-x^{-\alpha }& \text{for}x>1.\end{array}(35)$

Using Eq. 33, we find the corresponding Fréchet extreme value distribution to be

$\tilde{\varphi }\left(x\right)=N\alpha x^{-\alpha -1}{e}^{-N{x}^{-\alpha }},(36)$

valid for all $x>1$. We show $p\left(x\right)$ and the corresponding $\tilde{\varphi }\left(x\right)$ for $\alpha =3$ and $N=100$ and $N=\mathrm{1,000}$ in Figure 2A.

FIGURE 2

FIGURE 2. (A) The curve that has its maximum at $x=1$ is the probability distribution 34 with $\alpha =3$. The curve that has its maximum in the middle is $\tilde{\varphi }\left(x\right)$, Eq. 36 with $N=100$ and the curve that has its maximum to the right is $\tilde{\varphi }\left(x\right)$ with $N=\mathrm{1,000}$. (B) The histograms shown here are based on data according to the probability distribution 34 with $\alpha =3$. The histogram having its maximum to the left shows all the generated data. The histogram having its maximum in the middle shows the largest number among each sequence of numbers of length 100, and the histogram having its maximum to the right shows the largest number among each sequence of numbers of length 1,000. For each sequence length, ${10}^7$ such sequences were generated.

In order to compare with numerical results, we generate numbers distributed according to 34 by solving the equation $P\left(x\right)=r$ where r is drawn from a uniform distribution on the unit interval. From Eq. 35, we get

$x=r^{-1/\alpha }.(37)$

We generate a sequence of numbers using this algorithm, grouping them together in sequences of $N=100$ or $N=\mathrm{1,000}$. We generate ${10}^7$ such sequences. The histograms based on the random numbers themselves generated with Eq. 37, and of the extreme values for each sequence of length either 100 or 1,000 we show in Figure 2B. This figure should be compared to Figure 2A.

4

4Gumbel Class

We now assume we have a probability distribution that takes the form

$p\left(x\right)=f^{\prime }\left(x\right)e^{-f\left(x\right)}\text{for}x>x_0,(38)$

where $f^{\prime }\left(x\right)=df\left(x\right)/dx$. We have that $x_0$ is any number, positive or negative, and $f\left(x\right)$ is an increasing function of x. We will later on introduce a sufficient criterion imposed on $p\left(x\right)$ to produce the Gumbel distribution, see Eq. 53. This criterion is equivalent to $f\left(x\right)$ fulfilling

$\underset{x\to \infty }{\text{lim}}\frac{d}{dx}\left(\frac{1}{f^{\prime }\left(x\right)}\right)=0.(39)$

This criterion is e.g., fulfilled by any polynomial $f\left(x\right)$.

The cumulative probability is

$P\left(x\right)=1-e^{-f\left(x\right)}\text{for}x>x_0.(40)$

We do not care about the form of $p\left(x\right)$ or $P\left(x\right)$ for $x\le x_0$.

The extreme value cumulative probability for N samplings is given by

$P{\left(x\right)}^N={[1-e^{-f\left(x\right)}]}^N,(41)$

for $x>x_0$. We introduce the variable change

$\tilde{u}=f\left(x\right)-f\left(x_N\right),(42)$

where $x_N$ is given by

$P\left(x_N\right)=1-\frac{1}{N}.(43)$

Even though $x_N$ is defined by 43, we may interpret its meaning. We do so in the conclusion, see Eq. 71. From Eq. 40 we then have that

$f\left(x_N\right)=\text{ln}N.(44)$

Let us now define

$\text{Δ}x=x-x_N.(45)$

We then expand $f\left(x\right)$ around $x_N$,

$f\left(x\right)=f\left(x_N+\text{Δ}x\right)={\displaystyle \sum }_{n=0}^{\infty }\frac{f^{\left(n\right)}\left(x_N\right)}{n!}\text{Δ}{x}^n,(46)$

where $f^{\left(n\right)}\left(x\right)=d^{n}f\left(x\right)/d{x}^n$. If we now set

$\text{Δ}x=\frac{1}{f^{\prime }\left(x_N\right)},(47)$

so that the first order term in the expansion becomes constant as N increases, we will have that

$f^{\prime }\left(x_N\right)\text{Δ}x+{\displaystyle \sum }_{n=2}^{\infty }\frac{f^{\left(n\right)}\left(x_N\right)}{n!}\text{Δ}{x}^n=1+{\displaystyle \sum }_{n=2}^{\infty }\frac{f^{\left(n\right)}\left(x_N\right)}{n!f^{\prime }{\left(x_N\right)}^n}.(48)$

Hence, if we have that

$\underset{N\to \infty }{\text{lim}}\frac{f^{\left(n\right)}\left(x_N\right)}{f^{\prime }{\left(x_N\right)}^n}=0,(49)$

for $n\ge 2$, then in this limit, we will find

$f\left(x\right)=f\left(x_N\right)+f^{\prime }\left(x_N\right)\text{Δ}x=f\left(x_N\right)+u,(50)$

where we define

$u=f^{\prime }\left(x_N\right)\text{Δ}x=Np\left(x_N\right)\left(x-x_N\right).(51)$

Here we have used Eqs (40) and (44).

4.1

4.1Sufficient Criterion for the Gumbel Class

If we combine Eq. 49 for $n=2$ with Eqs 38 and 40, we find

$\underset{N\to \infty }{\text{lim}}\frac{f^{\prime \prime }\left(x_N\right)}{f^{\prime }{\left(x_N\right)}^2}=\underset{N\to \infty }{\text{lim}}\frac{d}{dx}{\left[\frac{1-P\left(x\right)}{p\left(x\right)}\right]}_{x=x_N}=0,(52)$

which is equivalent to

$\underset{x\to \infty }{\text{lim}}\frac{d}{dx}\left[\frac{1-P\left(x\right)}{p\left(x\right)}\right]=0.(53)$

Equation 53, which is equivalent to Eq. 39, is in fact a sufficient condition for 49 to hold for all $n>1$. We may show this through induction. We have that

$\frac{f^{\left(n+1\right)}\left(x\right)}{f^{\prime }{\left(x\right)}^{n+1}}=\frac{1}{f^{\prime }\left(x\right)}\frac{d}{dx}\left(\frac{f^{\left(n\right)}\left(x\right)}{f^{\prime }{\left(x\right)}^n}\right)+\frac{f^{\left(n\right)}\left(x\right)}{f^{\prime }{\left(x\right)}^{n+2}}.(54)$

If condition 52 is fulfilled, that is when the expression above is zero in the limit $x\to \infty$ for $n=2$, we also have that

$\underset{N\to \infty }{\text{lim}}\frac{f^{\left(3\right)}\left(x\right)}{f^{\prime }{\left(x\right)}^3}=0,(55)$

since both terms on the right hand side of Eq. 54 are zero in this limit. We now assume Eq. 49 to be true for some $n>3$. We then have that

$\underset{N\to \infty }{\text{lim}}\frac{f^{\left(n+1\right)}\left(x_N\right)}{f^{\prime }{\left(x_N\right)}^{n+1}}=0,(56)$

again due to both terms on the right hand side of Eq. 54 are zero in this limit. This completes the proof.

4.2

4.2Return to the Derivation

We now combine Eq. 42 with Eq. 41 to find

$P{\left(x\right)}^N={\left[1-e^{-u-f\left(x_N\right)}\right]}^N={\left[1-e^{-u-\ln N}\right]}^N={\left[1-\frac{e^{-u}}{N}\right]}^N.(57)$

In the limit of $N\to \infty$, this becomes

$\text{Φ}\left(u\right)=\underset{N\to \infty }{\text{lim}}P{\left(x\right)}^N=e^{-e^{-u}},(58)$

which is the Gumbel cumulative probability. Here $-\infty <u<\infty$. The Gumbel probability density is given by

$\varphi \left(u\right)=\frac{d\text{Φ}\left(u\right)}{du}=e^{-u-e^{-u}}.(59)$

We express the Gumbel cumulative probability in terms of the original variable x using Eq. 51,

$\text{Φ}\left(u\right)=\text{Φ}\left(Np\left(x_N\right)\left(x-x_N\right)\right)=e^{-e^{-Np\left(x_N\right)\left(x-x_N\right)}}=\widetilde{\text{Φ}}\left(x\right).(60)$

Hence, in terms of the original variable x, the Gumbel extreme value distribution becomes

$\tilde{\varphi }\left(x\right)=\frac{d\widetilde{\text{Φ}}\left(x\right)}{dx}=Np\left(x_N\right)e^{-Np\left(x_N\right)\left(x-x_N\right)-e^{-Np\left(x_N\right)\left(x-x_N\right)}}.(61)$

4.3

4.3An Example: The Gaussian

Here is an example: the Gaussian. The Gaussian probability density is given by

$p\left(x\right)=\frac{e^{-x^2/2\sigma }}{\sqrt{2\pi \sigma }}\hspace{0.17em},(62)$

where σ is the square of the standard deviation. The cumulative probability is

$P\left(x\right)=\frac{1}{2}\left[1+\text{erf}\left(\frac{x}{\sqrt{2\sigma }}\right)\right],(63)$

where $\text{erf}\left(z\right)$ is the error function. In order to verify that the Gaussian generates the Gumbel extreme distribution, we use the sufficient condition 53,

$\underset{x\to \infty }{\text{lim}}\frac{d}{dx}\left[\frac{1-P\left(x\right)}{p\left(x\right)}\right]=\underset{x\to \infty }{\text{lim}}\sqrt{\frac{\pi }{2\sigma }}{e}^{x^2/2\sigma }x\left[1-\text{erf}\left(\frac{x}{\sqrt{2\sigma }}\right)\right]=0.(64)$

The Gaussian cumulative probability in Eq. 63 has the asymptotic form

$P\left(x\right)=1-\sqrt{\frac{\sigma }{2\pi }}\frac{e^{-x^2/2\sigma }}{x},(65)$

for large x. We determine $x_N$ solving Eq. 43 using this asymptotic form. We find

$x_N=\sqrt{\sigma W\left(\frac{N^2}{2\pi }\right)},(66)$

where $W\left(z\right)$ is the Lambert W function, also known as the product logarithm, which is the solution to the equation $W\left(z\right)\text{exp}\left[W\left(z\right)\right]=z$. For large arguments, it approaches the natural logarithm, $W\left(z\right)\to \text{log}\left(z\right)$ as $z\to \infty$ [16]. This gives us

$Np\left(x_N\right)=\sqrt{\frac{1}{\sigma }W\left(\frac{N^2}{2\pi }\right)},(67)$

when inserting the expression for $x=x_N$, Eq. 66 into Eq. 62. Thus we may now express the variable u in the Gumbel cumulative probability 57 in terms of the variables x, σ and N using Eq. 51,

$u=x\sqrt{\frac{1}{\sigma }W\left(\frac{N^2}{2\pi }\right)}-W\left(\frac{N^2}{2\pi }\right).(68)$

We show in Figure 3A the Gaussian and the corresponding Gumbel distributions for $\sigma =1$ and $N=100$ and $N=\mathrm{1,000}$. We find that $x_{100}=2.375$ and $x_{1000}=3.115$. These are the confidence intervals for 99% and 99.9%.

FIGURE 3

FIGURE 3. (A) The Gaussian and the corresponding Gumbel distributions for $\sigma =1$ and $N=100$ and $N=\mathrm{1,000}$. (B) The histograms shown here are based on data generated using the Box-Müller algorithm which produces numbers distributed according to a Gaussian. Here $\sigma =1$. The histogram with the maximum to the left shows all the generated data. The histogram with its maximum in the middle shows the largest number among each sequence of numbers of length 100, and the histogram with the rightmost maximum shows the largest number among each sequence of numbers of length 1,000. For each sequence length, ${10}^7$ such sequences were generated.

We show in Figure 3B a histogram based on numbers distributed according to a Gaussian distribution using the Box-Müller algorithm [14]. These numbers were grouped together in sets of either $N=100$ or $N=\mathrm{1,000}$ elements. I generated ${10}^7$ such sets. The figure displays the two extreme distributions for the two set sizes. This figure should be compared to Figure 3A. In contrast to the two other extreme value distributions, we see that there are visible discrepancies between the calculated Gumbel distributions in Figure 3A and the extreme value histograms in Figure 3B. We see furthermore that the histogram for $N=\mathrm{1,000}$ is closer to the calculated Gumbel distribution than the histogram for $N=100$. This is due to the very slow convergence induced by the Lambert W functions. Slow convergence is typical for the Gumbel extreme value distributions. This slow convergence has been analyzed and recently and through clever use of scaling methods remedied [17].

5

5Concluding Remarks

We summarize the main results presented in this mini-review in Table I.

We have only discussed the distributions associated with the largest values of x except for the Weibull extreme value distribution, Section 2. It is, however, easy to work out: just transform$x\to -x$. Otherwise, the story presented here is rather complete.

There is one remark that needs to be made, though. In the derivation of the Gumbel extreme value distribution, Section 4, we defined a variable $x_N$ in Eq. 43. First of all, $x_N$ defined in Eq. 43 may be calculated for any cumulative probability $P\left(x\right)$ and it has an interpretation making it very useful.

The probability density for the largest among N numbers drawn using the probability distribution $p\left(x\right)$ is given by

$p_N\left(x\right)=\frac{dP{\left(x\right)}^N}{dx}=NP{\left(x\right)}^{N-1}p\left(x\right).(69)$

We calculate the average of the cumulative probability $P\left(x\right)$ for the extreme value based on N samples,

$\langle P\left(x\right)\rangle ={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}P\left(x\right)p_N\left(x\right)dx}={\displaystyle \underset{0}{\overset{1}{\int }}{P}^{N}dP}=\frac{N}{N+1}=1-\frac{1}{N+1}\hspace{0.17em}.(70)$

For large N, we may write this as

$\langle P\left(x\right)\rangle =P\left(x_N\right)=1-\frac{1}{N},(71)$

using here Eq. 43. Hence, we may interpret $x_N$ as the x value corresponding to the average confidence interval of the largest observed value in sequences of N numbers. It is essentially the typical size of the extreme value for a sample of size N.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Funding

This work was partly supported by the Research Council of Norway through its Centers of Excellence funding scheme, project number 262644.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be constructed as a potential conflict of interest.

Acknowledgments

I thank Eivind Bering, Astrid de Wijn, H. George, E. Hentschel, Srutarshi Pradhan, and Itamar Procaccia for numerous interesting discussions on this topic.

Footnotes

¹

IID variables. Independent and identically distributed random variables, a terminology used in some communities.

REFERENCES

1. Gumbel EJ. Statistics of extremes. New York: Columbia University Press(1958).

Google Scholar

2. David HA. Order statistics. 2nd ed. New York: Wiley (1981).

Google Scholar

3. Galambos J. The asymptotic theory of extreme order statistics. Malabar, FL: Krieger(1987).

Google Scholar

4. Embrechts P, Klüppelberg C, Mikosh T. Modeling extreme events for insurance and finance. Berlin: Springer(1997).

Google Scholar

5. Coles S. An introduction to statistical modeling of extreme events. Berlin: Springer(2001).

Google Scholar

6. Bouchaud J-P, Mézard M. Universality classes for extreme-value statistics. J Phys Math Gen(1997). 30:7997. doi:10.1088/0305-4470/30/23/004

CrossRef Full Text | Google Scholar

7. Antal T, Droz M, Györgyi G, Rácz Z. 1/f noise and extreme value statistics. Phys Rev Lett(2001). 87:240601. doi:10.1103/physrevlett.87.240601

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Randoux S, Suret P. Experimental evidence of extreme value statistics in Raman fiber lasers. Opt Lett(2012). 37:500. doi:10.1364/OL.37.000500

CrossRef Full Text | Google Scholar

9. Taloni A, Vodret M, Costantini G, Zapperi S. Size effects on the fracture of microscale and nanoscale materials. Nat Rev Mater(2018). 3:211–24. doi:10.1038/s41578-018-0029-4

CrossRef Full Text | Google Scholar

10. Hansen A, Hemmer PC, Pradhan S. The fiber bundle model. Berlin: Wiley VCH(2015).

Google Scholar

11. Pal A, Eliazar I, Reuveni S. First passage under restart with branching. Phys Rev Lett(2019). 122:020602. doi:10.1103/PhysRevLett.122.020602

CrossRef Full Text | Google Scholar

12. Fortin J-Y, Clusel M. Applications of extreme value statistics in physics. J Phys Math Theor(2015). 48:183001. doi:10.1088/1751-8113/48/18/183001

CrossRef Full Text | Google Scholar

13. Majumdar SN, Pal A, Schehr G. Extreme value statistics of correlated random variables: a pedagogical review. Phys Rep (2020) 840:1. doi:10.1016/j.physrep.2019.10.005

CrossRef Full Text | Google Scholar

14. Press WH, Teukolsky SA, Vetterling WT, Flannery BP. Numerical recipes. 3rd ed. Cambridge: Cambridge University Press(2007).

Google Scholar

15. Rinne H. The Weibull distribution. Boca Raton: CRC Press (2008).

Google Scholar

16. Corless RM, Gonnet GH, Hare DEG, Jeffrey DJ, Knuth DE. On the LambertW function. Adv Comput Math (1996) 5:329–59. doi:10.1007/BF02124750

CrossRef Full Text | Google Scholar

17. Zarfaty L, Barkai E, Kessler DA. Accurately approximating extreme value statistics(2020). arXiv:2006.13677.

Google Scholar