Basic Properties of the Log-Mean-Exp Operator
Overview
Consider any $\eta > 0$ Sometimes, $\eta$ is called the temperature because of analogues from thermodynamics. and vector $q = [q_1, \ldots, q_n] \in \Reals^n$. Define the log-mean-exp operator
$$ \LME_{\eta}[q] := \frac{1}{\eta}\log \Big( \sum_i \pi_i e^{\eta q_i} \Big) $$where the $\pi_i$ are positive weights summing to one.
\[ \sum_i^n \pi_i q_i \le \LME_\eta [q] \le q^* := \max_i q_i. \]Proposition
The log-mean-exp operator satisfies for any $\eta > 0$
$$ \sum_{i}^{n} \pi_i q_i \le \LME_\eta [q] \le q^* := \max_i q_i. \label{eq:bounds} $$Moreover,
$$ \lim_{\eta \rightarrow \infty} \LME_\eta[q] = q^* \label{eq:lmelimit1} $$and
$$ \lim_{\eta \downarrow 0^+} \LME_\eta[q] = \bar{q} := \frac{1}{n} \sum_i^n q_i. \label{eq:lmelimit2} $$Proof. The lower bound in $\eqref{eq:bounds}$ is an immediate consequence of Jensen’s inequality. I.e.
$$ \LME_\eta[q] \ge \frac{1}{\eta} \sum_i \pi_i \log (e ^{\eta q_i}) = \sum_i \pi_i q_i. $$For the upper bound, note that
$$ \LME_\eta[q] = q^* + \LME_\eta[\tilde{q}] $$where $\tilde{q}_i := q_i - q^* \le 0$; and since $\sum_i \pi_i e^{\eta \tilde{q_i}} \le 1$, we have $\LME_\eta[\tilde{q}] \le 0$ from which we have the desired inequality.
To verify $\eqref{eq:lmelimit1}$ observe that
$$ \LME_\eta[\tilde{q}] = \frac{1}{\eta} \log \Big( \sum_{i \in I} \pi_i + \sum_{j \notin I} \pi_j e^{\eta q_j} \Big) \xrightarrow[\eta \rightarrow \infty]{} 0^- $$where $I = \{ i \in \{1, \ldots, n\} \ | \ q_i = q^* \}$.
Finally, $\eqref{eq:lmelimit2}$ is a straightforward application of L’hôpital’s rule.