Summing an Infinite Table: Iterated and Double Series

1. The problem: an infinite matrix

We are given an infinite matrix

\[ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n} & \cdots \\ a_{21} & a_{22} & a_{23} & \cdots & a_{2n} & \cdots \\ a_{31} & a_{32} & a_{33} & \cdots & a_{3n} & \cdots \\ \vdots & & & & & \\ a_{m1} & a_{m2} & a_{m3} & \cdots & a_{mn} & \cdots \\ \vdots & & & & & \end{bmatrix}, \]

and we want to give meaning to "the sum of all its elements". One natural idea: list all the elements \(a_{ij}\) in some order as a sequence

\[ u_1, u_2, u_3, \dots \tag{1} \]

(every element appearing exactly once) and form the ordinary series

\[ \sum_{n=1}^\infty u_n. \tag{2} \]

But there are infinitely many ways to list the elements, and we know from the previous post that rearranging a series can change its sum. So we need specific, well-defined methods.

Such sums appear more often than you'd expect: when multiplying two series (the Cauchy product from the previous post is one), in probability (summing over pairs of outcomes), in physics (lattice sums), and as a trick for evaluating ordinary series.

2. Three methods of summation

Figure 1. The three methods. Left: summing along antidiagonals. Middle: summing ever larger rectangles \(A_{nm}\). Right: summing each row to infinity first, then adding the row sums.

1° The triangle (diagonal) method

Definition Form the sums \[ S_1 = a_{11}, \qquad S_2 = S_1 + a_{12} + a_{21}, \qquad S_3 = S_2 + a_{13} + a_{22} + a_{31}, \qquad \dots \] adding one antidiagonal at a time. If the limit of \(S_1, S_2, S_3, \dots\) exists and is finite, the series (2) with \[ u_n = \sum_{i=1}^{n} a_{i,\,n-i+1} \] is said to converge in Cauchy's sense, and its sum is \( S = \lim_{n\to\infty} S_n \).

Each \(S_n\) is the sum of a "triangle" in the upper-left corner of the matrix. This is exactly how the Cauchy product of two series was built in the previous post: \(c_n\) collected the products \(a_ib_k\) along one antidiagonal.

2° The method of finite rectangles

Definition Form the finite sums over the rectangle of the first \(n\) rows and \(m\) columns, \[ A_{nm} = \sum_{i=1}^{n}\sum_{j=1}^{m} a_{ij}, \] and look for the double limit \( \lim\limits_{\substack{n\to\infty\\ m\to\infty}} A_{nm} \). If it exists and is finite, the double series \[ \sum_{i,j=1}^{\infty} a_{ij} \tag{3} \] converges, and its sum is \( S = \lim\limits_{\substack{n\to\infty\\ m\to\infty}} A_{nm} \).

"Double limit" means that \(n\) and \(m\) go to infinity independently: for every \(\varepsilon > 0\) there is an \(N\) such that \( |A_{nm} - S| < \varepsilon \) for all \(n > N\) and \(m > N\), however the rectangle grows (tall and narrow, wide and flat, or square).

3° The method of infinite rectangles (iterated series)

Definition If every row of \(A\) forms a convergent series, \[ \sum_{j=1}^\infty a_{ij} = A_i, \] and if the series \( \sum_{i=1}^\infty A_i \) converges, we say that the iterated series \[ \sum_{i=1}^\infty\sum_{j=1}^\infty a_{ij} \tag{4} \] converges. Summing by columns first gives the other iterated series \[ \sum_{j=1}^\infty\sum_{i=1}^\infty a_{ij}. \tag{4'} \]

Here each step is an ordinary infinite series: first sum each row to infinity (an "infinitely wide rectangle" of height 1), then sum the results.

3. A warning: the order can matter

Two iterated sums, two different answers Consider the matrix with \(1\) on the main diagonal, \(-1\) just to the right of it, and \(0\) everywhere else:

1	−1	0	0	⋯
0	1	−1	0	⋯
0	0	1	−1	⋯
0	0	0	1	⋯
⋮				⋱

By rows (4): every row sums to \(1 - 1 = 0\), so \( \sum_i\sum_j a_{ij} = 0 \).
By columns (4′): the first column sums to \(1\); every other column contains one \(-1\) and one \(1\), summing to \(0\). So \( \sum_j\sum_i a_{ij} = 1 \).

Same numbers, different order, different sums. The culprit: \( \sum|a_{ij}| = \infty \). The matrix contains infinitely many \(+1\)'s and \(-1\)'s, exactly like a conditionally convergent series, and the two orders cancel them differently.

This mirrors Riemann's theorem from the previous post. And the cure is the same: absolute convergence.

4. The main theorems

Theorem 1 If the series (2) converges absolutely, then \[ \sum_{n=1}^\infty u_n = \sum_{j=1}^\infty\sum_{i=1}^\infty a_{ij}. \]

Theorem 2 If the iterated series of absolute values \( \displaystyle\sum_{j=1}^\infty\sum_{i=1}^\infty |a_{ij}| \) converges, then the series \( \displaystyle\sum_{j=1}^\infty\sum_{i=1}^\infty a_{ij} \) and \( \displaystyle\sum_{n=1}^\infty u_n \) also converge, and \[ \sum_{j=1}^\infty\sum_{i=1}^\infty a_{ij} = \sum_{n=1}^\infty u_n. \]

Theorem 3 · Changing the order of summation If \( \displaystyle\sum_{j=1}^\infty\sum_{i=1}^\infty |a_{ij}| \) converges, then \[ \sum_{j=1}^\infty\sum_{i=1}^\infty a_{ij} = \sum_{i=1}^\infty\sum_{j=1}^\infty a_{ij}. \]

Theorem 4 · Non-negative terms If \( a_{ij} \ge 0 \), the double series (3) converges if and only if its partial sums \[ A_{nm} = \sum_{i=1}^{n}\sum_{j=1}^{m} a_{ij} \] are bounded.

Theorem 5 · All methods agree Suppose the sequence (1) contains every element of \(A\) (each exactly once). If any one of the series (2), (3), (4), (4′) converges after its terms are replaced by their absolute values, then all of them converge and have the same sum.

How to read these theorems. Theorem 5 is the summary of the whole topic: check absolute convergence in whichever form is easiest (often an iterated sum of \(|a_{ij}|\)), and then you are free to sum in any order you like. Theorem 4 is the double-series version of the basic fact from the first post: a series with non-negative terms converges exactly when its partial sums are bounded. For non-negative terms, absolute convergence and convergence are the same thing, so for \(a_{ij} \ge 0\) every method gives the same answer (possibly \(+\infty\) for all of them).

These theorems are the discrete twins of the theorems on changing the order of integration from the posts on integrals with a parameter (non-negative integrand, or absolute integrability).

5. Worked examples

Example 1 · A product of two series For \(|x| < 1\), \(|y| < 1\), let \( a_{ij} = x^iy^j \) (\(i, j \ge 1\)). The rectangle sums factor: \[ A_{nm} = \Big(\sum_{i=1}^n x^i\Big)\Big(\sum_{j=1}^m y^j\Big) \longrightarrow \frac{x}{1-x}\cdot\frac{y}{1-y}. \] Since the same holds with \(|x|, |y|\), the double series converges absolutely, and by Theorem 5 every method gives \[ \sum_{i,j=1}^\infty x^iy^j = \frac{xy}{(1-x)(1-y)}. \] Similarly, \( \sum_{i,j\ge1}\frac{1}{i^2j^2} = \Big(\sum\frac1{i^2}\Big)^2 = \frac{\pi^4}{36} \).

Example 2 · Which double p-series converge? For which \(p\) does \( \displaystyle\sum_{i,j=1}^\infty \frac{1}{(i + j)^p} \) converge?

The terms are positive, so (Theorem 5) we may use the triangle method. On the antidiagonal \(i + j = k\) there are \(k - 1\) terms, all equal to \( \frac{1}{k^p} \):

\[ \sum_{i,j\ge1}\frac{1}{(i+j)^p} = \sum_{k=2}^\infty \frac{k - 1}{k^p}. \] Since \( \frac{k-1}{k^p} \sim \frac{1}{k^{p-1}} \), this converges exactly when \(p > 2\). In two dimensions, the borderline exponent moves from 1 to 2.

Example 3 · Evaluating an ordinary series by swapping Compute \( \displaystyle\sum_{j=1}^\infty jx^j \) for \(|x| < 1\).

Write \(j\) as \(1 + 1 + \cdots + 1\) (\(j\) times), i.e. use the matrix with \( a_{ij} = x^j \) for \(j \ge i\) and \(0\) otherwise. Column \(j\) contains \(j\) copies of \(x^j\), so summing by columns gives our series. Summing by rows instead:

\[ \sum_{j=i}^\infty x^j = \frac{x^i}{1 - x}, \qquad \sum_{i=1}^\infty \frac{x^i}{1 - x} = \frac{x}{(1 - x)^2}. \] Everything converges absolutely (repeat with \(|x|\)), so by Theorem 3 \[ \sum_{j=1}^\infty jx^j = \frac{x}{(1 - x)^2}. \]

Example 4 · A surprising identity Show that \( \displaystyle\sum_{k=2}^\infty\Big(\frac{1}{2^k} + \frac{1}{3^k} + \frac{1}{4^k} + \cdots\Big) = 1 \).

Let \( a_{kn} = \frac{1}{n^k} \) for \(k \ge 2\), \(n \ge 2\). All terms are positive, so the order may be swapped. Summing over \(k\) first (a geometric series):

\[ \sum_{k=2}^\infty \frac{1}{n^k} = \frac{1/n^2}{1 - 1/n} = \frac{1}{n(n-1)}. \] Then \[ \sum_{n=2}^\infty \frac{1}{n(n-1)} = \sum_{n=2}^\infty\Big(\frac{1}{n-1} - \frac1n\Big) = 1. \] (In the notation \( \zeta(k) = \sum_{n\ge1}\frac{1}{n^k} \) this reads \( \sum_{k\ge2}\big(\zeta(k) - 1\big) = 1 \).)

6. Summary

Method	What is summed	Sum defined as
1° Triangles	antidiagonals \( u_n = \sum_{i=1}^n a_{i,\,n-i+1} \)	\( \lim S_n \), \( S_n = u_1 + \cdots + u_n \)
2° Finite rectangles	\( A_{nm} = \sum_{i\le n,\,j\le m} a_{ij} \)	double limit \( \lim_{n,m\to\infty} A_{nm} \)
3° Infinite rectangles	rows (or columns) to infinity first	\( \sum_i\sum_j a_{ij} \) or \( \sum_j\sum_i a_{ij} \)

The golden rule: if the sum of the absolute values is finite by any one method, all methods converge to the same sum. For non-negative terms this is automatic, and convergence just means bounded partial sums. Without absolute convergence, different methods may give different answers.

7. Practice problems

Try each one before opening the solution. More solved problems from this topic are in Series.

Problem 1: Compute \( \displaystyle\sum_{i,j=1}^\infty \frac{1}{2^i\,3^j} \).

The series factors (Example 1): \( \Big(\sum\frac1{2^i}\Big)\Big(\sum\frac1{3^j}\Big) = 1\cdot\frac12 = \frac12 \).

Problem 2: For which \(p, q\) does \( \displaystyle\sum_{i,j=1}^\infty \frac{1}{i^p\,j^q} \) converge?

The terms are positive and the rectangle sums factor into \( \Big(\sum_{i\le n}\frac1{i^p}\Big)\Big(\sum_{j\le m}\frac1{j^q}\Big) \). These are bounded exactly when \(p > 1\) and \(q > 1\) (Theorem 4).

Problem 3: Does \( \displaystyle\sum_{i,j=1}^\infty \frac{1}{i^2 + j^2} \) converge?

Since \( i^2 + j^2 \le (i + j)^2 \), we have \( \frac{1}{i^2 + j^2} \ge \frac{1}{(i+j)^2} \), and the double series \( \sum\frac{1}{(i+j)^2} \) diverges (Example 2 with \(p = 2\)). So it diverges.

Problem 4: Show that \( \displaystyle\sum_{k=1}^\infty\sum_{n=2}^\infty \frac{1}{n^{2k}} = \frac34 \).

Positive terms, so swap: \( \sum_{k\ge1}\frac1{n^{2k}} = \frac{1}{n^2 - 1} \). Then \( \sum_{n\ge2}\frac{1}{n^2-1} = \frac12\sum_{n\ge2}\Big(\frac{1}{n-1} - \frac{1}{n+1}\Big) = \frac12\Big(1 + \frac12\Big) = \frac34 \) (telescoping).

Problem 5: Using the idea of Example 3, compute \( \displaystyle\sum_{j=1}^\infty \frac{j}{2^j} \).

With \(x = \frac12\): \( \frac{1/2}{(1/2)^2} = 2 \).

Problem 6: Why doesn't the matrix in Section 3 contradict Theorem 3?

Theorem 3 requires \( \sum_j\sum_i|a_{ij}| < \infty \). For that matrix, every column \(j \ge 2\) has absolute sum \(2\), so the iterated sum of absolute values is \( 1 + 2 + 2 + \cdots = \infty \). The hypothesis fails, so the theorem says nothing, and indeed the two orders give different results.

Final thoughts

An infinite table of numbers has no single natural order, so "its sum" depends on how you walk through it: by diagonals, by growing rectangles, or row by row. The good news is a single clean rule: if the absolute values have a finite sum, the walking order doesn't matter. That rule turns the order of summation into a tool. Swapping rows and columns is one of the most effective tricks for evaluating series, as Examples 3 and 4 show.

Next in the series: series of functions and uniform convergence, then power series.