Overview

In digital image processing, it is often necessary to convert color images to grayscale images, which requires the use of the luminance equation Y = r R + g G + b B. By substituting the R, G, and B values of each pixel in the color image into the equation, the corresponding Y value can be calculated. However, there are several versions of the commonly used luminance equations, differing slightly in the coefficients r, g, and b. Many people are often troubled by the following questions:

● Where does the luminance equation come from?
● How are the coefficients of the luminance equation determined?
● Which version of the luminance equation should I use?

In fact, this is a fundamental issue in colorimetry. Let's explore it together from the very beginning.

Where does the luminance equation come from?

The world's first widely used color television standard was proposed by the US NTSC (National Television System Committee) in 1953, hence it is also known as the NTSC system. It can be said that color television was one of the earliest commercial applications of the CIE 1931 $\bar{x}(\lambda),\bar{y}(\lambda),\bar{z}(\lambda)$ standard colorimetric observer and the XYZ color space.

Figure 1 CIE 1931 xy chromaticity diagram (Image source: https://en.wikipedia.org/wiki/CIE_1931_color_space#/media/File:CIE1931xy_blank.svg)

It is not difficult to see from the CIE 1931 xy chromaticity diagram that by choosing red, green, and blue light to superimpose, it is easy to mix a wider variety of colors. Therefore, electronic image display devices have unanimously chosen these three primary colors, which is why R, G, and B appear in the luminance equation. However, before the birth of color television, black-and-white television broadcasting had already begun commercial use. The so-called “black-and-white” actually refers to grayscale images, so black-and-white television only required a single signal representing image luminance, namely Y in the luminance equation. The luminance equation was born in this context; it builds a bridge between Y and R, G, B, enabling color television signals to be compatible with black-and-white television receivers. Of course, color television receivers also need two additional signals besides Y to reproduce color images.

Television was born for broadcasting, and broadcasting is one-to-many; therefore, the cost of receivers must be as low as possible. Thus, color television signals are based on the aforementioned three display primaries, facilitating image signal processing and display by receivers. Similar to the meaning of R, G, B pixel values in digital images, the information carried by color television signals is the “amount” of these three primaries. This leads to two questions: 1. What exactly are these three primary colors? 2. What is the amount of each primary?

The answers to these two questions are “hidden” in the source standards related to color images.

Television was born for broadcasting, and broadcasting is one-to-many; therefore, the cost of receivers must be as low as possible. Thus, color television signals are based on the aforementioned three display primaries, facilitating image signal processing and display by receivers. Similar to the meaning of R, G, B pixel values in digital images, the information carried by color television signals is the “amount” of these three primaries. This leads to two questions: 1. What exactly are these three primary colors? 2. What is the amount of each primary?

The answers to these two questions are “hidden” in the source standards related to color images.

How are the coefficients of the luminance equation determined?
Taking the 1953 NTSC color television standard as an example, the standard defines the xy chromaticity coordinates of the three primaries and the reference white.

From the above chromaticity coordinates and the definition that the sum of chromaticity coordinates is 1 (i.e., x+y+z = 1), we can calculate the chromaticity coordinate z of the three primaries and list them in matrix C
$\rm{C}= \begin{bmatrix} x_{[R]} & x_{[G]} & x_{[B]} \\ y_{[R]} & y_{[G]} & y_{[B]} \\ z_{[R]} & z_{[G]} & z_{[B]} \end{bmatrix} = \begin{bmatrix} 0.67 & 0.21 & 0.14 \\ 0.33 & 0.71 & 0.08 \\ 0.00 & 0.08 & 0.78 \end{bmatrix} \tag{1}$

If color is regarded as a vector in the color space, then the above chromaticity coordinates define the directions of the three basis vectors NTSC [R][G][B] in the CIE 1931 XYZ color space, which answers the question “what exactly are these three primary colors”. However, the length of the vectors (i.e., the amount of each primary) is to be determined. We know that for colored light, the “amount” is reflected by luminance. So how is the luminance of each primary determined?
When defining a new color space or display system, for convenience, we generally follow a convention: 1 [W] = 1 [R] + 1 [G] + 1 [B], meaning that 1 unit of primary [R], 1 unit of primary [G], and 1 unit of primary [B] added together can match 1 unit (i.e., with a luminance of 1) of reference white [W], that is
$\begin{bmatrix} X_{[W]} \\ Y_{[W]} \\ Z_{[W]} \end{bmatrix} = \begin{bmatrix} X_{[R]} & X_{[G]} & X_{[B]} \\ Y_{[R]} & Y_{[G]} & Y_{[B]} \\ Z_{[R]} & Z_{[G]} & Z_{[B]} \end{bmatrix}\tag{2}$ $\frac{1}{y_{[W]}} \cdot \begin{bmatrix} x_{[W]} \\ y_{[W]} \\ z_{[W]} \end{bmatrix}= C \cdot\begin{bmatrix} k_{[R]} \\ k_{[W]} \\ k_{[W]} \end{bmatrix} = \rm{C\cdot k}\tag{3}$ $\frac{1}{0.3162} \cdot \begin{bmatrix} 0.3101 \\ 0.3162 \\ 0.3737 \end{bmatrix} = \begin{bmatrix} 0.67 & 0.21 & 0.14 \\ 0.33 & 0.71 & 0.08 \\ 0.00 & 0.08 & 0.78 \end{bmatrix} \cdot \rm{k}\tag{4}$ After solving for k in equation (4) and right-multiplying it to matrix C, we obtain $Y_{[R]} = 0.2989, Y_{[G]} = 0.5866, Y_{[B]} = 0.1144$. This is the luminance of each unit of the three primaries. Thus, we have obtained the lengths of the three basis vectors (i.e., the “amount” of each primary). By definition, the luminance equation calculates the total luminance produced when the three primaries are added and mixed in a certain proportion, which is $\rm{Y= Y_{[R]} R + Y_{[G]} G + Y_{[B]} B}$. Let's compare this with the luminance equation given in the NTSC 1953 standard, $\rm{Y = 0.299 R + 0.587 G + 0.114 B}$, which is consistent with our calculated results.

Which version of the luminance equation should I use?

From the above analysis, it can be seen that once the chromaticity coordinates of the three primaries are determined, the luminance of the three primaries can be calculated using the chromaticity coordinates of the reference white, and the luminance equation is subsequently determined. Therefore, from the perspective of colorimetry, the choice of the luminance equation depends on the chromaticity coordinates of the corresponding three primaries and the reference white, and these coordinates are available in standards for reference. However, the following two issues are worth noting. First, because the color standard of NTSC 1953 was too “idealized”, it was not widely adopted. The chromaticity coordinates of the three primaries and reference white actually used in analog color television broadcasting differ from NTSC 1953, yet the luminance equation from the NTSC 1953 standard was adopted by these standards. Obviously, from a colorimetric perspective, this is incorrect. Second, the R, G, B components in most color television systems are the products of non-linear gamma correction (OETF). Therefore, in television engineering, the non-linear R', G', B' values are substituted into the luminance equation calculated using colorimetric methods. Obviously, this is also incorrect.

Obviously, both of the above issues will result in the Y calculated by the luminance equation failing to accurately represent luminance. However, from the perspective of “alignment” with standards, one has to refer to the standards and make the best of a bad situation. After all, as the use of electronic imaging has expanded from television broadcasting to various industries, this error has long been “deeply rooted” and “widespread”.