1. Why is a Bayer Filter Needed?
The core of an image sensor is a silicon-based photodiode, which can only measure the amount of charge generated by incident light and cannot directly perceive the color of the light. When photons are absorbed by the silicon material, electron-hole pairs are generated, forming a charge signal. Regardless of whether these photons come from red, green, or blue light, the generated charges are electrically indistinguishable. The photoelectric conversion process only reflects the intensity of the incident light and cannot retain the wavelength information of the photons. Therefore, to generate a color image, it is necessary to acquire signals for the red (R), green (G), and blue (B) channels.

Early color imaging mostly used a dichroic prism to split light into three RGB paths, projecting them onto separate sensors. This approach offers high color accuracy and requires no interpolation, but it is bulky, expensive, has a complex optical path, and is difficult to align, making it hard to meet the demands of consumer electronics and mass production.

To solve this problem, Bryce Bayer, an engineer at Eastman Kodak, proposed a patented solution in 1976: the Bayer filter. A CFA (Color Filter Array) is overlaid on the top surface of a single sensor, spatially sampling in an interleaved arrangement with an R:G:B ratio of 1:2:1. Each pixel records only one color to achieve color sampling, and a color image is then reconstructed via algorithms. This solution is compact, low-cost, and easy to mass-produce, quickly becoming the mainstream approach. (Image source: https://en.wikipedia.org/wiki/File:Bayer_pattern_on_sensor.svg)

2. How Does the Bayer Filter Achieve Color Imaging?
The Bayer filter achieves this by covering different pixels with R, G, and B filters, allowing each pixel to receive only the monochromatic luminance value of the corresponding wavelength band.
The data output by the sensor is not a complete RGB image, but a mosaic of monochromatic sampled values, known as RAW data. To obtain a full-color image, a demosaicing algorithm is required to estimate the missing color components based on neighboring pixel information, thereby reconstructing the RGB values for each pixel.
(Image source: https://en.wikipedia.org/wiki/Bayer_filter#/media/File:Bayer_pattern_on_sensor_profile.svg)

3. Why Adopt the Bayer Pattern?
The Bayer pattern is the spatial arrangement of color filters on the pixel array in a Bayer filter, used to define the distribution rules of the red (R), green (G), and blue (B) filters.

To balance luminance resolution and color information with a limited number of pixels, the Bayer pattern adopts an R:G:B sampling ratio of 1:2:1, where green pixels account for 50% of the total pixels, and red and blue pixels each account for 25%. This design stems from the fact that the human visual system is far more sensitive to changes in luminance than to changes in color, and the luminance information in an image is primarily contributed by the green channel. Therefore, increasing the sampling density of green pixels can achieve higher luminance resolution and detail rendition with the same number of pixels, while maintaining color reconstruction accuracy.

Common Bayer patterns include: RGGB, BGGR, GBRG, and GRBG. These arrangements have the same sampling ratio and density, differing only in the starting position or orientation of the filters. The specific arrangement adopted depends on the sensor design and ISP configuration.

4. What Challenges Does the Bayer Filter Face?
(1) Loss of Light Utilization
Each pixel only allows light of a specific wavelength band to pass through, while the rest of the spectrum is filtered out. Consequently, a large amount of incident light cannot be utilized, leading to a decrease in photon utilization efficiency, which is a key factor limiting the low-light performance of imaging systems.

(2) Sampling and Interpolation Artifacts
Due to the limited spatial sampling frequency, aliasing occurs when capturing high-frequency textures, manifesting as moiré patterns. Meanwhile, incorrect interpolation by the demosaicing algorithm in edge regions can lead to false color artifacts and zippering artifacts.

False color artifact:
(Image source: https://en.wikipedia.org/wiki/Bayer_filter#/media/File:False_colour_artifact.JPG)

Zippering artifact:
(Image source: https://en.wikipedia.org/wiki/Bayer_filter#/media/File:Zippering_artifact.JPG)

5. Evolution of the Bayer Filter
New color filter arrays (CFAs) replacing the Bayer pattern:

• RGBW array: Introduces colorless, transparent white pixels (White) into the pixel array. White pixels do not filter the spectrum, allowing full-band light to pass through, which can increase light intake by up to 60%, significantly reducing noise in night scenes when combined with algorithms.
• RYYB array: Replaces green filters with yellow filters (Yellow). Yellow filters allow both red and green light to pass through simultaneously, increasing light intake by about 40% compared to traditional RGB, greatly enhancing low-light performance in night scenes.