Chapter 13. Sensor Fusion: Integrating Radar, EO/IR, and ESM Data

Sensor fusion: Radar·EO/IR·ESM data integration The secret to winning an air battle is simple. Look first, shoot first. However, “seeing” is a more complicated matter than you might think. The human pilot's eyes can only see the sky outside the canopy. This is only effective on a clear day and when there are enemies within visible range. Modern air combat is decided hundreds of kilometers away. Radars detect the enemy, missiles are launched, and the enemy plane is shot down long before the enemy can be seen with the naked eye. This is called “Beyond Visual Range (BVR).” The problem is that no sensor is perfect. Let's take radar as an example.

Radar emits radio waves and detects reflected waves. No matter how far away the enemy is, you can know their location and speed when the radio waves that hit the metal surface bounce back. However, stealth fighters are designed to minimize radar reflection. So it doesn't really go under the radar. Additionally, because radar emits radio waves, it also reveals your location. If the enemy's ESM (electronic warfare support equipment) traces the radio waves back, the hunter may become the hunted. Electro-optical/infrared (EO/IR) sensors see the enemy in a different way.

It detects heat from the enemy aircraft's engines and heat from gas and air friction. It is stealthy as it does not emit radio waves. Even stealth aircraft cannot hide engine heat, so it can be picked up by IR sensors. However, its distance measurement accuracy is lower than that of radar, and it is vulnerable to clouds and bad weather. ESM passively listens to radio waves emitted by enemies. The moment the enemy radar is turned on, the frequency pattern is analyzed to determine the enemy's location and type. But it's no use if the enemy has their radar turned off. Each sensor has both advantages and disadvantages.

Radar knows the distance well but is vulnerable to stealth, IR is good at seeing heat but does not know the distance, and ESM knows the type of enemy but has difficulty in precise aiming. It is nearly impossible for a human pilot to mentally integrate data from all these sensors. You have to look at the radar screen, check the IR image, listen to the RWR (Radar Warning Receiver), and make a decision by combining all that information in a split second. This is the limit of human “cognitive load.” This is where the real power of AI is revealed. “Sensor Fusion”.

AI simultaneously analyzes and integrates data from radar, EO/IR, and ESM to create a complete situational diagram. The radar reports “object 50 kilometers ahead,” and the IR sensor reports “object 48 kilometers ahead.”

When a "circle detection" is reported, the AI determines whether it is the same enemy aircraft or two different objects. It corrects for slight errors in position and speed and combines them into a single "track". This is called "data association". The next step is to identify the enemy. It analyzes the radar frequency pattern collected by the ESM. The first identification is made by saying, "This signal is the radar of a Russian Su-35". The enemy aircraft's speed measured by the radar and The F-35 Lightning II is the culmination of this sensor fusion technology.

The AN/APG-81 AESA radar, AN/AAQ-40 EOTS (Electro-Optical Targeting System), AN/AAQ-37 DAS (Distributed Aperture System), and AN/ASQ-239 electronic warfare equipment are mounted on the aircraft. The 360-degree image is displayed on the helmet display no matter where the pilot looks. When you look back, the AI fuses all of this sensor data into a single, unified picture of the battlefield on the pilot's helmet. What happens when an enemy tries to jam it? The human pilot is confused, but the AI cross-checks the ESM data and the IR image.

If there is no heat signature, the AI determines it is a "decoment" or an "illusion" and erases it from the screen. Even if the enemy tries to fool the IR sensor with flares, it can distinguish between real enemy aircraft and decoys by comparing them with radar data. FCAS and GCAP, a European 6th generation fighter aircraft project, are taking this one step further by sharing sensor information among all manned and unmanned aircraft in the squadron. It aims for "fusion".

The leading drone turns on the radar to detect the enemy (taking the risk of revealing its location), and the manned aircraft behind turns off the radar and receives the information and fires a missile. Although it is not visible to the sensors of the plane, it can target the enemy hidden behind the mountain using the data sent by the friendly drone. In the end, sensor fusion is a technology that clears the fog of war. The most scary thing in war is not the enemy. It is not knowing where the enemy is. Fighting in situations where you don't know what anti-personnel or armed forces are.

If you do this, no matter how good your weapon is, it will be of no use. AI’s sensor fusion sheds light on the darkness of that ignorance. It filters out noise from the flood of sensor data, distinguishes between false targets, and provides a single, integrated picture that pilots can trust. This is the real power of 5th and 6th generation fighters. It's not stealth. It's fusion.