Radio Frequency Interference in Microwave Radiometry: Statistical Analysis and Study of Techniques for Detection and Mitigation

Ph. D. Thesis Dissertation

by Jose Miguel Tarongi Bauza Dept. of Signal Theory and Communications Universitat Politecnica de Catalunya – Barcelona Tech.

Barcelona, October 2012

Advisor: Prof. Adriano Jose Camps Carmona Dept. of Signal Theory and Communications Universitat Politecnica de Catalunya – Barcelona Tech.

Acknowledgements
This work has been conducted as part of the award "Passive Advanced Unit (PAU): A Hybrid L-band Radiometer, GNSS Reflectometer and IR-Radiometer for Passive Remote Sensing of the Ocean" made under the European Heads of Research Councils and European Science Foundation European Young Investigator (EURYI) Awards scheme in 2004, and it was supported in part by the Participating Organizations of EURYI, and the EC Sixth Framework Program. It has also been supported by the Spanish National Research and EU FEDER Projects TEC2005-06863-C02-01 and AYA2008-05906-C02-01/ESP.

First of all I would like to express my gratitude to Prof. Adriano Camps, for giving me the possibility to do my Ph. D. thesis in the remote sensing group as well as for his guidance, dedication, advices, and funding during these years; and to Merce Vall-llosera for her financial support in this project.

I would like to express my gratitude in a special manner to Joaquim Giner, Ruben Tardío and Albert Martón for the collaboration on the development of the MERITXELL instrument that would not been possible without his enthusiastic support in the project.

I would like to express my gratitude to Isaac and Giuseppe for their contributions in the MERITXELL development.

I would like to also thank to Veronica Depau for her contributions in the study of the RFI mitigation algorithms during her final project under my supervision.

Thanks to all the people of the Radiometry team especially to: Enric, Neri, Xavi, JuanFer, Marco Talone, Sandra, Alberto, Roger, Fran, Hugo, Dani, and Hyuk for their support.

I would like to thanks my office mates for the great understanding, especially to: Pere, Santi, Javi, Sergi, Marta, Dani, Rene, Vero, Juan Carlos (King and Prince), Raquel, Oscar, Marc, Enrique, and Jordi.

Thanks to the UPC people that have helped me during this time: Josep Pastor, Teresa, Aynie, Josep M. Haro, and Alicia.

Finally I would like to thank to all the people of the Department of Signal Theory and Communications (TSC) for its support during these years.

Dedicado a mi madre y a Pilar por su paciencia y sus ánimos durante el transcurso de esta tesis Dedicado también a todos los que me han ayudado de una alguna forma en el desarrollo de esta tesis

5

Preface
Microwave radiometry field has been increasing its performance with higher accuracy measurements, leading to a more presence in the remote sensing field. Several space-borne, air-borne and ground-based radiometers have been developed to perform measurement campaigns; however, the actual sensitivity of a radiometer is often limited by man-made radio emissions such as radars, broadcasting emissions, wireless communications and many other communication systems based on electromagnetic waves, limiting the improvement in the radiometers’ performance. Consequently, in order to maintain the accuracy in the radiometric measurements, it has been researched in the Radio Frequency Interference (RFI) detection and mitigation systems and algorithms for the microwave radiometry field.

The scope of this doctoral thesis is the development, testing and comparison of different RFI detection and mitigation algorithms based on several methods, such as time and frequency domain, wavelet and statistical analysis of the retrieved radiometric signal; some of the algorithms studied in this thesis belong to the state-of-the-art and others have been developed during this thesis. The comparison of the different RFI detection and mitigation algorithms have the final purpose of enhancing the radiometric measurements performed by the Multifequency Experimental Radiometer with Interference Tracking for Experiments over Land and Littoral (MERITXELL). The MERITXELL has been developed during this thesis with the idea studying the RFI present in several radiometric bands and the way to mitigate it, as well as to obtain data from diverse frequency bands and devices in only one measurement campaign.

7

Contents

Contents
Part I. Introduction 1. Microwave Radiometry basics ................................................................................ 13 1.1 Microwave radiometry and applications.......................................................... 14 1.2 Basic concepts on microwave radiometry ........................................................ 14 1.2.1 Thermal Radiation ...................................................................................... 15 1.2.2 Planck’s Radiation Law and black body radiation .................................. 15 1.2.3 Brightness temperature............................................................................... 17 1.2.4 Antenna Surrounded by a Black Body ...................................................... 18 1.2.5 Gray Body Radiation .................................................................................. 20 1.2.6 Apparent Temperature ............................................................................... 20 1.2.7 Emission theory ........................................................................................... 22 1.2.7.1 Emission from a specular surface ....................................................... 23 1.2.7.2 Emission from a perfectly rough surface ........................................... 24 1.3 Types of microwave radiometers ...................................................................... 24 1.3.1 Total Power Radiometer (TPR) ................................................................. 25 1.3.2 Dicke Radiometer (DR)............................................................................... 29 1.3.3 Noise Injection Radiometer (NIR) ............................................................. 31 1.4 Conclusions ......................................................................................................... 32 2. Radio Frequency Interference in Microwave Radiometry Measurements......... 33 2.1 Soil Moisture and Ocean Salinity (SMOS) RFI contamination case ............. 35 2.2 WindSat RFI contamination case ..................................................................... 36 2.3 AMSR-E RFI contamination case..................................................................... 37 2.4 General RFI contamination cases ..................................................................... 38 2.4.1 RFI originated by 24-GHz automotive radars.......................................... 38 2.4.2 RFI present in the calibration process....................................................... 38 2.5 Conclusions ......................................................................................................... 39 Part II: Theoretical Analysis 3. State of the Art in Radio Frequency Interference Detection and Mitigation Systems .......................................................................................................................... 41 3.1 RFI detection and mitigation in the time and frequency domains................. 42 3.1.1 RFI Detection in the Time domain............................................................. 43 3.1.2 RFI Detection in Frequency domain.......................................................... 44 3.1.3 RFI Detection in the combination of Frequency and Time-domains ..... 45 3.2 RFI detection by Statistical Methods................................................................ 45 3.3 Suppression using Filtering Techniques........................................................... 47 3.4 Adaptive Interference Cancellation using Reference Channels..................... 48 3.5 Spatial Filtering using Multi-element Systems ................................................ 49

9

Contents

3.6 Conclusions ......................................................................................................... 50 4. Normality Analysis for RFI Detection in Microwave Radiometry ...................... 51 4.1 Normality tests .................................................................................................... 52 4.1.1 Kurtosis test ................................................................................................. 53 4.1.2 Skewness test ................................................................................................ 54 4.1.3 Jarque-Bera test (JB) .................................................................................. 55 4.1.4 D’Agostino K-squared test (K2)................................................................. 55 4.1.5 Kolmogorov-Smirnov (KS) and Lilliefors (L) tests.................................. 56 4.1.6 Anderson-Darling test (AD)........................................................................ 57 4.1.7 Shapiro-Wilk test (SW) ............................................................................... 57 4.1.8 Cramer-von Mises test (CM) ...................................................................... 58 4.1.9 Lin-Mudholkar test (LM) ........................................................................... 59 4.1.10 Chi-square test (CHI2).............................................................................. 59 4.2 Validation of Normality Tests in the Absence of RFI ..................................... 60 4.3 Performance of Normality Tests for Different Types of RFI ......................... 64 4.3.1 Pulsed Sinusoidal and Chirp Signal........................................................... 67 4.3.2 Pseudo-random noise signal ....................................................................... 69 4.3.3 Pulsed PRN signal........................................................................................ 71 4.3.4 Telegraphic signal........................................................................................ 71 4.3.5 Pulsed Telegraphic Signal........................................................................... 74 4.3.6 Pulsed OFDM signal.................................................................................... 75 4.4 Blind spot detection in Kurtosis based algorithms.......................................... 76 4.5 Conclusions ......................................................................................................... 79 5. RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis ...... 81 5.1 Spectrogram calculation .................................................................................... 83 5.2 Spectrogram analysis algorithms ...................................................................... 86 5.2.1 Smoothing algorithm description............................................................... 86 5.2.2 Wiener Filter Algorithm description ......................................................... 89 5.2.3 Edge detection algorithm description ........................................................ 92 5.2.4 Frequency/Time Averaging and Thresholding (FIAT) Algorithm description ............................................................................................................. 95 5.3 Simulation results ............................................................................................... 98 5.3.1 Smoothing algorithm simulation results ................................................... 99 5.3.1.1 Chirp and sinusoidal RFI tests............................................................ 99 5.3.1.2 Broadband PRN and OFDM RFI tests ............................................ 107 5.3.2 Wiener filter algorithm simulation results.............................................. 115 5.4 Conclusions ....................................................................................................... 121 6. RFI Mitigation in Microwave Radiometry using Wavelets................................ 123 6.1 Principles of Denoising..................................................................................... 125 6.2 Optimum Parameters Selection ...................................................................... 128 6.2.1 Threshold selection and Sequence length................................................ 128 6.2.2 Decomposition level ................................................................................... 129 6.2.3 RFI mitigation performance vs. interference-to-noise ratio.................. 130 6.3 Conclusions ....................................................................................................... 132

10

Contents

Part III: Experimental Results 7. MERITXELL: The Multifrequency Experimental Radiometer with Interference Tracking for Experiments over Land and Littoral ................................................. 133 7.1 General Overview ............................................................................................. 134 7.2 Specific Description .......................................................................................... 138 7.2.1 Radio-Frequency part ............................................................................... 139 7.2.2 Instrumental part ...................................................................................... 146 7.2.2.1 Spectrum Analyzer ............................................................................. 146 7.2.2.2 Cameras............................................................................................... 148 7.2.2.3 Inclinometer ........................................................................................ 149 7.2.2.4 Peripheal Interface Controller (PIC) devices .................................. 149 7.2.2.5 Computer devices ............................................................................... 150 7.2.3 Temperature control part ......................................................................... 151 7.2.4 Electrical part ............................................................................................ 153 7.2.5 Structural part ........................................................................................... 158 7.3 Conclusions ....................................................................................................... 160 8. RFI detection algorithms applied to radiometric data ....................................... 163 8.1 Description of the method to retrieve the radiometric measurements ........ 164 8.1.1 Using the ADC to retrieve data ................................................................ 164 8.1.2 Using the Spectrum Analyzer to acquire data ........................................ 165 8.2 RFI detection algorithms with I/Q data ......................................................... 166 8.2.1 Normality test-based algorithms .............................................................. 166 8.2.1.1 Normality tests in RFI free radiometric data .................................. 166 8.2.1.2 Normality tests in RFI contaminated radiometric data.................. 171 8.2.2 Spectrogram-based algorithms ................................................................ 181 8.2.2.1 Spectrogram-based algorithms with RFI-free radiometric data ... 181 8.2.2.1.1 Smoothing Algorithm behaviour with RFI-free data............... 182 8.2.2.1.2 FIAT Algorithm behaviour with RFI free data........................ 184 8.2.2.1.3 Combined Smoothing and FIAT algorithms behaviour with RFI-free data............................................................................................... 185 8.2.2.2 Spectrogram-based algorithms with contaminated data................ 187 8.2.2.2.1 Spectrogram-based algorithms behaviour with low RFI contaminated radiometric data ................................................................. 188 8.2.2.2.2 Spectrogram-based algorithms behaviour with medium RFI contaminated radiometric data ................................................................. 194 8.2.2.2.3 Spectrogram-based algorithms behaviour with high RFI contaminated radiometric data ................................................................. 199 8.2.2.2.4 Improvements of the Spectrogram-based algorithms for large RFI contaminated radiometric data ......................................................... 203 8.3 RFI detection algorithms with the power spectrograms............................... 210 8.3.1 Algorithms using power spectrograms with RFI-free radiometric data....................................................................................................................... 211 8.3.1.1 Edge Detector algorithm with RFI-free radiometric data.............. 211 8.3.1.2 FIAT Algorithm behaviour with RFI-free radiometric data ......... 212 8.3.1.3 Behaviour of the combined Edge Detection and FIAT algorithms with RFI-free radiometric data ..................................................................... 213 8.3.2 Algorithms using power spectrograms with RFI contaminated radiometric data ................................................................................................. 214

11

Contents

8.3.2.1 Behaviour of the power spectrograms based algorithms with an entire L-band measurement .......................................................................... 214 8.3.2.2 Behaviour of the power spectrogram based algorithms with a 2 MHz sub-band of the L-band (3rd sub-band) ........................................................ 219 8.3.2.3 Behaviour of the power spectrogram based algorithms over an entire L-band measurement with a long sweep time.............................................. 224 8.4 Optimal Pfa for the FIAT algorithm performance ........................................ 229 8.5 Conclusions ....................................................................................................... 238 9. Conclusions and future research lines .................................................................. 241 9.1 Conclusions and Summary .............................................................................. 242 9.1.1 Background presentation.......................................................................... 242 9.1.2 RFI detection algorithms .......................................................................... 242 9.1.3 Experimental results ................................................................................. 244 9.2 Future research lines ........................................................................................ 246 9.3 List of publications ........................................................................................... 247 9.3.1 Journal articles .......................................................................................... 247 9.3.2 Conference proceedings ............................................................................ 248 Bibliography................................................................................................................ 251 List of Acronyms......................................................................................................... 263

12

Microwave Radiometry basics

Chapter 1
1. Microwave Radiometry basics

This chapter provides an introduction to microwave radiometry and emission theory presenting and

developing the most important concepts on these fields. In addition, the main applications of microwave radiometry, and its frequency allocations are presented. Finally, the most typical types of radiometers are presented: the total power radiometer (TPR), the Dicke radiometer (DR), and the noise injection radiometer (NIR).

13

Chapter 1

1.1 Microwave radiometry and applications
Microwave radiometry is the science that studies and measures the spontaneous emission of electromagnetic energy radiated by all bodies at a physical temperature different of 0 K (indeed all bodies are at higher temperatures). This technique was born in radio-astronomy to measure the electromagnetic emissions coming from the outer space. Since the 1960’s it has become a common and powerful tool for Earth remote sensing. With the study and analysis of the physical processes related with this spontaneous emission, it is possible to infer the atmospheric or geophysical parameters that have caused it. There are many microwave radiometry applications. Mainly, they can be included in two groups: atmospheric applications and Earth’s surface applications. The main applications and their suitable frequencies are listed below (Table 1.1).

Table 1.1: Relationship between radiometry applications and their suitable frequencies [1]

Application Clouds water content Ice classification Sea oil spills tracking Rain over soil Rain over the ocean Sea ice concentration Sea surface salinity Sea surface temperature Sea surface wind speed Snow coating Soil moisture Atmospheric temperature profiles Atmospheric water vapour

Frequency (GHz) 21, 37, 90 10, 18, 37 6.6, 37 18, 37, 55, 90, 180 10, 18, 21, 37 18, 37, 90 1.4, 6.6 6.6, 10, 18, 21, 37 10, 18 6.6, 10, 18, 37, 90 1.4, 6.6 21, 37, 55, 90, 180 21, 37, 90, 180

1.2 Basic concepts on microwave radiometry
In this section the main concepts on microwave radiometry are presented and discussed.

14

Microwave Radiometry basics

1.2.1 Thermal Radiation
All bodies receive electromagnetic energy from its surroundings. Part of this incident energy is reflected on the body’s surface. The rest is absorbed and, in thermodynamic equilibrium, re-radiated. According to Bohr’s equation, the frequency f of the emitted radiation by a determined body is given by:

f =

ε1 − ε 2
h

[Hz],

(1.1)

where ε1 and ε2 are different energy levels in Joules ([J]), and h is the Planck's constant (h = 6.63·10-34 J·s).

The emission of radiation is caused by electrons changing its energy level. The probability of emission is a function of the density of the particles and the kinetic energy of their random motion. The radiated intensity increase is then proportional to the increase of its absolute temperature.

1.2.2 Planck’s Radiation Law and black body radiation
In general, part of the electromagnetic energy incident on a surface is absorbed, and part is reflected. In thermodynamic equilibrium the amount of absorbed energy reequals the amount of radiated energy.

A black-body is defined as an ideal body that absorbs all of the incident electromagnetic energy (at all wavelengths, directions and polarizations) and, in thermodynamic equilibrium, reradiates it back. According to Planck’s law, a black-body radiates uniformly towards all directions with a spectral brightness density given by eqn. (1.2):

Bbb ( f ) =

2hf 3 c2

1 e
hf k BTph

[Wm-2Hz-1sr-1],

(1.2)

−1 the speed of light in the vacuum

where Bbb is the black-body spectral brightness density, h is the Planck's constant, f is the frequency, c is (c = 299.792.458 [m·s-1]), kB is the Boltzmann constant (kB = 1.38·10-23 [J·K-1]), and Tph

15

Chapter 1

is the absolute physical temperature of the body in Kelvin. Natural surfaces absorb only a fraction of the incident power, the rest being reflected.

Figure 1.1: Plank’s radiation law [2]

In order to simplify eqn. (1.2), if hf/kBTph is much lower than 1, Taylor’s approximation can be applied to the exponent in the denominator of eqn. (1.2):
ex = ∑
n=0 ∞

xn ≈x n!

x

1.

(1.3)

At low microwave frequencies the Rayleigh-Jeans law can then be used as good approximation of the Planck’s law and can be written as:

2hf 3 Bbb ( f ) = 2 c

2 f 2 k BTph 2k BTph 1 ≈ = [Wm-2Hz-1sr-1]. hf c2 λ2 k BTph

(1.4)

At optical frequencies the Planck’s law reduces to Wien’s law:

2hf 3 − k T Bbb ( f ) = 2 e B ph [Wm-2Hz-1sr-1]. c

hf

(1.5)

16

Microwave Radiometry basics

Figure 1.2: Comparison of Planck’s law with its low-frequency (Rayleigh-Jeans law) and high frequency (Wien’s law) approximations at 300 K [2].

Figure 1.2 compares the approximations defined in eqns. (1.4) and (1.5) with Planck’s law, the higher the physical temperature, the higher the brightness and the frequency where the brightness reaches its maximum. The Stefan-Boltzmann law provides an expression for the total brightness, and it is obtained by integrating eqn. (1.2) over all the spectra:
Bbb = ∫ Bbb ( f )df =
0 ∞

σ Tph 4 π

[Wm-2Hz-1sr-1],

(1.6)

where σ = 5.637·10-8 [Wm-2K-4sr-1] is the Stefan-Boltzmann’s constant.

1.2.3 Brightness temperature
The brightness temperature can be defined as the power emitted by a body by unit solid angle and by unit surface. If the emitting surface radiates with a pattern
Ft(θ,φ), the brightness B(θ,φ) is then given by:

B (θ , ϕ ) =

Ft (θ , ϕ ) At

[Wsr-1m-2],

(1.7)

where At is the total area which is radiating. The power collected by an antenna surrounded by a distribution of incident power B can be computed as: P = Ft Aeff R
2

= BAt

Aeff R2

[W],

(1.8)

17

Chapter 1

being Aeff the effective area of the antenna, and R the distance to the radiating surface. Taking into account that the solid angle Ωt subtended by the transmitting antenna is defined as: Ωt = At , R2 (1.9)

then, the power collected by the antenna can be computed as: P = BAeff Ωt
[W].

(1.10)

Replacing the solid angle by a differential solid angle dΩ, the corresponding power received by the antenna from an extended source of incident brightness B(θ,φ) can be expressed as:
dP = Aeff B (θ , ϕ ) Fn (θ , ϕ ) ,
2

(1.11)

where |Fn(θ,φ)|2 is the normalized antenna radiation pattern. The total power collected by the antenna is then obtained by integrating eqn. (1.11) over the space and over the system’s bandwidth, as brightness can vary with frequency.
f +B / 2

1 P = Aeff 2

f −B / 2 4

∫ ∫∫ B ( f ,θ , ϕ ) F (θ ,ϕ ) π
n

2

d Ωdf

[W],

(1.12)

where B is the bandwidth of the receiving system. Since the antenna collects only half of the randomly polarized thermal power emitted, the collected power must be multiplied by a factor of ½.

Figure 1.3: Geometry of the radiation incident over the antenna [2]

1.2.4 Antenna Surrounded by a Black Body
Assuming a lossless antenna with a normalized radiation pattern |Fn(θ,φ)|2, a bandwidth B around a working frequency f, surrounded by a black-body at a constant
18

Microwave Radiometry basics

physical temperature Tph, as shown in Fig. 1.4, the power received by the antenna in Fig. 1.3 can be obtained using the eqns. (1.2) and (1.6), and it is described by:
Pbb = 1 Aeff 2
f +B / 2

f −B / 2 4

∫ ∫∫ π

2k BTph

λ

2

Fn (θ , ϕ ) d Ωdf
2

[W]

(1.13)

The detected power is limited by the receiver’s bandwidth B. If this bandwidth is small enough to assume that the spectral brightness density does not change over the frequency range, eqn. (1.13) reduces to:

Pbb = k BTph B which leads to:
Pbb = k BTph B

Aeff

λ

2

∫∫ F (θ , ϕ ) π
n 4

2

dΩ

[W],

(1.14)

Aeff

λ2

Ωeff = k BTph B

[W],

(1.15)

when integrating the normalized antenna radiation pattern over the entire space and using Ωeff = λ2/Aeff. The result in eqn. (1.15) was also found by Johnson and Nyquist in 1928 when calculating the available thermal noise power from an electrical resistor at a physical temperature Tph.

Figure 1.4: Antenna surrounded by an ideal black-body has the same delivered power than a resistor maintained at the same Tph (assuming each one is connected to a matched receiver of bandwidth B) [2]

Equation (1.15) shows that the power in a bandwidth B received by a lossless antenna surrounded by a black-body is linearly dependent on the physical temperature of the body. Using Johnson’s and Nyquist’s results, an ideal receiver with a bandwidth
B collects as much power from a matched resistor at a physical temperature Tph than

from a lossless antenna connected to it. This means that, for an ideal receiver of bandwidth B, the antenna delivers to the load the same power as a resistor at a temperature TA, which is called the antenna temperature (Fig. 1.4).

19

Chapter 1

1.2.5 Gray Body Radiation
A black-body is an idealized body and it is a perfect absorber and emitter. These bodies absorb all the incident energy at all frequencies directions and polarizations, and when the thermodynamic equilibrium is reached at a physical temperature Tph, the energy is radiated back without any preferred direction. However, actual materials (usually called gray-bodies) emit less energy than a black-body, since they are not capable to absorb all the energy incident on them. If the emitted brightness depends on the direction B(θ,φ), a similar equation to eqn. (1.4) can be defined as:

B (θ , ϕ ) =

2k B

λ2

TB (θ , ϕ ) B

[Wm-2Hz-1sr-1],

(1.16)

where TB(θ,φ) is the equivalent temperature associated to the brightness and it is called the brightness temperature. Emissivity is defined as: e (θ , ϕ ) = B (θ , ϕ ) TB (θ , ϕ ) , = Bbb Tph (1.17)

where Bbb is the brightness of the black-body at temperature Tph. The brightness temperature emitted by a black-body coincides with its physical temperature, hence its emissivity is 1. Consequently the brightness temperature emitted by real bodies is less than the physical temperature, and then their range of emissivity values is between 0 and 1. In conclusion, the emissivity of a perfect reflecting material is equal to zero, and the emissivity of a black body is one.

1.2.6 Apparent Temperature
In a real measuring environment, it is not possible to isolate the brightness temperature of the target from other sources that radiated energy. In this situation, it is convenient to define another magnitude, the apparent brightness temperature TAP(θ,φ) which accounts for different sources of thermal noise radiating over the antenna. Figure 1.5 shows the relationship between them, in this case, the apparent temperature TAP is the key parameter which can be defined as:
TAP = TUP + 1 (TB + TSC ) , La (1.18)

20

Microwave Radiometry basics

where TB is the brightness temperature of the surface under observation, TUP is the atmospheric upwards radiation, TSC is the atmospheric downwards radiation scattered reflected by the surfaces and La are the atmospheric losses. When the atmospheric losses are high, the apparent temperature is almost equal to the atmospheric physical temperature, which happens at high frequencies or at the absorption peaks of some gases. In the frequency range from 1 GHz to 10 GHz losses for a cloud-free atmosphere are very small and can be mostly neglected, consequently, in this case the apparent brightness temperature (TAP) can be approximated by the brightness temperature (TB).

Figure 1.5: Relationship between antenna temperature, apparent temperature and brightness temperature [2].

According to Fig. 1.5 and taking into account the normalized antenna pattern |Fn(θ,φ)|2, and the antenna pattern solid angle Ωp, the antenna temperature is given by: TA = 1 Ωp

∫∫ T (θ , ϕ ) F (θ , ϕ ) π
AP n

2

d Ω [K].

(1.19)

4

Since in reality the antenna absorbs a certain amount of the power incident on it, and hence it also introduces some additional noise, the resultant antenna temperature including losses is defined as:
TA′ = ηΩTA + (1 − ηΩ ) Tph ,

(1.20)

where TA′ is the equivalent apparent temperature at the antenna output including losses, ηΩ is the efficiency of the antenna, and Tph is the physical temperature of the antenna.

21

Chapter 1

1.2.7 Emission theory
When an electromagnetic wave propagating through a medium with an index of refraction n1 reaches a surface of a different medium with a index of refraction n2, part of the power of the electronic wave will be transmitted through this medium, and another part will be reflected. If this surface is perfectly flat, the reflected wave will follow the specular direction only, on the other hand, if the reflection of the incident wave is not produced over a flat surface, the incident power will be scattered over the space. Some of the scattered power maintains the phase, and it is reflected in the specular direction (coherent term), but the rest of the radiation loses its phase and polarization characteristics and it is scattered. Part of the power transmitted through the body is absorbed, thus this power will be emitted as radiation. This radiated power passes through the interface surface, and it is transmitted over a range of directions, similar to the reflection case. In Fig. 1.6 some of these cases are illustrated.

Figure 1.6: Specular and rough surface scattering and emission. a) specular reflection; b) diffuse scattering; c) diffuse emission; d) contributions to TB coming from many directions [2].

Consequently, TB(θ; p) has contributions coming from several directions of the inner part of the body. As it has been previously mentioned, the emissivity links the capability of a surface to emit and absorb radiation. Moreover its value has a dependency with the incidence angle, polarization, and the surface roughness. This section is devoted to present the emissivity of two extreme and idealizes cases: specular surface and completely rough surface.

22

Microwave Radiometry basics

The scattering of a rough surface can be modeled by its cross-section by the unit area as σ0(θ0, φ0, θs, φs, ps). This parameter relates the scatter power in the (θs, φs) direction with polarization ps for an incident plane wave at the (θ0, φ0) direction with polarization p0. When the p0 and ps are the same, σ0pp is called horizontal or vertical scattering coefficient, where pp designates the same polarization. If p0 and ps are different, σ0pq is called the cross-polar scattering coefficient, where pq indicates different polarizations (incident wave at p-polarization, scattered wave at qpolarization). The general expression for the emissivity is presented in eqn. (1.21) [2]:
2 1 0 σ 0 + σ pq  sinθ s dϕ s dθ s . e (θ 0 , ϕ0 ; p0 ) = 1 − pp  4π cos θ 0 ϕs∫ 0 θs∫ 0  = = 2π

π

(1.21)

1.2.7.1 Emission from a specular surface
The scattering produced at the specular surface consists of the coherent reflection of the incident wave only. Consequently, the cross-polar scattering coefficient σ0pq is zero, and the horizontal or vertical polarization scattering coefficients become delta functions:
cos θ 0 δ (θ s − θ sp ) δ (ϕ s − ϕ sp ) , cos θ sp

σ 0 = 4πΓ (θ 0 ; p0 ) pp

(1.22)

where Γ is the specular reflection coefficient at p0 polarization, and the subindex sp in the angles denotes the specular direction:

θ sp = θ 0
the next expression is obtained:

and ϕ sp = π − ϕ0 .

(1.23)

Substituting eqn. (1.22) in (1.21), and after some straightforward manipulations
e (θ 0 , ϕ0 ; p0 ) = 1 − Γ (θ 0 ; p0 ) .

(1.24)

It is the ideal case, when the reflection is specular, the emissivity can be expressed as a function of the reflection coefficient.

23

Chapter 1

1.2.7.2 Emission from a perfectly rough surface
When the incident wave is not reflected over plane surface it produces a diffuse scattering and its power is scattered over the space (Fig. 1.6b). Some of the scattered power remains with the same phase and the other part is changed. The extreme case is when the plane surface is perfectly rough. In this particular case the scattering surface is called a Lambertian surface, and the scattering coefficient depends only on the product cos θ0 cos θs.

σ 0 + σ 0 = σ 00 cos θ 0 cos θ s , pp pq

(1.25)

where σ00 is a constant related to the dielectric properties of the scattering surface. Substituting eqns. (1.25) in (1.21) the emissivity is obtained:
2 σ0 1 0 e (θ 0 , ϕ0 ; p0 ) = 1 − σ 0 cos θ 0 cos θ s sinθ s dϕ s dθ s = 1 − 0 . 4π cos θ 0 ϕs∫ 0 θ s∫ 0 4 = = 2π

π

(1.26)

Actually, natural surfaces do not have neither specular, nor Lambertian characteristics. They exhibit a mixed behavior depending on its dielectric properties and the surface roughness compared to the wavelength. Particular cases for natural surfaces can be found in [3].

1.3 Types of microwave radiometers
As it has been seen, if an antenna is pointing to a body, the power that is collected at its output (expressed in term of antenna temperature TA) is related to the brightness temperature TB of this body. A microwave radiometer is an instrument that measures the antenna temperature (TA) with highly resolution and accuracy. In fact, a microwave radiometer is a well calibrated and high sensitive microwave receiver. The performance of a radiometer is characterized by two main factors: resolution and accuracy [4]. The first one determines the smallest change in TA that can be detected by the radiometer output. The second one indicates the correspondence of the measurement of the true value.

In order to illustrate these two aspects, the following example is analyzed; a radiometer is connected to an antenna which is exposed to a temperature TA′ = 200 K,

24

Microwave Radiometry basics

and the radiometric resolution requirement of the measure is of 1 K. The noise temperature introduced by the radiometer, like any receiver, has to be taken in account; a typical value will be for example TREC = 500 K. Then the aim of the radiometer is to perform a measurement which matches with a variation of 1 K over 700 K (200 K + 500 K). In order to achieve this resolution, a radiometer uses an integration technique. Therefore, if the radiometer’s gain G and the noise temperature TREC are added in eqn. (1.27), the resulting output power is:

P = kGB TA′ + TREC

(

)

[W].

(1.27)

As it is shown, the stability of the power measurement depends on the stability of the factors in eqn. (1.27): B, G and TREC. As B is a parameter which mainly depends on the filter (passive device), it is assumed to be rather constant. Back to the previous example, if the required resolution is 1 K, it means that G and TREC have to be stable within ≤0.5 %, which corresponds to about 0.004 dB. Therefore the following problem appears that it will be difficult to get these requirements from an amplifier. After having seen the two main problems linked to the design of a radiometer, the main radiometer types and their behavior are presented in terms of resolution and accuracy. That will help to understand the MERITXELL design in Chapter 7. A radiometer block diagram consists basically of an antenna, a super-heterodyne receiver which translates the radio frequency signal to an intermediate frequency, a detector and a low-pass filter.

1.3.1 Total Power Radiometer (TPR)
The TPR is the most common radiometer used. It is easy to understand and it can illustrate the most important notions of the performance of such instrument. Figure 1.7 is used to explain it with more detail.

Figure 1.7: Total power radiometer block diagram [2].

25

Chapter 1

In the radiometer of the Fig. 1.7, the gain G is represented by an amplifier and B is the system noise equivalent bandwidth. To measure the noisy input signal, a square-law detector is used. Its output is directly proportional to the input signal power and so to the temperature TA′. An integrator (low-pass filter) is used to reduce the fluctuations in the detected signal, and therefore to increase the stability of the measurement.

In order to describe the power and voltage values in the different parts of the radiometer, Fig. 1.8 is included.

Figure 1.8: Waveform and spectra of the voltage in the different stages of a total power radiometer [2].

Taking into account that the input signal is thermal noise, the voltage output of the IF frequency, VIF is a complex sum of two Gaussian random variables, having 0 mean and a variance equal to the sum of variance of both Gaussian random variables, which can be assumed to be 2σ2, while the envelope of VIF, Ve has a Rayleigh distribution:

 Ve − 2Ve 2  e σ for Ve ≥ 0. p (Ve ) σ 2 0 for Ve < 0. 

(1.28)

For a Rayleigh distribution, the mean value of Ve2, which is the available power at the output of the IF amplifier over a 1Ω resistor, is equivalent to: 26

Microwave Radiometry basics

PFI = Ve2 = 2σ 2
Vd = CdVe2

[W].

(1.29)

After the square-law detector the detector’s voltage is:
[V].

(1.30)

where Cd stands for the power sensibility constant of the power detector, with the units volts over Watts ([V/W]). In this case, Vd presents an exponential distribution, as the square of the complex sum of two Gaussian variables is equal to an exponential distribution.

1 − p (Vd ) = e Vd . Vd
The mean value of Vd can be expressed as:

Vd

(1.31)

Vd = CdVe2 = 2Cd σ 2 = Cd PFI = Cd GkBTSYS

[V],

(1.32)

On the other hand, the LPF output voltage, Vout depends on two factors, a constant value, Vd and a random component, Vac(t). The parameter Vac(t) accounts for the standard deviation of Vd and is related with the uncertainly created by the intrinsic noise of the system, Psys. The constant value is related with the input power PA′, thus the radiometric temperature TA′ using the following equation:

Vout = GLPFVd

[V].

(1.33)

where GLPF is the gain of the LPF. So that, the output of a TPR is proportional to the radiometric temperature and its value is given by the following equation:

Vout ( t ) = Vout + Vac ( t ) = GLPF Cd GkBTSYS + Vac ( t ) = GsTSYS + Vac ( t ) [V].

(1.34)

For an exponential distribution, the squared mean value is equal to its variance, which means that the standard deviation and the mean value at the output of the square law diode are the same:

σd
Vd

= 1 → σ d = Vd

[V].

(1.35)

Equation (1.35) implies that the measurement uncertainly has the same value of its mean, which is unacceptable. The main function of the LPF is to decrease the uncertainly by integrating Vd over a period of time τ (which, in fact is the time constant of the filter, τ = 1/2BLPF). In that way the variance of the measurement is reduced by a factor N = Bτ, where N is the number of independent samples used for the integration. Therefore, the relationship between the standard deviation and the mean value at the LPF is:

27

Chapter 1

σ out
Vout

=

V 1 → σ out = out . Bτ Bτ

(1.36)

Hence, assuming that the parameters of eqn. (1.36) remain constant, this relationship can be re-written as a function of the standard deviation associated to the mean value:
∆TSYS 1 = , TSYS Bτ where: (1.37)

TSYS = TA′ + TREC

[K].

(1.38)

From eqn. (1.38) it is possible to infer the radiometric resolution (∆T) which is defined as the minimum input temperature which the radiometer is able to detect as a change in the output voltage. The radiometric resolution of a TPR is then computed as [2]: ∆TN ∆TSYS = TSYS TA′ + TREC = Bτ Bτ
[K],

(1.39)

However, eqn. (1.39) does not take into account the system gain fluctuations, so all the real fluctuations that occur in a receiver are missing. The gain uncertainly can be defined as ∆GS/GS and translates into an uncertainty of the estimated system’s temperature:  ∆GS  ∆TG = TSYS    GS 
[K],

(1.40)

where GS is the total receiver gain and ∆GS is the root mean square (RMS) variation of the detected power for a constant power input signal. Taking into account that the noise and the gain fluctuations are statistically independent, the final system resolution can be written as:

 1  ∆G   2 2 S ∆T = ( ∆TN ) + ( ∆TG )  = TSYS  +       Bτ  GS    
1 2 2

1 2

[K].

(1.41)

From eqn. (1.41) it can be inferred that the radiometric sensibility of a TPR has a strong dependence on the gain fluctuations. It is important to note that the best theoretical radiometric resolution can be achieved with an ideal TPR. However due to the gain fluctuations, an absolute calibration is frequently required.

28

Microwave Radiometry basics

1.3.2 Dicke Radiometer (DR)
With the aim to correct the stability problems associated to the gain fluctuations of the TPR, Dicke published in 1946 a radiometer design which is named after him (Fig. 1.9) [5]. The Dicke radiometer (DR), instead of measuring directly the antenna temperature, performs the measurement of the difference between TA′ and a known reference temperature TREF. With this method, the noise temperature instability TREC is filtered out and the impact of the gain is largely reduced.

Figure 1.9: Dicke radiometer block diagram [2].

As it is shown in Fig. 1.9, a DR is a modified TPR with an input switch that changes of position at a given frequency fs between the antenna and the reference temperature TREF and a synchronous demodulator (±1 multiplier). Therefore, two different outputs in distinct time slots are obtained. The detector output depends on the half period of fs: Vd ANT = Cd GkB TA′ + TREC

(

)

for 0 ≤ t ≤

τs
2

[V],

(1.42) (1.43)

Vd REF = Cd GkB (TREF + TREC ) for

τs
2

≤ t ≤τs

[V],

where TREF is the reference noise temperature, τs is the switching period, and
TREC is the receiver’s noise temperature, including the noise of the input switch. On the

29

Chapter 1

other hand, the synchronous demodulator has another synchronous switch. This switch commutes the input signal among to two unitary gain amplifiers, which have opposite signs, one amplifier has the Vd ANT and the other has Vd REF . The outputs of these amplifiers are added, and finally low-pass filtered. If the switching frequency is sufficiently fast to consider the parameters TA′, TREF and G constant during an entire period, and also that the period is smaller than the integration time ( f s radiometer’s output can be expressed as [2]:
Vout =

τ −1 ), then the

1 (Vd ANT − Vd REF ) = 1 Cd GkB TA′ − TREF 2 2

(

)

[V].

(1.44)

As it can be observed in eqn. (1.44), the output of the Dike radiometer is proportional to the (TA′ – TREF) term. So that, the uncertainly of TREF has to be taken into account in the radiometric resolution calculation. The resolution of a Dicke radiometer can be expressed as follows:
 ′  TA + TREC ∆T =  Bτ  2 

(

)

2

(T + TREC ) + REF
Bτ 2

2

 ∆GS  +  TA′ − TREF GS  

2

(

)

2

   [K].  

1 2

(1.45)

It is said that a Dicke radiometer is balanced in the case in which the antenna and the reference temperatures are identical (TA′ = TREF). In this case, the resolution reduces to:

∆T =

2 TA′ + TREC

(

Bτ

) = 2∆T

TPR

[K],

(1.46)

where ∆TTPR is the radiometric resolution of a TPR in the total absence of fluctuations. As it can be observed, in this ideal case (eqn. (1.46)), there are not gain fluctuations on the radiometric resolution, but the resolution is twice worse than in an ideal TPR, due to the integration time has been split by 2, half the period it is looking to the antenna and the other half the period it is looking to a reference load.

In a real case, when the temperature TREF is chosen close to the antenna temperature TA, the impact of G in fluctuations is small. Then if (TA′ – TREF) (TA′ + TREC) is fulfilled, the DR decreases the resolution respect to the TPR. Although the stability of the system is improved, by measuring the antenna temperature just half of the time, there is a loss of resolution as compared to a TPR. Indeed, on each half period,

30

Microwave Radiometry basics

the radiometer can be assimilated to a TPR pointing to the antenna or to the reference load, using an integration time of τ/2.

1.3.3 Noise Injection Radiometer (NIR)
The noise injection radiometer is a particular case of a Dicke radiometer. It has been optimized to ensure that its output is always independent on the gain fluctuations and on the receiver noise. To achieve that purpose a NIR has a feedback loop which is shown in Fig. 1.10.

Figure 1.10: Noise injection radiometer block diagram [2].

The aim of the feedback loop is to balance the radiometer (obtaining the same result as in the ideal case of a balanced Dicke radiometer) by injecting noise in the system input through a directional coupler ensuring that always is fulfilling:

TA′′ = TREF = 0 [K].

(1.47)

The amount of injected power is controlled by a variable attenuator, which is controlled by the feedback loop. Hence, the amount of power entering into the system can be calculated as:

 T ′ 1  TA′′ = 1 −  TA′ + N Fc  Fc 

[K],

(1.48)

where Fc is the coupling factor of the directional coupler, and TN′ is the amount of injected noise, attenuated by the variable attenuator. The voltage Vc, which controls the attenuation is proportional to the antenna, and the system physical (Tph≈290 K) temperature difference, and it is given in the following expression:

31

Chapter 1

Vc =

Fc − 1 Tph − TA′ TN − Tph

(

)

[V].

(1.49)

Using this technique, the output of the NIR is independent on the gain fluctuations, and on the receiver’s noise. The radiometric resolution of a NIR in the case of Tph = TA′ can be described as it follows: ∆T = 2 (Tph + TREC ) Bτ = 2∆TTPR
[K].

(1.50)

As it can be seen in eqn. (1.50), the NIR radiometric resolution is the same as that of a balanced DR (eqn. (1.46)), but with the advantage that it is always working as in the ideal case.

1.4 Conclusions
In this chapter the fundamentals of microwave radiometry theory have been presented in addition to a revision of the radiometry applications through the spectrum. The brightness temperature and the apparent temperature concepts have been defined, as well as the black and gray-body relationships through the emissivity.

Different radiometer types have been presented and discussed, and their advantages and disadvantages respect each other. This will help to understand the configuration and operation of the MERITXELL radiometer, which is the instrument developed to obtain some of the radiometric measurement used in this thesis. It follows a TPR configuration with a frequent calibration (in the order of seconds) using an internal matched load.

32

Radio Frequency Interference in Microwave Radiometry Measurements

Chapter 2
2. Radio Frequency Interference in Microwave Radiometry Measurements

Radio-frequency interference present in microwave radiometry measurements leads to erroneous radiometric results. Sources of RFI include spurious signals and harmonics from lower frequency bands, spread-spectrum signals overlapping the “protected” band of operation, or out-of-band emissions not properly rejected by the predetection filters due to its finite rejection. The presence of RFI in the radiometric signal modifies the detected power, and therefore the estimated antenna temperature from which the geophysical parameters will be retrieved. In this chapter several radiometric measurement missions will be described, where the collected radiometric data is somehow degraded by RFI.

33

Chapter 2

Microwave radiometry data is obtained by measuring the power of the thermal noise radiated by bodies in the microwave part of the spectrum. The larger the sensitivity, the smaller the power variations that could be detected. The radiometric sensitivity, (or radiometric resolution), can be improved by reducing receiver’s noise and increasing the noise bandwidth, and the integration time. However, the actual sensitivity of a radiometer is often limited by man-made radio emissions such as radars, broadcasting emissions, wireless communications, and many other communication systems based on electromagnetic waves. All these signals present in the protected bands are known as Radio Frequency Interference (RFI), and are one of the main problems in passive remote sensing of the Earth’s surface and atmosphere.

The problem created by RFI is related to the accuracy of the radiometric measurements. The power of the RFI signal increases the signal measured power, thus leading to erroneous interpretations. Since the thermal noise measured by a radiometer is a very weak signal, even strongly attenuated RFI sources are potentially dangerous for the radiometric measurements.

Although there are frequency bands reserved for passive remote sensing where transmissions are not permitted; RFI may be present in these bands due to spurious from other bands, and poor out-of-band attenuation of other communication systems adjacent to these protected bands.

Radio Astronomy suffered from RFI well before microwave radiometry [6-8], therefore it is correct to say that the RFI mitigation subject was firstly introduced by the Radio Astronomy community.

Several RFI surveys and studies have been performed in different ‘protected’ frequency bands used to measure thermal noise radiation; in L-band [9-12], C- and Xbands [13-15], and K-band [16-17].

34

Radio Frequency Interference in Microwave Radiometry Measurements

2.1 Soil Moisture and Ocean Salinity (SMOS) RFI contamination case
Global maps of L-band brightness temperatures are available since November the 2nd, 2009, when ESA’s SMOS mission was launched, showing many RFI hot spots [18].

MIRAS is the single payload instrument in the SMOS mission. It consists of an Y-shape interferometric radiometer formed by 72 receivers called LICEF placed along the three arms of the Y array. Each LICEF receiver is a dual polarized L-band radiometer working in the protected band of 1.400-1.427 GHz [19].

As it can be seen in Fig. 2.1, RFI is present mainly on Asia and Europe, but in some small areas of Africa and Greenland as well. Moreover, although RFI is usually localized in urban and industrial areas, its power is so large than the SMOS radiometer impulse response extend this localized RFI to the whole FOV, due to the way the SMOS image is processed from data obtained by a synthetic aperture array.

Figure 2.1. Global view of RFI sources from SMOS data [18].

35

Chapter 2

2.2 WindSat RFI contamination case
WindSat is the primary payload on the Coriolis mission, a polarimetric microwave radiometer, designed to demonstrate the capability of polarimetric microwave radiometry to measure the ocean surface wind vector from space. In addition, WindSat is capable to measure other environmental parameters such as sea surface temperature, total precipitable water, integrated cloud liquid water, rain rate over the ocean, soil moisture and sea ice [20]. The WindSat radiometer operates at discrete bands at 6.8, 10.7, 18.7, 23.8, and 37.0 GHz; the 10.7, 18.7, and 37.0 GHz channels are fully polarimetric while 6.8 and 23.8 GHz channel are dual-polarized (vertical and horizontal).

WindSat polarimetric data from C- and X-bands is shown in Fig. 2.2 [13]. Only the continental United States area is shown, however, the presence of RFI is obvious in both bands. Note the RFI present at X-band has been detected taking advantage of the third and fourth Stokes parameters, as X-band linear polarizations do not reveal significant RFI [13].

Figure 2.2. RFI present in WindSat data. a) C-band maximum brightness temperature values at V polarization [K]; b) C-band maximum brightness temperature values at H polarization [K]; c) X-band maximum brightness temperature values for the third Stokes parameter [K]; d) X-band maximum brightness temperature values for the fourth Stokes parameter [K] [13].

36

Radio Frequency Interference in Microwave Radiometry Measurements

2.3 AMSR-E RFI contamination case
Another example of RFI contamination at C- and X-bands, but with a different measurement instrument, the AMSR-E is described in this section. The AMSR-E is a dual polarization microwave radiometer with six different frequencies: 6.9, 10.65, 18.7, 23.8, 36.5, and 89.0 GHz [21].

Figure 2.3 shows RFI detected by comparing AMSR-E data from different frequency bands. The multi-frequency feature of the AMSR-E allows this instrument to combine information from two different frequency bands to detect RFI present in its measurements. In this figure, RFI contamination is again localized in urban areas in Europe, Asia and North America [14].

Figure 2.3. Global maps of standard deviations of AMSR-E brightness temperature spectral difference (RFI6V = TB6V + TB10V) for January 2003. The statistics are derived by aggregating data within 0.25º latitude and longitude bins. Color scale units are in [K] [14].

2.4 General RFI contamination cases
In addition to the particular RFI examples of the radiometers explained in the previous sections, there are many RFI sources affecting all radiometers. The following

37

Chapter 2

sections summarize two issues affecting several microwave radiometers produced by RFI sources.

2.4.1 RFI originated by 24-GHz automotive radars
Nowadays, new cars include short-range radars (SRR) operating near 24 GHz to improve road traffic safety, and make parking maneuver easier. These sensors are intended to operate in the frequency bandwidth of 22-29 GHz in an UWB mode, occupying up to a 5 GHz bandwidth [16].

The frequency band used by these sensors interferes with the bands 22.21–22.50 and 23.6–24.0 GHz from which radiometric data related to the atmospheric water-vapor is obtained [16].

This ‘protected’ frequency band is used by many space-borne radiometers already launched: WindSat, AMSR-E, the AMR of the JASON, the MWR of the ERS-2 among others, or in preparation such as: the GMI of the GPM mission.

2.4.2 RFI present in the calibration process
Furthermore, RFI may be present even in the calibration data, producing a systematic error in the whole data set; as it has been reported with the AMSR-E at Cband, where RFI periodically appears in the cold sky mirror calibration process, probably cause by Globastar 54 LEO satellite [22].

2.5 Conclusions
The problems of the RFI present in the radiometric measurements is presented, and some examples are shown with real radiometric data. These four cases are only one part of the problem that is continuously increasing as the telecommunication systems increase its density and frequency range.

38

Radio Frequency Interference in Microwave Radiometry Measurements

Nowadays, the situation of microwave radiometry leads to the motivation of this thesis, as RFI detection and mitigation has become a main concern in microwave radiometry. In this moment, the highest radiometric resolution is useless without a RFI detection algorithm.

39

State of the Art in Radio Frequency Interference Detection and Mitigation Systems

Chapter 3
3. State of the Art in Radio Frequency Interference Detection and Mitigation Systems

At the present time, man made electromagnetic interferences are continuously increasing as the

telecommunication systems technology is expanding. These RFI sources affect the radiometric data obtained in several missions leading to erroneous retrieval of geophysical parameters. In order to mitigate these errors, RFI detection and mitigation systems and algorithms are being developed, thus more reliable measurements can be obtained. In this section several RFI detection and mitigation algorithms are described, and examples of its actual uses in the radiometry and radio astronomy fields are presented.

41

Chapter 3

As it has been stated in Chapter 2, radio astronomy has faced the RFI problem in the past, well before in Earth’s remote sensing. At this time, hereby it could be convenient to revise previous researches in this field. In this section RFI mitigation methods used in the radiometry field which almost all have been previously used in radio astronomy are described [23].

3.1 RFI detection and mitigation in the time and frequency domains
RFI detection algorithms used in the time and frequency domains try to search for RFI components concentrated in determined parts of the frequency and/or time domains which present a higher power value than the clean radiometric signal. This finding is based in the comparison of the received data with a determined threshold value.

Every system used to detect or mitigate RFI will require a threshold value to discriminate between RFI-contaminated samples and RFI-clean samples. On one hand, this threshold must be determined by means of an estimation of the RFI free radiometric signal power; in addition, a compromise between the probability to detect (probability of detection, Pdet), the RFI, and the probability to eliminate clean RFI data falsely detected as RFI (probability of false alarm, Pfa), must be accomplished. In a simple manner, the threshold can be defined by the following equation:

ˆ2 Th = c ( Pfa ) ⋅ σ n ,

(3.1)

where Th represents the threshold to be used, c(Pfa) is a constant that depends on
ˆ2 the probability of false detection of a RFI, and σ n is the RFI-free radiometric signal

power estimation.

The RFI-free radiometric signal power can be estimated using directly the measured radiometric power (assuming that the total RFI power is much lower than the clean radiometric signal power) [24] or, if an ADC and a FPGA are present in the front end of the radiometer, this power can be estimated by cutting off lower and/or upper percentiles [25,26]. 42

State of the Art in Radio Frequency Interference Detection and Mitigation Systems

The probability of false alarm as a function of the threshold is a critical parameter to avoid the elimination of clean RFI radiometric data. An approximate proportion of clean RFI samples equal to the value of Pfa will be eliminated, but typically only the ones with the highest power, thus biasing the measured brightness temperature to a lower value. The c(Pfa) constant depends on the selected Pfa, and also on the PDF of the observable used in the RFI detection algorithm. Thus, if our radiometer samples directly the antenna voltage, the Probability Density Function (PDF) is a Gaussian function, as this voltage is Additive Gaussian White Noise (AGWN), but if the observable is obtained from the output of a square law detector, the PDF is exponential, as the power of an AGWN follows an exponential distribution, or if the detector is an envelope detector, the PDF will have Rayleigh distribution. Even more, if the observable is obtained from the output of the low-pass filter located after the detector, its PDF will follow the Chi-square PDF which, in some cases, can be approximated by a Gaussian PDF [27].

Once the concept of threshold is defined, time and frequency domain algorithms can now be described.

3.1.1 RFI Detection in the Time domain
Time-domain RFI detection and mitigation algorithms are the simplest ones to implement, as they only need to sample the radiometric data and compare its power with a determined threshold directly related to the power of the RFI-free radiometric signal. These algorithms are effective when dealing with short high powered bursts of RFI. Sampling with a sufficiently high frequency, and subsequent thresholding may give good results. However, when RFI is comparable or smaller than noise power, it becomes more difficult to detect, as it is not detected by the threshold and the estimated power is erroneous. In addition, since the detected power is a smoothed (averaged) version of the instantaneous one, if the duration of the RFI peaks is shorter than the integration time they may pass undetected.

43

Chapter 3

RFI detection in time domain has been previously used in radio astronomy with pulsed RFI contamination such as the one coming from a radar near the Arecibo radiotelescope [28].

In microwave radiometry, this technique also has been used to detect and eliminate pulsed RFI in [24]; although the time-domain RFI detection algorithms are usually combined with frequency-domain RFI detection algorithms as interference may be any kind of signal [26].

3.1.2 RFI Detection in Frequency domain
Usually, man-made RFI is composed by a base-band signal multiplied by a carrier frequency. Continuous Wave (CW) RFI is easy to detect using frequencydomain algorithms even when its power is comparable or lower to the radiometric signal.

Frequency-domain RFI detection algorithms are more complex than the algorithms in the time-domain as it is necessary a subbanding process; either digital (FPGA-based digital filtering [29], or FFT calculation [26]), or analog (RF filtering [30]). However, as digital processing hardware increases continuously its performance, digital subbanding is becoming the most common technique.

After subbanding, RFI detection is performed by calculating the total power in every subband and applying a threshold directly related to the power of the radiometric signal which will be similar to the time domain threshold as the power is preserved. By this way, CW or harmonic signals will be detected if its power is higher than the noise in a determined subband.

44

State of the Art in Radio Frequency Interference Detection and Mitigation Systems

3.1.3 RFI Detection in the combination of Frequency and Time-domains
As it has been explained, RFI present in radiometric measurements can be formed by several types of signals, therefore, it is useful to combine algorithms in both frequency and time-domains [28, 29].

In addition, if the RFI is produced in a determined time, and only occupies a determined frequency band of the spectrum; blanking an entire frequency band or a temporal segment of data will lead to a higher radiometric data loss than the blanking of only the frequency band in the exact time the RFI is produced. A simulation of RFI localized in time and frequency, and its elimination is represented in Fig. 3.1.

1412.5 -50 -60 -70 1412.3 -80 -90 1412.2 -100 -110 -120 1412 0 0.01 0.02 0.03 0.04 0.05 0.06 -130

1412.5

Frequency (MHz)

Frequency (MHz)

1412.4

1412.4

1412.3

1412.2

1412.1

1412.1

1412

0

0.01

0.02

0.03

0.04

0.05

0.06

Time (s)

Time (s)

a)

b)

Figure 3.1. Time-Frequency plane of simulated radiometric data. a) RFI is present in well localized areas with higher power; b) elimination of the contaminated areas, without eliminating an entire frequency or time segment.

The most effective way to combine both domains is by means of the calculation of the radiometric signal spectrogram [25]. In Chapter 5 this algorithm will be explained in detail, as RFI detection in both domains has been one of the scopes of this Ph. D.

3.2 RFI detection by Statistical Methods
Radiometer signals are generated by noise and so, they are inherently zero-mean random Gaussian variables. In the absence of RFI, the pre-detection analog signal in a

45

Chapter 3

microwave radiometer is generated exclusively by thermal emission sources. These sources include both the natural thermal emission incident on the antenna from the Earth and sky (antenna temperature) as well as the radiometer noise generated by ohmic losses and noisy active components in the hardware (receiver’s temperature). In this case, the PDF of the amplitude of both signals is Gaussian distributed. Signal sources other than thermal noise (i.e. RFI) will, in almost all cases, have non-Gaussian PDF’s.

Normality analysis is the way to take advantage of this physical phenomenon to detect man made RFI signals present in the radiometric data, which cannot be detected by other RFI detection methods. Normality analysis such as the calculation of the kurtosis of the radiometric signal [29] and the Shapiro-Wilk normality test applied to the radiometric signal [31] have been previously used in radiometry. In fact, the kurtosis method was previously applied to detect RFI present in radio astronomy data [32]. The kurtosis (K) is a statistical parameter defined as the 4th central moment normalized by the square of the 2nd central moment (variance), and for a zero mean random variable is equal to:

E ( X − E [ X ])  µ4   , K= 4 = 2 2 σ  
4

(

E

( X − E [ X ]) 



)

(3.2)

where µ4 is the fourth order moment of random process X, and σ is the standard deviation of random process X. This statistical parameter has the property of having a value equal to 3 for Gaussian distributed signals and it is usually different from 3 for non-Gaussian signals. However, some examples do exist of RFI signals for which K≡3, e.g. a pulsed sinusoidal signal of 50% of duty cycle [33]. The kurtosis does not depend on the signal variance, and weak RFI can be detected even under the noise level.

Another statistical method is the Shapiro-Wilk normality test, which is capable to detect pulsed sinusoidal signal of 50% of duty cycle. It has been previously used in [31], where it has been implemented in an FPGA using histograms to avoid the complex task of ordering the samples.

In Chapter 4 of this thesis ten different normality tests are studied and compared, to evaluate which one has the best performance for different types of RFI signals. 46

State of the Art in Radio Frequency Interference Detection and Mitigation Systems

The advantage of this technique in front of the suppression in time and frequency domain technique is that low level RFI’s in both domains can be detected, at the expense of a more complicated front-end to be able to calculate the 2nd and 4th moments of the received signal. However, a method to calculate an approximation of the kurtosis entirely by hardware is described in [34].

Nevertheless this technique is usually used subdividing the measured spectrum in frequency bands, as the interferences are usually not spread among the whole spectrum, and blanking the whole spectrum every time a RFI is detected will not be efficient. This process can be done by digital filtering the input radiometric signal to obtain a determined number of subbands [29]; or by calculating the Fast Fourier Transform (FFT) of the radiometric signal, and applying the kurtosis to the different subbands that are present in the FFT taking advantage of the fact that the Fourier Transform of a Gaussian signal is a complex Gaussian signal, which indeed is the RFI mitigation method that will be used by the upcoming SMAP mission [35].

It is also useful to define the Spectral Kurtosis, which consists on the calculation of the FFT of the signal and calculating the kurtosis value for every frequency bin independently, as defined in [36]. In order to dispose of more than one sample for every frequency bin to calculate de spectral kurtosis, radiometric signal must be divided into

M several blocks, hence the spectral kurtosis will be calculated with M samples, all of
them belonging to the same frequency bin [36]. This definition of spectral kurtosis has been previously used in radioastronomy [37] and in radiometry [38].

3.3 Suppression using Filtering Techniques
Temporally spread and strongly correlated RFI can be suppressed using cancellation techniques based on estimating the RFI waveform and subsequently subtracting it from the received signal.
ˆ xCLEAN ( t ) = x ( t ) + xRFI ( t ) ,

(3.3)

47

Chapter 3

ˆ where x ( t ) is the received signal, xRFI ( t ) is an estimation of the RFI waveform included in the received signal, and xCLEAN ( t ) is the thermal noise, free of RFI components.

In principle, the RFI waveform can be estimated using any available filtering technique (i.e. spline-moothing, Wiener filtering, wavelet denoising, parametric identification). Subsequently, the RFI estimate can be subtracted from the received signal in the temporal or frequency domains.

This technique is valid when the RFI source is perfectly known or at least correctly estimated, and due to its complexity, fast digital signal processing hardware is required to deal with it.

An example of a parametric identification approach of this type of RFI cancellation can be found in [39], where the interfering signal of a GLONASS satellite is represented in a parametric model with parameters (Doppler frequency, phase code and complex amplitude) that are calculated for each separated data block. The parametric model of the RFI was used to calculate an estimation of the RFI waveform which was subtracted from the received signal.

RFI suppression methods based on wavelet denoising have also been developed in [40], with an increase of performance, when the sampling rate of the radiometric signal increases. This wavelet denoising method is explained in Chapter 6.

3.4 Adaptive Interference Cancellation using Reference Channels
A separate, dedicated reference channel is used in order to obtain an independent estimate of the RFI signal. This technique has been widely used in digital signal processing, and it is known as adaptive noise cancelling [41].

48

State of the Art in Radio Frequency Interference Detection and Mitigation Systems

Figure 3.2 represents a block diagram explaining the application of this technique. There are two data channels: a main channel pointing to the source and containing the RFI signal; and a reference channel (separated antenna pointing off source) that contains also the RFI signal. Both channels contain the RFI signal, which are different due to the different propagation paths, but correlated as they come from the same source. Taking advantage of this correlation it is possible to, by means of adjusting and subtracting the RFI from the reference channel, eliminate the RFI from the received signal [42].

Figure 3.2. Adaptive noise cancelling concept.

This procedure can be applied both in the time-domain (adaptive filtering) and in the frequency domain (FFT → adaptive filtering in each frequency bin → FFT-1). This kind of RFI cancellation is especially useful when the RFI and the signal of interest occupy the same frequency domain.

3.5 Spatial Filtering using Multi-element Systems
Spatial filtering methods use the difference in the Direction-of-Arrival (DOA) of the signal of interest and the RFI. The RFI emission from spatially localized sources could be suppressed using multi-element radio interferometers based on an adaptive array philosophy, forcing the zeros of a synthesized antenna pattern to coincide with the DOAs of undesirable signals (adaptive nulling). However, usually RFI sources will not be localized in only one point in the space; hence this method will work with a maximum number of RFI sources.

Another way of RFI cancellation is the RFI estimation by means of the combination of the complex spectra of the different antennas. Then, this RFI estimation

49

Chapter 3

is removed from the received signal. This method is very similar to the method presented in the section 3.4, but assuming a larger number of reference channels.

Obviously, this technique can only be applied to interferometers, since a real aperture radiometer has only one antenna pointing to a given place at a given time.

This technique has been used firstly in radio astronomy, applying spatial filtering post-processing techniques in data obtained from the Westerbork Synthesis Radio Telescope (WSRT) [43].

In radiometry, an example of spatial filtering post-processing technique has been developed as a RFI detection and mitigation algorithm for Soil Moisture and Ocean Salinity (SMOS) data [44]. In addition DOA RFI detection algorithm has been proposed in [44].

3.6 Conclusions
Several RFI detection methods have been exposed in this chapter, in addition to examples of actual measurement devices that use these methods.

The problem with the RFI is that there is not a perfect algorithm able to detect any kind of RFI, each algorithm performs best for a determined RFI, so the best RFI method will be composed by a combination of two or more RFI algorithms.

However, not all RFI algorithms can be used in all radiometers; as some algorithms require arrays of antennas (sections 3.4 and 3.5), others require an ADC to measure the radiometric signal without losing any feature (sections 3.1.3, 3.2 and 3.3), and others will also require fast digital processing hardware (section 3.3). It is obvious that not all the radiometers will accomplish all these hardware requirements, so the radiometer design must be taken into account to apply any of the RFI algorithms described in this chapter.

50

Normality Analysis for RFI Detection in Microwave Radiometry

Chapter 4
4. Normality Analysis for RFI Detection in Microwave Radiometry
Nowadays, man-made RFI is composed by numerous signals using different modulations. Pulsed signals and tones are usually easy to detect and eliminate simply by thresholding either in the time or frequency domains. Broadband modulations distribute the power of the emitted signal over the time and frequency domains, therefore, it can remain undetectable by RFI time frequency-domains detection in algorithms. These

broadband modulations are nowadays in constant increase so it is necessary an algorithm specialized in this kind of RFI. The received radiometric signal in the absence of RFI, must be a zero-mean Gaussian process, while man-made RFI usually is not Gaussian. Therefore, the study of the statistics of the radiometric signal can detect interferences which remain undetected with RFI detection algorithms in time and frequency domains.

51

Chapter 4

As it is widely known, radiometric signal (thermal noise) follows a determined probability distribution which, in the absence of RFI, is a zero-mean random Gaussian variable. As a consequence, the probability density function (PDF) and the statistical parameters, such as the moments, are perfectly known.

Therefore, it is possible to detect RFI present in radiometric data with the statistical analysis. The most widely used time/frequency domain statistical analysis in microwave radiometry is the Kurtosis (ratio of the fourth moment and the square of the second moment) which must be equal to 3 in RFI-free conditions [29, 45, 46]. However, the sixth order moment [47], and other algorithms [31, 34, 40, 48] have also been studied.

In this Ph. D. thesis, the suitability of several normality tests for RFI detection in microwave radiometry has been analyzed. The normality tests involved are: Jarque-Bera (JB), Shapiro-Wilk (SW), Chi-square (CHI2), Anderson-Darling (AD), LillieforsSmirnov-Kolmogorov (L), Lin-Muldhokar (LM), Agostino-Pearson K squared (K2), Cramer-von Mises (CM), in addition to the Kurtosis (K) and Skewness (S) statistical parameters to detect signals non-normality. The ultimate objective is to compare these normality tests to obtain an omnibus test to detect RFI, or at least, the best normality test for a determined type of RFI. Extensive Monte Carlo simulations have been used to compare tests performance.

A brief description of these tests is given in section 4.1. Section 4.2 analyzes the validity of these tests in relation to its probability of false alarm. Section 4.3 shows the results obtained for the different tests applied to a simulated scenario of thermal noise signal (Gaussian) contaminated with different RFI signals. Finally Section 4.4 summarizes the conclusions obtained from the simulation results.

4.1 Normality tests
The rationale behind the use of normality tests to detect RFI in microwave radiometry is the fact that the thermal noise signal measured by radiometers, follows a zero-mean Gaussian distribution, while in general, man-made RFI are not Gaussian.

52

Normality Analysis for RFI Detection in Microwave Radiometry

Normality tests used in this study have been widely used in statistical literature and are described in this section. Some of these tests can be used with probability distributions different from the normal distribution.

The radiometric signal must be sampled following the Nyquist theorem, which states that the sampling frequency fs must be at least twice the signal’s bandwidth B. In this study, a sample is defined as the value of the received signal amplitude obtained by an ADC every Ts = 1/fs seconds. Thus, the number of samples or sample size is the total number of values obtained in the sampling process of the signal.

A brief summary of the statistical tests used in this study is provided

4.1.1 Kurtosis test
The kurtosis is a statistical parameter related to the shape of the PDF of a random variable. The kurtosis of a Gaussian random variable is always 3 independently of its mean and variance. Assuming a random process X, the Kurtosis (K) follows:

E ( X − E [ X ])  µ4   , K= 4 = 2 2 σ  
4

( E ( X − E [ X ] )  )
∑( X
N i =1 i

(4.1)

ˆ K=

1 N 1 N 

−X −X

) )

4

∑( X
N i =1

2

i

  

2

,

(4.2)

where N is the sample size, and X represents the sample mean of X. Equation
ˆ (4.2) defines the Kurtosis estimator K used in this study. In the specific case that the
ˆ random process is a zero-mean Gaussian process, the value of K tends to 3 as the

sample size increases. The Kurtosis test consists of comparing the estimated kurtosis value of the received signal with tabulated values of the cumulative distribution function (CDF) of the kurtosis of a Gaussian random variable of N samples; in Fig. 4.1 contours of these CDF tabulated values are represented. The kurtosis parameter has been used in microwave radiometry RFI detection, although it exhibits some problems in detecting some particular signals [29, 40, 45, 46].
53

Chapter 4

95 0.9

3.6
9 0.9 5 0.9

C D F

3.4 Kurtosis value

= 9 99 0.

0.9

3.2
0.7 5

3

0.5

0.2 5

2.8 2.6 0.0 5 2.4

0.1
1 0.0
01 0.0

0. 00 5

10

3

10 N samples

4

10

5

10

6

Figure 4.1: Contours of the CDF for the kurtosis value as a function of the number of independent samples N when the samples follow a Gaussian distribution.

4.1.2 Skewness test
Skewness is a statistical parameter related to the asymmetry of the PDF of a random variable. In this case, the skewness of a Gaussian random variable is always 0. Assuming a zero mean random process X, the skewness (S) follows:
3 E ( X − E [ X ])    2 E ( X − E [ X ] )   

µ S= 3 = σ3

(

)

3/ 2

,

(4.3)

ˆ S=

1 N 1 N 

∑( X
N i =1 N i

i

−X

)
2

3

∑( X
i =1

−X

)

  

3/2

.

(4.4)

ˆ Equation (4.4) defines the skewness estimator S used in this study. In the ˆ specific case that the random process is Gaussian, S tends to 0 as the sample size
increases. The Skewness test is based on comparing the estimated skewness value of the received signal with tabulated values of the skewness of a Gaussian random variable, as it has been done in the Kurtosis test (Fig. 4.2). Kurtosis and skewness CDF tables have been computed from 216 Monte-Carlo simulations.

54

Normality Analysis for RFI Detection in Microwave Radiometry

0.4
95 0.9

0.3 0.2 Skewness value 0.1 0 -0.1 -0.2 -0.3 -0.4

0. 95
0.9

CD

F

0.5
0.2 5
0.1

99 0.
0.7 5
1 0.0

=

0.9 99

05 0.
0.0 05

01 0.0

10

3

10 N samples

4

10

5

10

6

Figure 4.2: Contours of the CDF for the skewness value as a function of the number of independent samples N when the samples follow a Gaussian distribution.

4.1.3 Jarque-Bera test (JB)
JB test is a normality test based on the skewness and the kurtosis of the process, e.g. analyses the normality of a process taking into account both the kurtosis and the skewness of this process. JB test is defined as:

 ˆ N  ˆ2 K − 3 JB = S + 6 4  

(

)

2

 ,   

(4.5)

ˆ ˆ where N is the sample size, S is the skewness estimator of the process, and K is ˆ ˆ the kurtosis estimator of the process. In case of normality S and K are asymptotically independent, and hence, the JB test asymptotically follows a Chi-square distribution with two degrees of freedom. Unfortunately, this fact leads to an error measurement when the sample size is low [49-51].

4.1.4 D’Agostino K-squared test (K2)
K2 test, like JB test, is also based on the skewness and the kurtosis of the process, with the particularity that the skewness and the kurtosis of the process must be first approximated to avoid the error measurements present in the JB test when the sample size is small. In case of normality K2 test, it follows a chi-squared distribution 55

Chapter 4

even with a low sample size. The definition of the K2 test is complex and it can be consulted in [52] for the interested readers.

4.1.5 Kolmogorov-Smirnov (KS) and Lilliefors (L) tests
KS test is based on the empirical distribution function (EDF); given N ordered values of a sample X the EDF is defined as:

ˆ FN ( x ) =

# { X i | X i ≤ x}∀i =1.. N N

=

1 N

∑I (X
i =1

N

i

≤ x) ,

(4.6)

where I(·) is the indicator of the event, Xi is the ith element of the sample to be
ˆ tested, whose values must be ordered from the lowest to the highest, and FN ( x ) is a step function that increases by 1/N at the value of each ordered data point. KS test correlates the empirical distribution function with the normal distribution function, with a determined mean and variance that must be known. Since the mean and the variance are usually unknown parameters, this test is replaced by the L test to avoid the errors introduced by a wrong variance estimation. The L test is a slight modification of the KS test in which the mean and variance of the normal distribution are obtained from the sample X [53]. The L test is defined as:
ˆ L = max F (Yi ) − F ( X i ) ,
1≤ i ≤ N

(4.7)

ˆ where F ( X i ) is the value of the ith element of the EDF of X, and F (Yi ) is the value of the ith element of the normal distribution function with (L test case) mean and variance ( σ Y 2 ) equal to:

Y=

1 N

∑X
i =1

N

i

,

(4.8)

σY 2 =

1 N ∑ Xi − X N − 1 i =1

(

)

2

,

(4.9)

L confidence values are obtained from the CDF of the L test result when applied to a Gaussian distribution. Hence, these values represent the result of the test in case of normality [53]. A total of 216 Monte-Carlo simulations have been performed to obtain these confidence values, which are tabulated as the previous Kurtosis and Skewness tests values. Lilliefors test confidence values are represented in Fig. 4.3a.

56

Normality Analysis for RFI Detection in Microwave Radiometry

On the other hand, the L test has still one limitation, it tends to be more sensitive near the centre of the distribution, than at the tails. In general, the probability of detection will be set as high as possible, making more important the tails of the distribution than the centre.

4.1.6 Anderson-Darling test (AD)
The AD test is a modification of the L test that gives more weight to the tails than the L test, thus, AD test is also a ECDF based test. As this test is also based on the comparison of distribution functions, the values of the sample to test must be ordered. This test consists of: AD*2 = − N − with:
Yi = Xi − X

1 N

∑ ( 2i − 1) ( ln Φ (Y ) + ln (1 − Φ (Y
N i =1 i

N +1−i

))) ,

(4.10)

σX

,

(4.11)

where Φ(·) represents the standard normal cumulative distribution function (CDF) operator. As it is described in [54], AD*2 must be adjusted for the sample size as follows:

 0.75 2.25  AD 2 = AD*2 1 + + 2 . N N  

(4.12)

Critical values can be consulted from tables [54], although an empirical development of these critical values for the normal case is presented in [55].

4.1.7 Shapiro-Wilk test (SW)
The SW test belongs to the ECDF comparison group of tests. Again, samples must be sorted from the lowest to the highest values in order to be able to use this normality test. The SW test is defined as:

 N   ∑ ai X i    SW = N i =1 ∑ Xi − X
i =1

2

(

)

2

.

(4.13)

57

Chapter 4

The main part of the SW test is the vector of coefficients ai, i = 1...N. These coefficients are tabulated in [56] for the case of less than 50 samples, or they can be analytically calculated [57]. Furthermore, to ease the application of the SW test, in [58] the SW test has been transformed to have a normal distribution in the case of normality of the tested signal. A drawback of this test is the limitation of the sample size to a maximum of 2,000 values [58]. Longer sample lengths can be tested by dividing it in several shorter length sets of samples and calculating the SW test on each set, and averaging the results as they are normally distributed [33, 48].

4.1.8 Cramer-von Mises test (CM)
The CM test is a variation of the L test (so, it is a ECDF based test) [59]. It is defined as:

CM =

N 1  2i − 1  + ∑ − Φ (Yi )  , 12 N i =1  2 N  2

(4.14)

where Yi has been already defined in (4.11). Confidence values are obtained following the same methodology as with the L test. CM test confidence values are represented in Fig. 4.3b; it can be noted that these confidence values do not depend on the number of samples N.

0.07 0.06 Lilliefors confidence value 0.05

Pfa = 0.001 Pfa = 0.005 Pfa = 0.01 Pfa = 0.05 Pfa = 0.1 Pfa = 0.5

0.25 Cramer von Mises confidence value

Pfa = 0.001 Pfa = 0.005 Pfa = 0.01 Pfa = 0.05 Pfa = 0.1 Pfa = 0.5

0.2

0.04 0.03

0.15

0.1

0.02 0.01

0.05

0 10

3

10 N samples

4

10

5

10

6

0 10

3

10 N samples

4

a)

b)

Figure 4.3. Confidence values for the Lilliefors and Cramer-von-Mises tests to detect non-normality, lower Pfa require higher confidence values. a) Lilliefors test case; b) Cramer-von-Mises test case.

58

Normality Analysis for RFI Detection in Microwave Radiometry

4.1.9 Lin-Mudholkar test (LM)
The LM test is based on the fact that the mean and variance of a random sample are independently distributed, if and only if, the parent population is normal (a simple test for normality against asymmetric alternatives). The LM test of a sample X is defined as [60, 61]:

 1+ R  LM = 0.5log  ,  1− R 
being R the cross-correlation between the samples X and Y:

(4.15)

R=

∑( X
N i =1

i

− X Yi − Y

)(

) )
,
2

∑(
N i =1

Xi − X

) ∑(
2 N i =1

(4.16)

Yi − Y

and
  1 Yi =  N  
2       ∑ X j     i≠ j   2  ∑ X j  − N − 1   .   i ≠ j      1/3

(4.17)

The LM parameter presents a normal distribution, however it is not normalized neither in the mean, nor in the variance. In [61] it is specified the procedure to normalize the LM test. This test is sensitive only to departures from normality due to Skewness. As shown in [62], this procedure is generally much more powerful at detecting Skewness than the Skewness coefficient itself, although it has little power in detecting nonnormal symmetrical distributions. As an example, uniform or platykurtic distributions will pass this test easily, and therefore other tests such as AD test have to be also used in conjunction with it.

4.1.10 Chi-square test (CHI2)
The strong point of the chi-squared goodness-of-fit test is that it can be used to test if a data sample belongs to a process with a determined distribution. However, in our application this is not an advantage as the only distribution to be analyzed is the

59

Chapter 4

normal one. On the other hand the CHI2 test requires a sample size large enough for the chi-square approximation to be valid. CHI2 test of the sample X is defined as:

CHI2 = ∑
i =1

N

( Oi − Ei )
Ei

2

,

(4.18)

where CHI2 represents the result of the test, which asymptotically approaches to a χ2 distribution, Oi is the frequency of the ith possible outcome of the sample (assuming O to be an histogram of the sample X), Ei is the theoretical frequency (which in our case is the histogram of the zero mean unit variance random normal process), and N is the number of possible outcomes of each event.

4.2 Validation of Normality Tests in the Absence of RFI
In order to evaluate the performance of the different tests listed in section 2, two error types must be introduced first:

Type I error: rejection of a true hypothesis. In our context, type I error is also known as probability of false alarm (Pfa). This error is produced when, in the absence of RFI sources, the algorithm “detects” the presence of RFI in a determined sample, leading to the blanking (elimination) of correct data, thus reducing the total integration time.

Type II error: acceptance of a false hypothesis. In our context, type II error is known as probability of missed detection (Pmiss). This error is produced when a RFI is present in the signal, but it is not detected, leading to an erroneous measurement, but it is assumed to be correct. Probability of detection (Pdet) used in this work is defined as 1 – Pmiss. It is obvious that the objective is to obtain a low probability of false alarm and a high probability of detection; but both types of errors have a strong correlation, where the setting of the value of one of these parameters determines the value of the other. This way, if it is desired to minimize the probability of false alarm, the RFI detection

60

Normality Analysis for RFI Detection in Microwave Radiometry

threshold must be set to a relative high value, leading to a low probability of detection. On the other hand, if it is desired to maximize the probability of detection, the RFI detection threshold must be set to a relative low value, leading to a high probability of false alarm. A good way to evaluate the compromise between Pdet and Pfa, is the calculation of the so-called Receiver Operating Characteristic (ROC) curves Pdet(Pfa) [31, 38, 48]. In addition to ROC plots of the Interference to Noise Ratio (INR) as a function of the sample size and other parameters are shown.

The INR parameter that is used to determine the performance of the normality tests in this Ph. D. thesis is defined as:
2 σ RFI , 2 σ Noise

INR =

(4.19)

2 2 where σ RFI is the power of the interfering signal and σ Noise is the thermal noise

power.

Validation of normality tests is performed to minimize the errors in the threshold calculation for a determined pair of values Pdet and Pfa. Normality test errors introduced by the different tests in the Pdet and Pfa values must be acceptable. The method followed to check the normality tests is the calculation of the ROC curve by means of 215 Monte Carlo simulations of a Gaussian signal in the absence of RFI, for every test, and varying some determined parameters (sample size and quantization level). The rule of thumb followed in this Ph. D. thesis considers a test valid when the error between the ROC curve of the test and the RFI free case ROC curve (Pdet = Pfa) is less than the 5% (e.g. for Pfa = 0.1 → 0.095 < Pdet < 0.105). Quantization has been modeled by varying the number of discretization levels (determined by the number of bits), and assuming a dynamic range of the ADC of ±8σ, where σ2 is the RFI-free noise power, to avoid signal clipping.

Figures 4.4a and 4.4b represent the ROC curves for the SW test in the case of calculating the test in blocks of 2,048 and 4,096 samples respectively, and averaging the results to obtain a Gaussian distribution [58]. Test validation has been performed to these two sizes of the sets to obtain the largest sample size that can be used with the SW

61

Chapter 4

test. For the 4,096 samples case, performance of the SW test decreases with the sample length, while in the case of 2,048 samples the SW test does not have significant variations with respect to the ideal behavior. Explanation of this behavior can be found in [57], where the performance of the SW test is guaranteed for a maximum of 2,000 samples, but not for larger sample lengths. The number of bits has been set to 20 (220−1 quantization levels) in both cases to avoid quantization errors.

a)

b)

Figure 4.4. Comparison between the performance of Shapiro-Wilk test with block size set of: a) 2,048 samples, and b) 4,096 samlpes. 20 quantization bits have been used to neglect quantization errors. For 4,096 samples (>2,000 [57]) the performance degrades, while it does not for 2,048 samples.

Figure 4.5 represent the validity of the SW test, as a function of the sample length and the quantization level. Solid lines represent the cases where the test is considered to be valid (error <5%), while dotted and dash-dotted lines represent invalid test cases. As it can be appreciated, the longer the sample size, the more the quantization levels are required since the quantization process introduces a discretization error of the PDF. Quantization makes the normal distribution to become similar to a binomial distribution, which is detected as non-normal. Therefore, as the sample size increases, the number of quantization levels must increase to avoid this “change of distribution” from normal to binomial.

62

Normality Analysis for RFI Detection in Microwave Radiometry

Figure 4.5. Shapiro-Wilk validity test as a function of the sample length and the quantization level. Dotted and dash-dotted lines represent invalid test cases (error >5%), solid lines represent cases where the test is valid (error <5%). Lower number of bits in lines of the same color (same sample size) lead to sample sets “detected” as interference.

Figure 4.6 shows the validity of the normality tests as a function of the quantization bits (and the sample size (the lower the better, except for K2 and JB test). Actually, the validity of the tests is more influenced by the number of quantization bits than by sample size, provided it is high enough.

Figure 4.6. Minimum number of bits to neglect the quantization error, ECDF based test need more quantization bits as the sample size increase. As chi2 test, K2 test and JB tests are asymptotic, do not work properly for a low sample size. K, S and LM tests do not appear in graph as performance of these tests is acceptable for less than 6 quantization bits even for a sample size of 216.

63

Chapter 4

4.3 Performance of Normality Tests for Different Types of RFI
The performance of each normality test is calculated for different types of RFI representative of the ones actually encountered.
Pulsed sinusoidal signal: this signal has been extensively studied in microwave

radiometry [29, 31, 38, 45-48], as it is a common interference signal (e.g. a radar signal or a third order intermodulation product spurious signal). The signal model of this RFI is described as:

PS [i ] = A cos ( 2π f 0Ts i + ϕ0 ) H [i ]

i= 1..N ,

(4.20)

where PS[i] is the sampled pulsed sinusoidal signal, A, f0 and φ0 are the amplitude, frequency, and initial phase of the RFI respectively, Ts is the sampling period, and H[i] is a train of pulses described as:
1 mod M ( i ) ≤ M i DC  H [i ] =  0 otherwise  i= 1..N , (4.21)

where N is the sample length, M is the pulse length, and DC is the duty cycle factor of every pulse. Hence the RFI is a train of N/M pulsed sinusoidal signals of DC duty cycle factor.

Figure 4.7. Representation of H[i] function, where N is the total sample size, N/M is the total number of pulses and DC*M is the pulse length (in samples).

Pulsed Chirp signal: A chirp consists of a linearly varying frequency sinusoidal

signal. The signal model of the RFI chirp is described as:

CH [i ] = A cos ( ( 2π f 0 + πβ Ts i ) Ts i + ϕ0 ) H [i ]

i= 1..N ,

(4.22)

where CH[i] is the sampled pulsed chirp signal, β is the chirp rate of the linear frequency modulation (which corresponds to the slope of the frequency variation), A, f0

64

Normality Analysis for RFI Detection in Microwave Radiometry

and φ0 are the amplitude, initial frequency, and initial phase of the RFI respectively, and H[i] is the train of pulses function described in eqn. (4.21).
Pseudo-Random Noise (PRN) signal: a PRN signal is a signal that satisfies one

or more standard tests for statistical randomness. This signal consists of a deterministic sequence of pulses with (-1 and 1 values) that repeats itself after a period, which is usually very long, leading to a spread spectrum behavior of the signal. Without loss of generality in this work, the firsts 10,230 output bits of a MLSG of 14 stages (Fig. 4.8) are used as the deterministic sequence of the PRN interfering signal.

Figure 4.8. MLSG from which the PRN interfering signal used in this work is obtained. PRN interfering signal is composed by the repetition of the first 10,230 output bits

Pulsed PRN signal: as it has been stated before, a pulsed sinusoidal signal of

50% of duty cycle is not detected by the Kurtosis algorithm. The purpose of testing a pulsed PRN signal is to check a blind spot exists for this type of signals. This signal can be defined as:

PRN [i ] = Av [i ] H [i ]

i= 1..N ,

(4.23)

where PRN[i] is the sampled pulsed PRN signal, A is the amplitude of the signal and v[i] is the bit obtained from the Maximum Length Sequence Generator (MLSG) described in Fig. 4.8 whose values can be 1 and -1.
Telegraphic signal: this signal is a baseband digital amplitude modulated signal

(i.e. Amplitude Shift Keying (ASK) or On-Off Shift Keying (OOSK)). Telegraphic signal consist of a sequence of pulses (which values can be 0 and 1 or −1 and 1) where every bit of the message is modulated as one independent pulse. Duration of the pulse

65

Chapter 4

will determine the transmission speed. This signal is usually scrambled as originally it contents a high entropy value.
Pulsed telegraphic signal: in the same way, pulsed PRN signals have been

tested in order to search for blind spots, is also interesting to test the pulsed telegraphic signals.
Pulsed Orthogonal Frequency Division Multiplexing (OFDM) signal:

OFDM modulation consists of the division of the transmitted data in several parallel bit streams and the modulation of each of these data streams onto individual subcarriers. As each independent data stream must be orthogonal, adjacent carriers are separated by an integer multiple of the inverse of symbol duration of the parallel bit, hence multiple carriers can be transmitted simultaneously without problems. In other words, the entire channel is occupied through the aggregated sum of the narrow orthogonal subbands [63]. The discrete signal model of an OFDM RFI is defined as [63]:
OFDM [ n ] = A ∑ S m cos ( 2π f mTs n ) ,
m=0 M C −1

(4.24)

where Mc is the number of independent carriers (i.e. data streams), Sm are the Mc parallel modulated source symbols, A is the amplitude of the RFI signal, and fm is the mth subcarrier, defined as:
fm = m , TSymbol m = 0,..., M C − 1 ,

(4.25)

where TSymbol is the OFDM symbol duration. Source symbols have a determined distribution which affects the distribution of the OFDM modulated signal. Therefore OFDM modulated signals with differently distributed source symbols will present different distributions, thus different results for our normality tests. This is the reason to include two different OFDM modulated signals, the two signals used as symbols of an OFDM modulated signals are two defined previously, the PRN signal and the telegraphic uniformly distributed signal.

The performance of the different normality tests in the detection of the described RFI types is measured in terms of the required INR to obtain a ROC curve with a Pdet = 0.9 for a Pfa = 0.1, these values have been chosen only to make a comparison of

66

Normality Analysis for RFI Detection in Microwave Radiometry

all normality tests. If the INR to obtain this ROC curve by a determined normality test is higher than other one, it means that for a fixed INR and Pfa values, the first normality test will have a lower Pdet than the second one; thus, a lower INR value will indicate a better performance of the normality test in the RFI detection.

In order to get reliable results, the performance has been calculated as the average of 215 Monte-Carlo simulations. In the figures which represent the performance of the different normality tests, some tests are not plotted due to its poor performance, as it is usually the case of S, LM and CHI2 tests.

4.3.1 Pulsed Sinusoidal and Chirp Signal
It is widely known [29, 45-47] that a pulsed sinusoidal interfering signal of 0.5 duty cycle cannot be detected by the Kurtosis test, as this signal has a Kurtosis equal to 3, independently of the frequency of the interfering signal. Hence, a study of different alternatives to detect non-Gaussian signals is performed. Figure 6a shows the performance of different normality tests for sample sizes of 1,024 samples (dotted lines) and 16,384 samples (solid lines).

a)

b)

Figure 4.9. Normality test performance in the detection of a pulsed sinusoidal interference of: a) 1,024 samples (dotted line) and 16,384 samples (solid line), and b) a chirp signal of 16,384 samples as a function of signal’s duty cycle. ROC curve with a Pdet = 0.9 for a Pfa = 0.1 in both cases.

67

Chapter 4

For the generation of the simulated RFI, parameters f0 and φ0 are selected at random for the pulsed sinusoidal interference, while are defined as f0 = 2 × 10−5 and fN = 0.15 with N =16,384; and φ0 = 0º for the pulsed chirp interference. As it can be seen in Fig. 4.9, the best normality tests to detect pulsed sinusoidal interfering of duty cycle around 0.5 are the AD, the SW, the CM and the L tests, with improving performance as the sample size increases. These four ECDF based normality tests perform in a similar manner, having a better performance the SW test for shorter sample sizes, and the AD test for longer sample sizes. The CM and L tests have a performance in between the other tests for large duty cycle. The performance of all the normality tests in the detection of an interfering chirp signal with a variable duty cycle is very similar to the case of the detection of a pulsed sinusoidal signal, comparing results of Figs. 4.9a and 4.9b.

However, for duty cycle values different from 0.5 and nearby values, K test and kurtosis-related tests outperform. In Fig. 4.9a and 4.9b skewness and kurtosis-related tests have a peak around 0.5 that narrows with increasing sample size. Performance of JB and K2 test are worse than the K test alone, since these two tests depend also on the skewness parameter, which is zero in the analyzed signal.

In Fig. 4.10 the performance of the AD, SW, L and CM normality tests in the detection of a pulsed sinusoidal signal of exactly 0.5 duty cycle (Fig. 4.10a), and a chirp signal is compared as a function of the sample size.

The K, JB and K2 tests are not present in Fig. 4.10a since they cannot detect sinusoidal signals of 0.5 duty cycle. The AD, L and CM tests follow almost the same trend while the SW test has a different trend for sample lengths of 4,096 and above, since blocks of 2,048 samples have to be averaged to ensure a good performance of the test. On the other hand, in the 4.10b the K algorithm has the best performance followed by the kurtosis-based normality tests (JB and K2). In both cases the SW test performance does not improve as fast as the others above 2,048 samples due to averaging and CM test has a slightly worse performance than AD test.

68

Normality Analysis for RFI Detection in Microwave Radiometry

a)

b)

Figure 4.10. Normality test performance in the detection of: a) 0.5 duty cycle pulsed sinusoidal interference, and b) 1 duty cycle chirp interfering signal; as a function of the signal’s sample size. ROC curve with a Pdet = 0.9 for a Pfa = 0.1 in both cases.

4.3.2 Pseudo-random noise signal
Figure 4.11 shows the performance of the different tests vs the sample size. The K test achieves the best performance in the detection of this kind of interfering signal, followed by JB and K2 tests. Hence, kurtosis based tests perform better than ECDFbased tests (AD, L, CM, SW) in the detection of PRN signals. Performance of ECDFbased tests is quite similar to the sinusoidal and chirp interfering signals, obtaining the best results with the AD and the SW tests for lower sample sizes, and with the AD and CM tests for higher sample sizes. SW test performance is not degraded for large sample size due to averaging.

69

Chapter 4

Figure 4.11. Normality test performance in the detection of a PRN interference as a function of the signal’s sample size, ROC curve with a Pdet = 0.9 for a Pfa = 0.1.

Figure 4.12 shows the ROC curves of the normality tests performance in the detection of a PRN signal of 16,384 number of samples and a INR of -5.2 dB; in this figure, K gets a value of Pdet = 0.9 for Pfa = 0.1. and is the only normality test to get this Pdet and Pfa results. In this figure it is clearly shown that the K test outperforms for the same INR over the rest of normality tests, followed by the two kurtosis and skewness-based tests (JB and K2 tests) which have almost the same behavior. The four ECDF-based tests (AD, CM, L and SW tests) have poorer performance. The worst performance is obtained by the S, LM and CHI2 tests. The S and LM tests fail as the interfering signal Skewness is zero. CHI2 test has a poorer performance in the RFI detection than the ECDF-based tests, and Kurtosis-based tests, therefore it is not recommended in the RFI detection.

Figure 4.12. Normality tests performance in the detection of a PRN interference signal.

70

Normality Analysis for RFI Detection in Microwave Radiometry

4.3.3 Pulsed PRN signal
Figure 4.13 shows the performance of the different normality tests in the detection of a Pulsed PRN interfering signal. In this figure, the JB and K2 tests are not represented as its performance is always slightly worse than the K test.

A blind spot in the Kurtosis detection algorithm can be observed, in this case for a duty cycle of 0.333, which is not present in the rest of the RFI detection algorithms. In fact, the four ECDF-based tests perform similarly; the SW tests performs slightly worse as it has been averaged. AD test performs better than the rest of ECDF tests. Although the blind spot issue, K test performs better than the others detection algorithms for duty cycle values outside the blind spot.

Figure 4.13. Normality test performance in the detection of a pulsed PRN interference of 16,384 samples as a function of the signal’s duty cycle, ROC curve with a Pdet = 0.9 for a Pfa = 0.1.

4.3.4 Telegraphic signal
For this type of interfering signal, depending on the actual message transmitted the performance of the different normality test is quite variable. In this study, three different interfering signals have been used, called messages 1, 2 and 3. Message 1 is a plain text file (very low randomness), message 2 is a zipped file, and message 3 is a jpg file (high level of randomness, as redundancy is eliminated). In Fig. 4.14 and 4.15 ROC curves of message 1 are presented as a function of the sample size and the INR, respectively. In Fig. 4.14, the INR is fixed to -16 dB, and in Fig. 4.15 the sample size is fixed to 2048 samples. As for other kinds of interfering signals increasing the sample 71

Chapter 4

size increases the probability of RFI detection, while increasing the INR increases the probability of RFI detection.

In Fig. 4.14 it is shown that the best normality tests for detecting message 1 varies depending of the sample size: for 2,048 samples it is the SW test, while for 1,024 samples they are the JB and the K2 tests (highest Pdet with a low Pfa), as both values of kurtosis and skewness are non-zero; the performance of these tests is far better than AD, CM and L tests. Performance of the normality tests varying the INR value is very similar than the case when the sample size is varied, as it can be seen in Fig. 4.15, where the test which perform better are the SW, the JB and the K2.

In Fig. 4.16 and 4.17 the ROC curves of the three different messages are presented. Figure 4.16 presents the normality test performance in the detection of the three different messages with the same INR of -20dB and the same sample size of 16384. Message 1 is relatively easy to detect by any normality test due to its statistical nature (plain text file with high redundancy) while messages 2 and 3 are undetectable for INR = -20dB due to the low redundancy of these messages (compressed data), therefore, a new simulation with a higher INR value (INR = -5.2dB) is presented in Fig. 4.17. As it as been said in the previous figures the best normality tests for detecting message 1 is JB and K2 tests as both values of kurtosis and skewness are non-zero, followed by the SW test.

Figure 4.17 presents the normality test performance in the detection of the messages 1, 2 and 3 with a higher INR of -5.2dB and the same sample size of 16384. The message 1 is detected by all normality tests as the INR is high for the detection of this highly redundant signal. On the other hand, messages 2 and 3 are not so easily detected due to its low redundancy, in this case the best normality tests to detect low redundant telegraphic signal is again the K test. In fact, the relative performance of the normality tests is exactly the same as the PRN case (Fig. 4.12).

To simulate message scrambling and encryption, all 3 messages are scrambled by means of an XOR operation between original message and the PRN code previously studied in this work, using the parameters of simulation of INR = -5.2dB and sample size = 16384. Results obtained are shown in Fig. 4.18 which has to be compared with 72

Normality Analysis for RFI Detection in Microwave Radiometry

the result obtained in Fig. 4.12. The high similarity between Fig. 4.12 and the result of the test detection of the scrambled signal of all three messages and the PRN signal (Fig. 4.18), shows that the scrambling process usually employed in communications makes the detection of RFI more difficult. It can be observed that all ROC curves are almost equal for each interfering signal, deducing that if the telegraphic signal is encrypted or scrambled, front the point of view of detectability, it behaves as a spread spectrum signal

a)

b)

c)

Figure 4.14. Normality tests performance in the detection of a telegraphic interference signal (message 1). INR = -16 dB for the three cases; a) 16384 samples. b) 2048 samples. c) 1024 samples.

a)

b)

c)

Figure 4.15. Normality tests performance in the detection of a telegraphic interference signal (message 1). 2048 samples for the three cases; a) INR = -10 dB. b) INR = -15 dB. c) INR = -20 dB.

a)

b)

c)

Figure 4.16. Normality test performance in the detection of a telegraphic interference signal. INR = -20 dB and 16384 samples for the three cases; a) Message 1. b) Message 2. c) Message 3.

73

Chapter 4

a)

b)

c)

Figure 4.17. Normality test performance in the detection of a telegraphic interference signal. INR = -5.2 dB and 16384 samples for the three cases; a) Message 1. b) Message 2. c) Message 3.

a)

b)

c)

Figure 4.18. Normality test performance in the detection of a telegraphic interference signal scrambled with the PRN signal obtained with the MLSG detailed in Figure 4.8. INR = -5.2 dB and 16384 samples for the three cases; a) Scrambled message 1. b) Scrambled message 2. c) Scrambled message 3.

4.3.5 Pulsed Telegraphic Signal
In this case, in order to choose a ‘message’ with the lowest entropy, the message sent has been substituted by an uniformly distributed stochastic process. Figure 4.19 shows the performance of the ECFD and K normality tests in the detection of a Pulsed 8-level ASK uniformly distributed telegraphic interfering signal, in this case K2 and JB tests are not included as perform slightly worse than K test.

Another blind spot appears in the Kurtosis detection algorithm (duty cycle of 0.587), which again is not present in the ECFD-based RFI detection algorithms. Also, the AD test performs better than the rest of ECFD tests followed by the CM test since the SW test performance is degraded due to averaging. Again, the K test outperforms for duty cycle values outside the blind spot.

74

Normality Analysis for RFI Detection in Microwave Radiometry

Figure 4.19. Normality test performance in the detection of a pulsed telegraphic signal interference of 16384 samples as a function of the signal’s duty cycle, ROC curve with a Pdet = 0.9 for a Pfa = 0.1.

4.3.6 Pulsed OFDM signal
Figure 4.20a shows the performance of the different normality tests in the detection of a Pulsed PRN over OFDM interfering signal which consists in a PRN signal OFDM modulated afterwards. In this case no blind spots are present in the Kurtosis algorithm, having the best performance for all duty cycle, followed by the AD and SW tests.

Figure 4.20b shows the performance of the different normality tests in the detection of a Pulsed 8-level ASK uniformly distributed telegraphic over OFDM interfering signal (uniformly distributed telegraphic signal OFDM modulated afterwards). Kurtosis blind spot is present for a duty cycle of 1, therefore if the signal is not pulsed, it will be undetectable for the K test. In addition, the rest of the normality tests seem not to perform properly, as they need a very high INR to be able to detect this interference, (for example SW test will detect the interfering signal with an INR higher than 9 dB). Again, K test obtains the best results outside the blind spot, followed by the AD test.

75

Chapter 4

a)

b)

Figure 4.20. Normality test performance in the detection of: a) pulsed PRN modulated over OFDM interference of 16,384 samples, and b) pulsed 8 level ASK Telegraphic uniformly distributed modulated over OFDM interference of 16,384 samples; as a function of signal’s duty cycle. ROC curve with a Pdet = 0.9 for a Pfa = 0.1 in both cases.

4.4 Blind spot detection in Kurtosis based algorithms
As it has been described in section 4.3, the K test and the kurtosis-based tests (JB and K2 tests) present blind spots for different pulsed signals.

The reason of all these blind spots present in the K test is that the Kurtosis takes a value of 3, the same value than for a Gaussian signal. In Fig. 4.21, values of the kurtosis parameter of five different pulsed functions are represented in function of its duty cycle. It can be observed that kurtosis value is very high for low duty cycles, diminishing with duty cycle until a determined value which is the value of the function when it is not pulsed (i.e. duty cycle of 1).

76

Normality Analysis for RFI Detection in Microwave Radiometry

Fig. 4.21. Kurtosis value as a function of signal’s duty cycle for 5 different signals. Note the red circles marking the duty cycle value where the K detection algorithm will present a blind spot.

Therefore, it would be useful to determine when this blind spot detection is present in a determined interfering signal, making necessary the study of this situation mathematically.

Assuming a determined zero-mean pulsed discrete signal XDC, with duty cycle value DC, and assuming KX as the value of the kurtosis parameter when the signal is not pulsed (duty cycle equal to 1), the kurtosis parameter of XDC ( K X DC ), can be computed as:

K X DC

E  X DC − X DC   = E X − X   DC DC  

(

)  
4 2

(

)

  

2

.

(4.24)

As XDC is zero-mean eqn. (4.24) becomes:

K X DC

4 E ( X DC )    . = 2 2 E ( X DC )   

(

)

(4.25)

In eqn. (4.26) the expectation operator is expanded, and a number of 1-DC·N values are reduced to zero, as XDC is pulsed. The rest of values can be represented as values from X as since the duty cycle is not taken into account, XDC and X are the same signals:

77

Chapter 4

K X DC = . 2 N  1  DC · N 2 2    ∑ ( X ) + ∑ ( 0)   n = DC · N +1   N  n =1

N 1  DC · N 4 4 ( X ) + ∑ ( 0)  ∑ N  n =1 n = DC · N +1 

(4.26)

Then, if the number of samples of the signal N tends to infinity (i.e. if it is high enough): 1 ( DC·N ·m4 )  N → , 2 2 N →∞  1  DC · N 1  2   ( DC · N ·m2 )    ∑ ( X )  N  N  n =1   1  DC · N 4  ∑ (X )  N  n =1 

(4.27)

where m2 and m4 are the second and the fourth order moments of X, respectively. Simplifying eqn. (4.27):

K X DC =

( DC·m2 )

DC ·m4

2

=

1 m4 , DC ( m2 )2

(4.28)

and taking into account that the kurtosis is equal to the fourth moment divided by the square of the second moment, eqn. (4.28) becomes:

K X DC =

1 KX , DC

(4.29)

where K X DC is the value of the kurtosis parameter of a determined pulsed signal XDC, with a determined duty cycle DC, and KX is the value of the kurtosis parameter of the zero-mean signal X (duty cycle of 1).

As it has been demonstrated, the kurtosis of a zero-mean pulsed signal depends only on the duty cycle and the value of the kurtosis parameter when it is not pulsed. As it can be seen in Fig. 4.21, kurtosis value of a pulsed signal will always cross the value of 3, if the kurtosis of the signal with duty cycle equal to 1 has a value lower than 3. This duty cycle value can be easily calculated from eqn. (4.29), leading to:
DC = KX 3 (4.30)

Thus, any interfering signal with a kurtosis value lower than 3 will present a blind spot in its detection for a duty cycle given by eqn. (4.30).

78

Normality Analysis for RFI Detection in Microwave Radiometry

4.5 Conclusions
In this chapter the performance of ten different normality tests has been analyzed in terms of their capability to detect radio-frequency interference in microwave radiometry. These tests have been first validated in terms of sequence length and number of quantization bits in the absence of interference. Their capability to detect sinusoidal, chirp, PRN, telegraphic and OFDM signals has then been analyzed. It has been shown that the Kurtosis is the best RFI detection algorithm for all
kinds of continuous interfering signals, although it is known that it presents blind spots for pulsed sinusoidal, chirp, PRN, telegraphic and OFDM interfering signals.

As compared to all the other normality tests, CHI2 normality test has a poor performance in analyzing all the presented RFI signals, therefore its use is not
recommended for RFI detection.

Skewness-based algorithms (S and LM) usually have a poorer performance than other tests as the Skewness of PRN, sinusoidal, and chirp interfering signals is almost zero. However, non-scrambled telegraphic signals present a higher Skewness parameter leading to a better performance of the S and LM tests than in case of sinusoidal, chirp, PRN, and telegraphic scrambled signals. Nevertheless, S and LM tests are not suitable
for the RFI detection for its overall poor performance.

Kurtosis-based normality tests (JB and K2 tests) have a good performance if both the kurtosis and the skewness are high enough. However, since skewness is usually almost zero, both tests have a performance slightly worse than the K test, except in the case of non-scrambled telegraphic interfering signals. Their performance is very similar, although the JB test has always a slightly better performance. In any case, both tests present the same blind spots as the K test in the detection of pulsed interfering signals. Hence, the use of the JB and K2 tests is redundant and not necessary if the K test is
being used.

79

Chapter 4

The four Empirical Distribution Function (EDF) based normality tests: AD, L, CM, and SW tests have a similar performance for OFDM, PRN, sinusoidal and chirp interfering signals. Among the AD, L and CM tests, which are based on the comparison of a normality distribution function with the EDF of the tested signal, the AD is the one that performs best as it assigns more weight to the tails than the other two tests, and the RFI is usually located in the tails of the distribution. For a low sample size the AD and SW tests perform better that the CM and L test, but as the sample size increases, SW test performance degrades in front of CM and L tests, as SW test must be averaged above 4,096 sample size to obtain a correct performance. Hence, among the ECDF tests,
the AD and the SW tests are recommended over the L and CM tests; and in case that the number of samples is higher of 4096, the AD test is recommended over the SW test.

In summary, the Kurtosis is the best RFI detection algorithm for almost all kinds of interfering signals, although it has a blind spot for several pulsed signals. The AD test is a complementary normality test that covers these blind spots, and has a very good performance for all the studied sample sizes. The combination of the K and the AD
tests seems capable to detect most types of RFI. The performance of the detection

tests improves with the sample size and depends on the duty cycle of the pulsed RFI.

Future research will be devoted to the optimum combination of these statistical analysis with time and frequency blanking methods, since these methods outperform statistical analysis in some specific cases for example low duty cycle pulsed sinusoidal signals (short pulses are easily detected in time domain), or high duty cycle pulsed sinusoidal signals or a CW (a tone is easily detected in frequency domain).

80

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

Chapter 5
5. RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

The radiometric signal’s spectrogram combines the time and frequency domains analysis jointly, standing as a powerful RFI detection tool. A spectrogram is obtained from the Short Time Fourier Transform (STFT), thus it arranges the signal information in time and frequency domains. Therefore, RFI very localized in the time and/or frequency domains present in the radiometric signal appears more concentrated in the spectrogram and it is easier to detect. The main idea of this chapter is taking into account that the spectrogram is a two-dimensional intensity plot which can be analyzed as an image, thus having several image processing tools at our hand.

81

Chapter 5

Combining the time and frequency domains analysis jointly, the spectrogram stands as a powerful tool which has been previously used in RFI detection in radioastronomy [25, 64]. The spectrogram consists of an intensity plot (usually on a decibel scale) of the STFT magnitude [65]. The STFT consists of a set of FT’s of consecutive windowed data segments from a longer data set. Windows usually overlap in time, thus data segments may have redundant information, as sketched in Fig. 5.1. The STFT provides time-localized spectral information of the frequency components of a signal varying over time, whereas the standard FT provides the frequency information averaged over the entire signal time interval [65]. As a spectrogram is a twodimensional intensity plot, it can be analyzed as an image, thus having all the image processing tools at our hand.

1) Thermal noise sampled signal

2) Windowing overlapped segments of 1)

3) FFT of every windowed segment of 2)

4) A two-dimensional image created by ordering the FFTs in columns

Figure 5.1. Process of obtaining a Spectrogram from a sampled signal of thermal noise.

In this chapter, two different thresholding algorithms to detect and eliminate interference patterns in radiometric signal spectrograms are developed, and simulation results are presented. The simulated RFI signals are generically from the sinusoidal and chirp families. By adjusting the signals’ parameters, the bandwidth and temporal duration, many different types of signals can be obtained. A two-dimensional (2D) Wiener filter is then applied to the spectrogram in order to try to improve the

82

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

cancellation of RFI components in the radiometric signal. A brief description of the spectrogram calculation is given in Section 5.1. Section 5.2 describes some image processing algorithm proposed in this chapter, which are called “Smoothing Algorithm”, “Wiener Filter Algorithm”, “Edge Detection Algorithm” and

“Frequency/time Interval Averaging and Thresholding (FIAT) Algorithm”. Section 5.3 shows the results obtained with the application of “Smoothing Algorithm” and the “Wiener Filter Algorithm” to a thermal noise signal contaminated with a set of RFI simulated signals. Finally, Section 5.4 summarizes the conclusions of this chapter.

5.1 Spectrogram calculation
In this work, the radiometric signals are assumed to be sampled at an adequate sampling rate satisfying the Nyquist criterion, and the spectrograms are obtained from these discrete time signals.

The spectrogram calculation depends on several parameters which are: the total number of signal samples, the FFT size, the window used for the FFT calculation of the data segments, and the overlapping between these data segments, as shown in Fig. 5.1.

The spectrogram used in this paper is calculated using discrete samples, each one corresponding to a measurement of the pre-detected voltage signal. The number of samples (NS) which forms the spectrogram will determine the radiometric resolution obtained by each spectrogram.

The FFT size is the number of samples used to compute each FFT and determines the spectral and temporal resolutions. If the FFT size (L) increases, the spectral resolution increases, while the temporal resolution decreases, since the time lapse between consecutive data segments increases. In contrast, smaller FFT sizes lead to lower frequency resolution, and higher temporal resolution. Therefore, selection of the FFT size must be chosen carefully depending on the RFI present in the radiometric measurements.

83

Chapter 5

The FFT calculation requires the use of an analysis window function in order to minimize unwanted “side lobes” and ringing in the FFT resulting from abrupt truncations at both ends of the data segment. This situation causes an oscillatory behavior in the FFT called the Gibbs phenomenon [66]. Several functions can be used to avoid this problem, though when the data source is unknown, the Hann window is one of the most common windows used in spectrum analysis, because of its excellent roll-off rate at 18 dB/octave [67].

In order to avoid loss of data when using a window in the FFT calculation, some overlapping (O) must exist between consecutive data segments (Fig. 5.1). In addition, overlapping increases the temporal resolution as the beginning of the data segments is reduced. However, increasing the overlap leads to an increase of the total number of samples to be computed and managed, complicating the hardware design. The recommended overlap factor is 75% for a Hann window [68, 69]; and this is the value used in this work. Figure 5.2 represents this 75% overlapping compared to the cases of 0% and 50% overlapping.

Figure 5.2. Representation of different levels of overlapping. It can be noted that low levels of overlapping will result in an information loss, and high levels of overlapping will lead to correlated FFT values on overlapped time segments.

84

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

The spectrograms developed for the “Smoothing algorithm” have the following parameters: • Number of samples: this parameter must be as high as possible to maximize the radiometric resolution, as a higher number of samples means a longer integration time. On the other hand, too large values increase the computation time required to perform the Monte-Carlo simulations; in addition it requires more storage capacity in real measurements. For simplicity, the number of samples will defined as 22(n-1). • Overlap: An overlap value of O = 75% is recommended [68, 69] to avoid information loss. • FFT size: The FFT size determines the frequency resolution and the number of FFT segments, determines the temporal resolution. For the simulations performed in this work, and supposing that any information of the RFI is known, the FFT size was chosen to be equal to the number of FFT segments, to have similar spectral and temporal resolutions (both resolutions may not be the optimal ones); thus the FFT size should be the square root of the number of samples (N). Taking into account that there exists a 75% overlap between consecutive Hann windows, the number of FFT segments has to be multiplied by 4. Thus, to equilibrate both resolutions (time and frequency), maintaining the original compromise, the FFT size has been selected to be L = 2n, and now the total number of samples is 22n (accounting for redundant points due to overlapping).

As it has been said, the spectrogram operation consists of the power (square of the absolute value) of the STFT magnitude. The spectrogram operation changes the probability density function of the received thermal noise which is a pair of Gaussian distributed random variables (one for the in-phase component and the other one for the quadrature component). The square of the amplitude is equal to the sum of the squares of both Gaussian distributions, which is the definition of a chi-square distribution with two degrees of freedom, which is equivalent to an exponential distribution [70]. Figure 5.3 sketches this process.

85

Chapter 5

Figure 5.3. Transformation of the zero-mean Gaussian noise distributions (in-phase and quadrature components) into an exponential distribution with same standard deviation and mean [71].

5.2 Spectrogram analysis algorithms
5.2.1 Smoothing algorithm description
The most obvious way to detect the presence of interference in a radiometric signal is by detecting power peaks in the received signal that are larger than the variance of the measured thermal noise in the absence of RFI. This detection can be performed either in both the time and frequency domains.

This technique can be straightforwardly extended to the spectrogram. Considering that a power peak is produced by an RFI signal, the threshold value must be a function of the thermal noise variance (power), and must maximize the probability of RFI detection (Pdet) while minimizing the probability of false alarm (Pfa). The probability of detection of RFI cannot be computed in advance since the presence of

86

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

RFI is unknown, but the probability of false alarm is easy to obtain as the thermal noise follows a known Gaussian distribution.

All simulations performed in this work have been performed assuming an antenna temperature (TA) of 300 K, and a radiometer receiver’s noise temperature of 100 K. In order to make a relationship between the noise power, and the RF interfering signal power, the INR parameter, described in eqn. (4.19) is used.

The threshold value of this algorithm must be selected to limit the error in the estimated antenna temperature produced by false alarms.

If an RFI-free spectrogram is observed (Fig. 5.1, panel 4) it is obvious that the noise power peaks are scattered (and usually isolated) in both frequency and time, while in an RFI contaminated spectrogram (e.g., Fig. 5.5a), the RFI is usually clustered in determined regions in the frequency-time plane. Considering the spectrogram as an image, noise power peaks present higher spatial frequency contents (rapid variations) than the RFI, which are usually clustered. Hence, applying a 2D smoothing filter (lowpass filter) will attenuate these exponential “peaks”, while the RFI contaminated regions will be preserved. RFI is preserved, so it can be detected with a more restrictive threshold.

The algorithm proposed in this Ph. D. thesis can be summarized as follows with the aid of Figs. 5.4 and 5.5:

22(n-1) samples data set Power Spectrogram 2n samples FFT Hanning window 75% overlap 2D Convolution Frequency equalization

*
NxN Hanning window

Thresholding

RFI mitigated Spectrogram

Σ

RFI mitigated TA

Figure 5.4. Smoothing algorithm block diagram.

87

Chapter 5

• Power spectrogram calculation of a data segment of 22(n-1) samples using data segments of 2n samples, with 75% overlap, and a Hann window (Fig. 5.5a).

• Convolution of the power spectrogram with a 2D low pass filter of a determined size (in this work the Hann window has been chosen, although a Gaussian or a Hamming window perform similarly (Fig. 5.5b)).

• Thresholding: a threshold is used in the smoothed power spectrogram, in order to detect clusters of RFI-contaminated pixels (Fig. 5.5c). The optimum threshold is discussed in Section 5.5.

• Antenna noise power is calculated by averaging all spectrogram pixels below the predefined threshold (RFI-free).

• Finally, TA is obtained from the antenna noise power.

88

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

a)

b)

c)

d)

Figure 5.5. Smoothing Algorithm applied to a simulated RFI contaminated radiometric signal. a) RFI contaminated signal’s spectrogram, σ2RFI = σ2n – 5 dB; b) Convolution of the spectrogram with a N × N = 15 × 15 Hann window; c) Thresholding for Pfa = 10−4: black pixels are considered RFI; d) RFI-mitigated spectrogram using “blanking” of the RFI contaminated pixels.

5.2.2 Wiener Filter Algorithm description
As already discussed, the spectrogram of a noise signal with sinusoidal interference signals can be considered as a noisy image, where the noise is the spectrogram of the radiometric signal (the one we want to measure), and the image to be detected is the spectrogram of the interference (the one to be cancelled). Therefore, designing a filter to eliminate the noise from the image is the way to estimate the RFI, for a later removal of the interference without loss of the radiometric data.

The Wiener filter is a well-known adaptive filter used in communications, which provides the best estimation of a signal, equalizes communications channel, and eliminates the noise present in the received signal. In this section the Wiener filter is used to estimate the RFI in the spectrogram image, for a later cancellation and its

89

Chapter 5

performance is compared to the smoothing algorithm. In Fig. 5.6 a diagram of the Wiener filter operation id presented.
s( tu , fv ) = y( tu , fv ) +n( tu , fv ) ˆ n ( tu , fv )

+

+

−

ˆ y ( tu , fv )
Figure 5.6. RFI mitigation technique: an estimation ŷ(tu, fv) of the RFI signal spectrogram is substracted from the received signal spectrogram.

In our case study, it is not necessary to know the effect of the communications channel in the spectrogram, thus, it will only be necessary to differentiate between the noise and the interfering signal. The way to perform this task is by using a Locally adaptive Linear Minimum Mean Square Error (LLMMSE) [72] to estimate the interfering signal’s spectrogram.

The LLMMSE algorithm consists of an optimal linear estimator of our interfering signal ŷ(tu, fv) in combination with additive noise:
ˆ y ( tu , f v ) = α s ( tu , f v ) + β ,

(5.1)

where

s ( tu , f v ) = y ( tu , f v ) + n ( tu , f v ) ,

(5.2)

and y(tu, fv) is the interfering signal spectrogram, n(tu, fv) is the spectrogram of the Gaussian noise (independent from the interfering signal spectrogram), tu is the uth time point of the spectrogram, fv is the vth frequency point, and α and β are two parameters chosen to minimize the mean square estimation error criterion ε described in eqn. (5.3):
2 2 ˆ ε (α , β ) =E ( y ( tu , f v ) − y ( tu , f v ) )  = E (α s ( tu , f v ) + β − y ( tu , f v ) )  .









(5.3)

The minimum error (ε) is found for:

∂ε (α , β ) =0 , ∂α

(5.4.a)

∂ε (α , β ) =0 , ∂β where ∂ denotes partial derivative.

(5.4.b)

90

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

From eqns. (5.3), (5.4.a) and (5.4.b), and taking into account that the additive noise power of the spectrogram does not have a zero mean, eqn. (5.5) can be derived:
ˆ y ( tu , f v ) = ( µ s ( tu , f v ) − µn ( tu , f v ) ) +

σ s2 ( tu , f v ) − σ n2 ( tu , f v ) ( s ( tu , fv ) − µs ( tu , fv ) ) , (5.5) σ s2 ( tu , f v )

where µs(tu, fv) represents the local mean of s(tu, fv) around a square of L × L pixels, (eqn. (5.6)), σs2(tu, fv) represents the local variance of s(tu, fv) around a square window of L × L pixels, (eqn. (5.7)), µn is the mean of the noise spectrogram, and σn2(tu, fv) is the variance of the noise spectrogram.

µ ¨ s ( tu , f v ) = σ s2 ( tu , f v ) =
with
M1 = M 2 =

1 L2

u+M2

i = u − M 1 j = v − M1

∑ ∑ s (t , f ) ,
i j 2 i j s u v

v+M 2

(5.6)

1 L2

u+M2

i = u − M 1 j = v − M1

∑ ∑ ( s (t , f ) − µ (t , f ))
L −1 ; for L odd , 2

v+M 2

,

(5.7)

(5.8)

and
M1 = L−2 L ; M 2 = ; for L even. 2 2

(5.9)

The use of different window sizes (L value, M1 + M2 + 1 = L) affects the resulting estimated interference ŷ(tu, fv). If L is too small, the noise filter algorithm is not effective. On the other hand, if L is too large subtle details of the interference will be lost in the filtering process. Figure 5.7 shows the simulated error in the estimation of the noise power as a function of the RFI power present in the received signal; assuming that the thermal noise power (σn2) is perfectly known. It is observed that the value of L depends on the actual RFI level: if the RFI level is 17 dB lower than the thermal noise, the best choice is a 5 × 5 window, otherwise, a 6 × 6 window will outperform. In our case of study, the 6 × 6 window is chosen as the maximum value of the algorithm error is 0.74 K.

91

Chapter 5
2.5 2 1.5 1 0.5 0 -60

4x4 filter size 5x5 filter size 6x6 filter size 7x7 filter size 8x8 filter size

ε T [K]

A

-40

-20

0

20

Interference to Noise Ratio (dB)
Figure 5.7. Error in the estimation of the antenna temperature as a function of the Interference to Noise Ratio (INR).

5.2.3 Edge detection algorithm description
The Smoothing Algorithm was developed under the assumption of time and frequency invariance of the received brightness temperature for the set of measured samples used to obtain the spectrogram. This is usually attained by limiting the time duration of the samples set to a time in which the brightness temperature variations are negligible and limiting the bandwidth so that the brightness temperature variations are also negligible.

However, even if the time and bandwidth have been properly selected, the frequency response of the radiometer will also affect the measured brightness temperature. Therefore, the frequency response of the instrument must be flat, or at least must be properly pre-equalized.

Edge detection algorithm has been developed in the context of using a spectrum analyzer as a back end radiometer; by this way measured brightness temperature is automatically separated in frequency bands (more information about this architecture is explained in Chapter 7). Using a spectrum analyzer allows several degrees of freedom in the selection of the frequency band and the bandwidth to measure; in the other hand, frequency response of the resolution bandwidth for the spectrum analyzer must be taken into account, and the obtained power samples have been averaged in time and frequency

92

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

by the detector of the spectrum analyzer thus having a power samples with a PDF different than the exponential. Edge detection algorithm does not need the equalization of the frequency response of the hardware used to measure, neither the limitation of the time duration or bandwidth of the measurement to avoid noise power variations on the spectrogram; moreover, this algorithm is independent of the PDF of the noise power, as it does not use the tables used by the smoothing algorithm.

The proposed edge detection algorithm applies a 2D edge-detection waveletbased filtering to detect the most important part of the RFI. Then, frequency and time averaging techniques are used to detect residual RFI both in time and frequency, respectively, and to eliminate the residual RFI afterwards.

The 2D filtering process consists of the sequential convolution of the spectrogram with two Wavelet Line Detection (WLD) filters. These filters consist of two different matrices, one to detect mainly frequency interferences (Frequency Wavelet Line Detection or FWLD), and the other to detect a temporaly peak (broadband) interferences (Time Wavelet Line Detection or TWLD).

The FWLD filter is a matrix with a determined number of rows where every row is composed by the coefficient values of a Mexican Hat wavelet of 11 samples length (Fig. 5.8); on the other hand, TWLD filter has a determined number of columns each one of them being also composed by the same coefficient values. Rows of the FWLB filter and columns of the TWLB filter are in function of the spectrogram size:
15 → TP < 280  = 2 ⋅ TP / 40  + 1 → 280 ≤ TP ≤ 600   31 → TP > 600 

FWLBrows

(5.10)

TWLBcolumns

15 → FP < 280  = 2 ⋅  FP / 40  + 1 → 280 ≤ FP ≤ 600   31 → FP > 600 

(5.11)

where ⋅ represents the round down operator and TP and FP are the time and  frequency points of the acquired spectrogram respectively.

93

Chapter 5

The TWLB and FWLB filters have a minimum size to ensure a sufficient increase on the convolved value in presence of RFI, and a maximum size to limit the free RFI pixel elimination.

Figure 5.8. Mexican hat coefficient values

This way, by correlating these filters with the spectrogram the vertical/horizontal lines whose pixels have a higher power value than its neighbours (RFI contaminated pixels) are enhanced. A similar technique has been performed previously in [25].

The thresholds used in the 2D filtering process must be calculated as a function of the standard deviation of the RFI-free thermal noise power value (σpow), so an estimation of this value ( σ pow ) must be first obtained.

The thresholds used to detect RFI in the filtered spectrograms are calculated as a function of σ pow by means of the central limit theorem; thus, threshold of the FWLD filtered spectrogram (ThFWLD) can be expressed as:

Th FWLD = Kσ FWLD , with:
σ FWLD =

(5.12)

∑ (c σ
5 i i=1

pow

2N

) + (c σ
2 6

pow

N

)

2

,

(5.13)

where K is a constant to determine the probability of false alarm (Pfa) of the detection algorithm, σ pow is the approximation of the standard deviation of the RFI free FWLD filtered signal, ci is the value of the ith coefficient of the Mexican hat wavelet of 11 samples length and N is the number of rows of the FWLD filter; threshold of the TWLD filtered spectrogram can be calculated similarly.

94

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

In principle, the distribution of the filtered spectrogram pixels should be Gaussian due to the central limit theorem. In our case of study the distribution of the filtered pixels quite approximated to a Gaussian distribution, thus that of the σ FWLD parameter too. Therefore, the K parameter can be selected with the aid of the erf
function.

To calculate the retrieved power after the RFI mitigation process only RFI free pixels are accounted for.

Spectrogram
FWLD filter

*
FWLD threshold

2D Convolution

*
TWLB threshold

TWLD filter

&

Yes

1st pass RFI mitigation

Frequency subbands & Time sweeps average

Any frequency subband or time sweep with relatively high power (6 times above σ freq or σ time) value? No

RFI cleaned signal power
Figure 5.9. Edge detection algorithm block diagram.

5.2.4 Frequency/Time Averaging and Thresholding (FIAT) Algorithm description
This algorithm has been developed to apply directly after the result of the algorithms described in Sections 5.2.1 and 5.2.3, as there is residual RFI in the result of both algorithms that can be eliminated.

After the application of one of the algorithms described in Sections 5.2.1 or 5.2.3, frequency subbands and time sweeps are averaged independently. Thus, two

95

Chapter 5

arrays are obtained; one is related to the RFI present in the time domain, and the other is related to the RFI present in the frequency domain. After this process, a threshold which depends directly on the standard deviation of these vectors (σfreq and σtime) is applied. In this case, it is not needed to sort and eliminate the most powerful samples as it is assumed that the remaining RFI will not bias the result of the standard deviation in excess. The distribution of these vectors will be approximately Gaussian because of the central limit theorem.

This way, low level broadband RFI is detected by means of the power integration in the time domain, and low level CW RFI is detected by means of the power integration in the frequency domain. This algorithm is actually a onedimensional algorithm which does not need to calculate the spectrogram, but it is included in this section as it is better used after a spectrogram based algorithm. In fact, a similar algorithm with the same functionality has been previously used in [26].

The Smoothing Algorithm and the Edge Detection Algorithm diagrams can then be modified to include this algorithm. Hence, with the inclusion of the FIAT Algorithm the Smoothing Algorithm diagram of Fig. 5.4 can be enhanced to the diagram presented in the Fig. 5.10.

96

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

22(n-1) samples data set Power Spectrogram 2n samples FFT Hanning window 75% overlap 2D Convolution Frequency equalization

*
NxN Hanning window

Thresholding

RFI mitigated Spectrogram

Frequency integration

Time integration

Frequency threshold

&

Time threshold

2nd pass RFI mitigation

∑
RFI mitigated TA

Figure 5.10. Smoothing algorithm with FIAT algorithm enhancement block diagram.

In the same way, the Edge Detection Algorithm diagram can be enhanced to the diagram presented in the Fig. 5.11.

97

Chapter 5

Spectrogram
FWLD filter

*
FWLD threshold

2D Convolution

*
TWLB threshold

TWLD filter

&

Yes

1st pass RFI mitigation

Frequency subbands & Time sweeps average

Any frequency subband or time sweep with relatively high power (X times above σ freq or σ time) value? No Frequency integration Time integration

Frequency threshold

&

Time threshold

2nd pass RFI mitigation

∑

RFI cleaned signal power

Figure 5.11. Edge detection algorithm with FIAT algorithm enhancement block diagram.

FIAT algorithm is not recommended to use as an isolated algorithm, as it will eliminate a higher number of pixels than the smoothing algorithm or the edge detection algorithm. However, simulation results of the isolated FIAT algorithm performance compared with the isolated smoothing algorithm results, for determined classes of RFI contaminated signals will be presented.

5.3 Simulation results
In this section, results obtained from the simulation of two of the image processing algorithms presented (Smoothing and thresholding and 2D Wiener filtering) are shown and discussed. The other two algorithms, (Edge Detection and FIAT) have not been simulated, thus the results of these algorithms using radiometric data jointly with the results of the Smoothing Algorithm with real data will be included in the Chapter 8 of this Ph. D. Thesis.

98

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

5.3.1 Smoothing algorithm simulation results
5.3.1.1 Chirp and sinusoidal RFI tests

The use of the FFT implies that the Smoothing Algorithm is implicitly searching for sinusoidal interferences [67-69], therefore, in order to determine the correct performance of this algorithm, it has been tested with a set of linear chirp and sinusoidal interfering signals as defined in eqn. (5.14).
RFIsin [ k ] = ∑ Ap e
p =1 P − k −t p d p2

(

)

2

cos ( 2π f p + πβ pTs k ) Ts k + ϕ p

(

)

 ν r + ηr k− 2 + ∑ Ar cos ( 2π f r Ts k + ϕr )rect  r =1  ηr −ν r  
R

   k= 1,..., N ,   

(5.14)

where P is the number of chirp signals that conform the chirped RFI, Ap, dp, tp,

fp, φp, βp are the amplitude, effective duration, central time, initial frequency, initial
phase, and chirp rate of the pth chirp signal respectively; R is the number of sinusoidal RFI signals, Ar, fr, φr, νr, ηr are the amplitude, frequency, phase, initial time and final time of the rth sinusoidal signal respectively; rect(·) is the rectangular function, Ts is the sampling period, and N is the number of samples of the RFI signal RFIsin. Figures 5.12-5.14 present the retrieved error in TA as a function of the threshold’s Pfa, the size of the smoothing filter, and the power of the interfering signal. To test the performance of the Smoothing Algorithm, a 15 × 15 Hann window is selected. Retrieved TA [K] and error in the retrieved TA as a function of the threshold’s Pfa are represented in Figs. 5.13 and 5.13, respectively. Figure 5.12 shows the retrieved

TA [K], for different interfering signal powers (and even with no interference, black
dotted line), with a fixed smoothing filter size (in this case a 15 × 15 Hann window), as a function of the Pfa of the threshold used in the algorithm. The TA [K] value is obtained by means of 1024 Monte Carlo simulations. The actual TA value is 300 K which is asymptotically achieved by the black dotted line (in abscense of RFI) as Pfa decreases. As it can be seen, the selection of the Pfa threshold is crucial, as a low Pfa values decreases the probability of detection of RFI-contaminated pixels, leading to a retrieved temperature higher than the real antenna temperature. On the other hand, a high Pfa value will produce a high number of false alarms which will produce a clipping in the

99

Chapter 5

probability distribution function of the power spectrogram pixels, leading to a retrieved

TA lower than the real antenna temperature.

Figure 5.12. Retrieved TA using the Smoothing Algorithm with a 15 × 15 Hann window as smoothing filter, TAideal = 300 K (gray line). TA is represented in Kelvin (y-axis), threshold value in Pfa (x-axis). Colored lines represent RFI contaminated radiometric signals with INR value determined by its label. Black dotted line represents a radiometric signal in abscense of RFI.

In Fig. 5.13, the error introduced by the Smoothing Algorithm in the retrieved antenna temperature εTA [K] is represented for different interfering signal powers, with a fixed smoothing filter size: (15 × 15 Hann window). In fact, Fig. 5.13 is the same figure as Fig. 5.12, but instead of plotting the measured TA for different RFI scenarios, plotting the absolute value of the difference between the measured TA (for different RFI scenarios) minus the TAideal (300 K), which in fact corresponds to εTA [K]. It can be observed that RFIs with high INR values are accurately eliminated with a threshold with a low Pfa, while in the case of RFIs with an INR value between −5 dB and −25 dB applying a low threshold, leads to an important error produced by the fact that a great part of the RFI contaminated pixels “pass under” the threshold with low Pfa. The selection of the Pfa threshold is a compromise between a not too high value to clip the power PDF (Pfa > 7.24×10−4 in Fig. 5.13, INR = 0 dB) and not too low to leave a high rate of undetected RFI contaminated pixels (Pfa < 7.24×10−4 in Fig. 5.13, INR = −15 dB).

100

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis
8 7 6 5
INR=-10dB INR=-20dB INR=-5dB INR=-15dB INR=5dB INR=0dB

ε T [K]
Max εTA

A

4 3 2 1 0 -5 10

( )
RFI=0dB

εTA

INR=-25dB INR=-30dB
-4 -3

INR=-∞dB

10

10

10

-2

Pfa

Pfa

Opt

Figure 5.13. Error obtained by the Smoothing Algorithm in the retrieved TA using a 15 × 15 Hann window as smoothing filter. Error is represented in [K] (y-axis), threshold value in Pfa (x-axis). Colored lines represent RFI contaminated radiometric signals with INR value determined by its label. Black dotted line represents a radiometric signal in abscense of RFI.

101

Chapter 5
6
7

35 × 35
5
INR=-10dB INR=-15dB INR=-20dB

6 5
INR=0dB
INR=-10dB INR=-20dB

25 × 25
INR=-15dB

4

ε T [K]

A

A

3
INR=-5dB

ε T [K]

4 3 2 1

INR=0dB

INR=-5dB INR=5dB INR=-25dB INR=-30dB

2 1

INR=-25dB

INR=5dB

INR=-30dB

0 -5 10

INR=-∞dB

10

-4

10

-3

10

-2

0 -5 10

INR=-∞dB

10

-4

10

-3

10

-2

Pfa
10 9 8 7
INR=-15dB INR=-10dB

Pfa

a)
12

b)
INR=-25dB

5×5

1×1
10
INR=5dB INR=0dB INR=-10dB

INR=-30dB

8

ε T [K]

ε T [K]

6 5 4 3 2 1 0 -5 10
INR=5dB INR=-∞dB INR=0dB INR=-20dB INR=-5dB

INR=-5dB INR=-15dB

A

A

6
INR=-20dB

4
INR=-30dB

2 0 -4 10

INR=-25dB

INR=-∞dB

10

-4

10

-3

10

-3

10

-2

Pfa

Pfa

c)

d)

Figure 5.14. Error obtained by the Smoothing Algorithm in the retrieved TA using a Hann window as smoothing filter. a) 35 × 35 Hann window; b) 25 × 25 Hann window; c) 5 × 5 Hann window; d) without using Smoothing Algorithm.

Combining different simulations with RFIs of different powers (INR from 5 dB to −30 dB compared to the noise power) it is observed that the threshold with optimum performance has a Pfa ~ 7.24 ×10−4 (Pfa|Opt in Fig. 5.13) with a maximum RMS error value of 2.33 K (Max( εTA ) in Fig. 5.13), and RMS error value without RFI of 1.84 K (in Fig. 5.13) which it will be the error obtained in case of RFI free situation; the threshold value associated to Pfa|Opt is equal to 1.37σ2n.

Three additional Monte Carlo sets of simulations have been performed with a size of the Hann window smoothing filter of 35 × 35, 25 × 25, and 5 × 5. Similar results have been obtained (Fig. 5.14a-c respectively) which are summarized in Table 5.1. In this table, the values of the threshold with the lower TA RMS error independently of the RFI power are presented, in addition to the Pfa associated with this threshold and the TA 102

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

RMS error obtained with this threshold in the absence of RFI. For all cases, it is important to have a low TA RMS error value for the most suitable threshold in the black dotted curve, as absence of RFI should be the most probable case. In Table 5.1 it is observed that, as the filter size used in the Smoothing Algorithm increases, the maximum TA RMS error value decreases; in addition this value also decreases in absence of RFI.

Table 5.1. Maximum retrieved TA error [K] for the best threshold Pfa for four different cases of Smoothing Algorithm filtering, including the case of not applying the algorithm. Maximum retrieved TA error [K] in absence of RFI for the most suitable threshold is also shown.

Filter window 35 × 35 Hann 25 × 25 Hann 15 × 15 Hann 5 × 5 Hann
Pfa Threshold value Max εTA Max εTA for RFI = 0 dB 3.52 × 10−3 1.24·σ2n 2.05 K 1.16 K 2.09 × 10−3 1.37·σ2n 2.09 K 1.41 K 7.24 × 10−4 1.72·σ2n 2.33 K 1.84 K 7.06 × 10−5 4.04·σ2n 3.71 K 3.71 K

Without Smoothing 2.35 × 10−3 6.08·σ2n 5.89 K
5.89 K

The best performance is obtained with the largest window (35 × 35 Hann window). However, large smoothing windows exhibit a poorer radiometric resolution, as explained below. The radiometric resolution of an ideal total power radiometer is inversely proportional to the square root of the product of the noise bandwidth and the integrating time.

∆T =

Trec + TA , Bτ

(5.15)

where ∆T is the radiometric resolution, Trec is the receivers temperature, B is the noise bandwidth and τ is the integration time. Spectrogram pixels have a resolution of a determined bandwidth ∆B and a determined integration time ∆τ, and the sum over all pixels of ∆B·∆τ corresponds exactly to the product of B and τ. Pixel elimination produced by the Smoothing Algorithm decreases the number of pixels available for radiometric measurements and the radiometric resolution degrades. In eqns. (16) and (17), the relationship between the radiometric resolution after and before applying the Smoothing Algorithm is developed.

103

Chapter 5

Bτ SA ∆T SA ∆B∆τ N = = = Trec + TA ∆T ∆B∆τ N SA Bτ
∆T =

(

Trec + TA

)

N = N SA

N , N − N el

(5.16)

SA

N ∆T , N − N el

(5.17)

where ∆T|SA is the radiometric resolution after applying the Smoothing Algorithm, ∆B and ∆τ are the frequency and time resolutions of the spectrogram pixels,

N is the number of pixels of the spectrogram, N|SA is the number of pixels of the
spectrogram that passes the Smoothing Algorithm filtering process, and Nel is the number of eliminated pixels by the Smoothing Algorithm.

Figure 5.15 shows the degradation of the radiometric resolution due to loss of radiometric data. It is observed that the degradation increases with the window size. When the window size increases, RFI power peaks are convolved over the spectrogram leading to an increase of the number of RFI-contaminated pixels. Degradation also increases with the RFI power as high powers are more likely to be detected by the algorithm even when they are smoothed, and contaminate a larger area of the timefrequency image. Therefore, as the filter size increases, the number of eliminated pixels increases, degrading the radiometric resolution.
1.3 1.25 1.2

∆T SA ∆T

1.15 1.1 1.05 1
INR=0dB INR=5dB

INR=-5dB

INR=-10dB

INR=-15dB

INR=-20dB INR=-∞dB

5

10

15

20

25

30

35

40

45

N

Figure 5.15. Radiometric sensitivity degradation due to pixel elimination as a function of the filter size. Eight RFI signals have been used, with total INR value labeled for each case. Thresholds used are the most suitable ones for each window size (Table 5.2). Radiometric sensitivity is normalized by the radiometric sensitivity value obtained when the Smoothing Algorithm is not applied; i.e., without the elimination of any pixel of the time-frequency image.

104

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

A Hann window arround 15×15 is therefore recommended, as for higher values, retrieved TA error [K] shows little improvement, while radiometric resolution may decrease excessively leading to a worsening of the retrieved geophysical parameters. Figures 5.16 and 5.17 show as the number of eliminated pixels increases with the enlargement of the window size when there are chirp and sinusoidal RFI signals present in the radiometric signal.
0.5 -115 -120 -125 0.3 0.2 -130 -135 -140 0.1 -145 0 0 200 400 600 800 1000 -150

Normalized Frequency

0.4

Number of FFT segment

Figure 5.16. Spectrogram of a simulated chirp and sinusoidal RFI contaminated radiometric signal. RFI power: σ2RFI = σ2n − 5 dB;

105

Chapter 5
0.5 0.5

Normalized Frequency

0.4 0.3

Normalized Frequency

-115 -120 -125

-120 0.4 0.3 -125 -130 -135 0.2 -140 -145 0.1 0 -150 -155 0 200 400 600 800 1000

0.2

-130 -135 -140 0 200 400 600 800 1000

0.1 0

Number of FFT segment

Number of FFT segment

a)
0.5
0.5

b)
-120

Normalized Frequency

0.4 0.3

Normalized Frequency

-120

0.4 0.3

-125 -130 -135

-125 0.2 -130

0.2

-140 -145

0.1 0

0.1

-150 -155 0 200 400 600 800 1000

0

200

400

600

800

1000

0

Number of FFT segment

Number of FFT segment

c)
0.5 -118 0.5

d)
-120

Normalized Frequency

0.4

Normalized Frequency

-120 -122

0.4

-125 -130 -135

0.3

-124 -126 -128

0.3

0.2

0.2

-140 -145

0.1

-130 -132

0.1

-150 -155 0 200 400 600 800 1000

0

0

200

400

600

800

1000

0

Number of FFT segment

Number of FFT segment

e)
0.5 -120 0.5 -122 -124 0.3 -126 -128 -130 0.1 -132 -134 0 200 400 600 800 1000

f)
-115 -120 0.4 -125 -130 -135 0.2 -140 -145 0.1 -150 -155 0 200 400 600 800 1000

Normalized Frequency

0.4

Normalized Frequency

0.3

0.2

0

0

Number of FFT segment

Number of FFT segment

g)

h)

Figure 5.17. Smoothing Algorithm applied to the simulated chirp and sinusoidal RFI contaminated radiometric signal shown in Fig. 5.15. Convolution of the spectrogram with: a) 5×5 Hann window; c) 15×15 Hann window; e) 25×25 Hann window; g) 35×35 Hann window; RFI-mitigated spectrogram using the most suitable threshold for this INR and a smoothing of a: b) 5×5 Hann window; d) 15×15 Hann window; f) 25×25 Hann window; h) 35×35 Hann window.

106

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

5.3.1.2 Broadband PRN and OFDM RFI tests

As broadband communication systems are increasing exponentially, it is likely to find broadband RFI added to the measured signal. Therefore, the Smoothing Algorithm has been also tested with PRN and OFDM RFI signals defined in eqns. (5.18) and (5.20).
 T  RFI PRN [ k ] = APRN ⋅ PRN  s k  cos ( 2π f PRN − f Rad Ts k + ϕPRN ) k= 1,..., N ,  Ts   PRN 

(5.18)

where APRN, TsPRN , fPRN, φPRN, are the amplitude, sampling frequency of the PRN signal, frequency, phase of the OFDM signal respectively, fRad is the frequency where the radiometer is measuring and PRN[k] is an ASK modulated signal composed by the repetition of a the sequence of the firsts 10,230 output bits of a MLSG of 14 stages (5.19).

MLSGPRN = X 14 + X 8 + X 7 + X 4 + X 3 + X 2 + 1 ,
RFI OFDM [ k ] = AOFDM ⋅
M C −1 m =0

(5.19) (5.20)

∑S

m

cos ( 2π f m k ) k = 1,..., N ,

where AOFDM, is the amplitude of the OFDM signal, Mc is the number of independent carriers (i.e., data streams), Sm is the Mc parallel modulated source symbols, and fm is the frequency of the mth subcarrier of the OFDM signal, defined as:

fm =

m TsOFDM

,

m = 0,..., M C − 1 ,

(5.21)

where TsOFDM is the OFDM symbol duration. The retrieved error in TA as a function of the threshold’s Pfa, the size of the smoothing filter and the power of the interfering signal has been calculated in the same way that in the previous section, and it is presented in Figs. 5.18-5.20.

107

Chapter 5
20 18 16 14 INR=-15dB INR=-10dB INR=0dB INR=-5dB
10

15 × 15

INR=5dB

9 8 INR=-15dB 7 INR=-5dB

15 × 15

ε T [K]

ε T [K]

12 10 8 6 4 2 0 -3 10 INR=-20dB INR=-30dB

6 5 4 3 INR=-20dB INR=-10dB

INR=-∞dB

A

INR=-25dB
2 1 0 -3 10

A

INR=5dB INR=-25dB INR=0dB INR=-30dB 10
-2

INR=-∞dB

10

-2

10

-1

10

-1

Pfa

Pfa

a)

b)

Figure 5.18. Error obtained by the Smoothing Algorithm in the retrieved TA using a 15×15 Hann window as smoothing filter. Error is represented in Kelvin (y-axis), threshold value in Pfa (x-axis). Colored lines represent RFI contaminated radiometric signals with INR value determined by its label. Black dotted line represents a radiometric signal in absence of RFI. RFI signal is: a) PRN interfering signal, and b) OFDM interfering signal.
40 35 30 25 INR=0dB
25

35 × 35
INR=5dB
20 INR=-10dB

25 × 25
INR=-5dB

INR=5dB INR=0dB

ε T [K]

ε T [K]

15 INR=-15dB INR=-∞dB INR=-30dB INR=-25dB

A

15 10 5 0 INR=-20dB INR=-15dB

INR=-30dB

INR=-∞dB

A

20

INR=-10dB

INR=-5dB

10

5

INR=-20dB

INR=-25dB 10
-2

10

-1

0 -3 10

10

-2

10

-1

Pfa
40 35 30 25

Pfa
80
INR=5dB

a)
5×5
INR=-5dB INR=-∞dB INR=-10dB

b)
1×1
INR=5dB INR=0dB

70 60
INR=0dB

50

ε T [K]

ε T [K]

INR=-10dB 40 30 20 10

INR=-5dB INR=-∞dB INR=-30dB

A

15 10 5

INR=-15dB INR=-20dB

INR=-30dB

A

20

INR=-15dB

INR=-25dB 0 -3 10 10
-2

INR=-25dB
10
-1

INR=-20dB
-2

0 -3 10

10

10

-1

Pfa

Pfa

c)

d)

Figure 5.19. Error obtained by the Smoothing Algorithm in the retrieved TA when the RFI is a broadband PRN like signal, using a Hann window as smoothing filter. a) 35 × 35 Hann window; b) 25 × 25 Hann window; c) 5 × 5 Hann window; d) without using the Smoothing Algorithm.

108

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis
6
10

35 × 35
5 INR=-15dB 4

9 8 7 INR=-5dB INR=-15dB

25 × 25

INR=-20dB

INR=-∞dB

ε T [K]

ε T [K]

6 5 4 3 INR=-25dB INR=-30dB INR=5dB INR=-20dB

INR=-5dB

A

INR=-5dB INR=0dB INR=-30dB

A

3

INR=5dB

INR=0dB

2

1 INR=-25dB 0 -3 10 10
-2

2 1

INR=-10dB 10
-1

0 -4 10

INR=-∞dB 10
-3

10

-2

10

-1

Pfa
30

Pfa

a)
60

b)
5×5 1×1
50 INR=-5dB INR=5dB 40 INR=-10dB

25 INR=-10dB 20 INR=-15dB

INR=0dB

ε T [K]

INR=-∞dB 15 INR=5dB INR=-5dB

ε T [K]

INR=-∞dB

A

A

30 INR=-15dB 20 INR=-30dB INR=-20dB

10

INR=-25dB

5 INR=-30dB INR=-20dB 10
-2

10 INR=-25dB INR=0dB
-1

0 -3 10

10

-1

0 -3 10

10

-2

10

Pfa

Pfa

c)

d)

Figure 5.20. Error obtained by the Smoothing Algorithm in the retrieved TA when the RFI is a broadband OFDM signal, using a Hann window as smoothing filter. a) 35 × 35 Hann window; b) 25 × 25 Hann window; c) 5 × 5 Hann window; d) without using the Smoothing Algorithm.

In Fig. 5.18, the error introduced by the Smoothing Algorithm in the retrieved antenna temperature εTA [K] is represented for different interfering signal powers, with a fixed smoothing filter size: (15×15 Hann window) for two different broadband interfering signals. It can be observed in Fig. 5.18a that a very high Pfa must be used to detect a PRN interfering signal. In this case, the compromise of the Pfa threshold selection leads to a minimum error of retrieved antenna temperature of 14.39 K, (Pfa = 0.0384) for an INR value of 5 dB.

On the other hand, Fig. 5.18b shows that an OFDM broadband signal can be detected with a threshold with a lower Pfa, thus leading to a minimum error of retrieved antenna temperature of 9.07 K, (Pfa = 0.02).

109

Chapter 5

Following the RFI study in the same way as in the previous section, three additional Monte-Carlo simulations have been performed for different Hann window smoothing filter sizes (35×35, 25×25, and 5×5) for both broadband RFI cases; which are presented in Figs. 5.19-5.20. Results of Fig. 5.18 show that a Pfa|Opt does not exist for all INR parameter values as in Fig. 5.13 due to the fact that an increase of the INR parameter leads to an increase of the error in the retrieved TA; therefore higher INR parameter values leads to higher error in the retrieved TA values. In contrast, for the OFDM RFI case, a Pfa|Opt exists for all INR parameter values as in the sinusoidal RFI case (Fig. 5.20). Table 5.2 summarizes the optimal Pfa and threshold values and the maximum error in the retrieved TA for the OFDM RFI case; this table is similar to Table 5.1, but with lower thresholds values and higher error in the retrieved TA values.
Table 5.2. Maximum retrieved TA error for the best threshold Pfa for four different cases of Smoothing Algorithm filtering, including the case of not applying the algorithm; for the OFDM interfering signal case. Maximum retrieved TA error in absence of RFI for the most suitable threshold is also shown.

Filter Window 35 × 35 Hann 25 × 25 Hann 15 × 15 Hann 5 × 5 Hann

Pfa 2.16 × 10−2 Threshold value 1.18·σ2n Max εTA 3.8 K Max εTA 3.51 K for RFI = 0 dB

2.4 × 10−2 1.24·σ2n 5.88 K 5.57 K

2 × 10−2 1.42·σ2n 9.07 K 9.01 K

1.76 × 10−2 2.21·σ2n 21.97 K 21.97 K

Without Smoothing 3.02 × 10−2 3.5·σ2n 44.5 K

44.5 K

Detection and elimination of a PRN signal results in a high error on the retrieved TA as applying an FFT to a PRN RFI signal does not concentrate the RFI signal, as it happens with a sinusoidal signal. PRN signal behaves like noise.

In contrast, error in the retrieved TA produced by the contamination of an OFDM interfering signal is lower than in the PRN signal’s case, as this signal is based in a frequency modulation, and the FFT process can concentrate the energy of the interfering signal.

The main problem of the presence of broadband RFI is that, even if it is correctly detected, radiometric resolution of the measurements will be degraded due to

110

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

the minimum error introduced in TA and the loss of radiometric resolution due to the high number of eliminated pixels, as it can be seen in Figs. 5.21-5.24. Errors in the retrieved antenna temperatures obtained in the Figs. 5.21-5.24 are summarized in Tables 5.3 and 5.4.
Table 5.3. Retrieved TA error for the best threshold Pfa for the four different cases of Smoothing Algorithm filtering, shown in Fig. 5.19 with a INR value of −5 dB (equivalent to 126.5 K of interference); for the PRN interfering signal case.

Filter Window Pfa Threshold value εTA

35 × 35 Hann 25 × 25 Hann 15 × 15 Hann 5 × 5 Hann 3.68 × 10−2 2.79 × 10−2 2.1 × 10−2 1.7 × 10−2 1.15·σ2n 1.23·σ2n 1.42·σ2n 2.22·σ2n 5.15 K 6.35 K 9.12 K 21.7 K

Table 5.4. Retrieved TA error for the threshold value used in Fig. 5.20 for the four different cases of Smoothing Algorithm filtering, with a INR value of −5 dB (equivalent to 126.5 K of interference); for the OFDM interfering signal case.

Filter Window Pfa Threshold value used εTA

35 × 35 Hann 25 × 25 Hann 15 × 15 Hann 5 × 5 Hann 2.16 × 10−2 2.4 × 10−2 2 × 10−2 1.76 × 10−2

1.18·σ2n 2.7 K

1.24·σ2n 4.48 K

1.42·σ2n 1.45 K

2.21·σ2n 22.3 K

Threshold used in Table 5.3 and Figs. 5.21-5.22 is the most suitable threshold to detect PRN RFI signals with an INR parameter value lower or equal to −5 dB. On the other hand, threshold used in Table 5.4 and Figs. 5.23-5.24 is the most suitable threshold to detect OFDM RFI signals with any INR parameter value.
0.5 -115 -120 -125 0.3 -130 -135 -140 0.1 -145 0 0 200 400 600 800 1000 -150

Normalized Frequency

0.4

0.2

Number of FFT segment

Figure 5.21. Spectrogram of a simulated PRN RFI contaminated radiometric signal. RFI power: σ2RFI = σ2n − 5 dB;

111

Chapter 5
0.5

-120

0.5

-120 -125 -130

Normalized Frequency

0.4

Normalized Frequency

-125

0.4 0.3

0.3 0.2

-130

-135 -140 -145

-135

0.2 0.1

0.1

-140

-150 -155 0 200 400 600 800 1000

0

0

200

400

600

800

1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5

a)
-124

b)
-120 -125 -130

Normalized Frequency

0.4

Normalized Frequency

-126 -128 -130 -132

0.4 0.3 0.2 0.1 0

0.3 0.2

-135 -140 -145 -150 -155 0 200 400 600 800 1000

0.1

-134 0 200 400 600 800 1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5

c)
Normalized Frequency
-126 -128 -130 -132 -134 0 200 400 600 800 1000

d)
-120 -125 -130

Normalized Frequency

0.4 0.3 0.2 0.1 0

0.4 0.3

-135 -140 -145

0.2 0.1

-150 -155 0 200 400 600 800 1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5

e)
Normalized Frequency
-126

f)
-120 -125 -130

Normalized Frequency

0.4 0.3 0.2 0.1 0

0.4 0.3 0.2 0.1 0

-128

-135 -140 -145 -150 -155 0 200 400 600 800 1000

-130

-132

-134 0 200 400 600 800 1000

Number of FFT segment

Number of FFT segment

g)

h)

Figure 5.22. Smoothing Algorithm applied to the simulated PRN RFI contaminated radiometric signal shown in Fig. 5.21. Convolution of the spectrogram with: a) 5×5 Hann window; c) 15×15 Hann window; e) 25×25 Hann window; g) 35×35 Hann window; RFI-mitigated spectrogram using the most suitable threshold for this INR (-5 dB) and a smoothing of a: b) 5×5 Hann window; d) 15×15 Hann window; f) 25×25 Hann window; h) 35×35 Hann window.

112

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis
0.5 -115 -120 -125 0.3 0.2 -130 -135 -140 0.1 -145 0 0 200 400 600 800 1000 -150

Normalized Frequency

0.4

Number of FFT segment

Figure 5.23. Spectrogram of a simulated OFDM RFI contaminated radiometric signal. RFI power: σ2RFI = σ2n − 5 dB

113

Chapter 5
0.5 -120
0.5 -120 -125 -130 0.3 0.2 -135 -140 -145 0.1 -150 -155 0 200 400 600 800 1000

Normalized Frequency

Normalized Frequency

0.4 0.3

-125 -130 -135 -140

0.4

0.2 0.1 0

0

200

400

600

800

1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5

a)
-124 -126 -128

b)
-120 -125 -130

Normalized Frequency

0.4

Normalized Frequency

0.4

0.3 -130 0.2 -132 -134 -136 0 0 200 400 600 800 1000

0.3 0.2

-135 -140 -145

0.1

0.1

-150 -155 0 200 400 600 800 1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5

c)
Normalized Frequency
-126

d)
-120 -125 -130

Normalized Frequency

0.4 -128 0.3 -130 0.2 -132 0.1 -134 0 0 200 400 600 800 1000

0.4

0.3

-135 -140 -145

0.2 0.1

-150 -155 0 200 400 600 800 1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5

e)
-126

f)
-120 -125 -130

Normalized Frequency

0.4 -128 0.3 -130 0.2 -132 0.1 -134 0 0 200 400 600 800 1000

Normalized Frequency

0.4 0.3 0.2

-135 -140 -145

0.1 0

-150 -155 0 200 400 600 800 1000

Number of FFT segment

Number of FFT segment

g)

h)

Figure 5.24. Smoothing Algorithm applied to the simulated OFDM RFI contaminated radiometric signal shown in Fig. 5.23. Convolution of the spectrogram with: a) 5×5 Hann window; c) 15×15 Hann window; e) 25×25 Hann window; g) 35×35 Hann window; RFI-mitigated spectrogram using the most suitable threshold for a smoothing of a (Table 5.3); b) 5×5 Hann window; d) 15×15 Hann window; f) 25×25 Hann window; h) 35×35 Hann window.

114

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

5.3.2 Wiener filter algorithm simulation results
Results obtained with the LLMMSE filter show that this algorithm is suitable for denoising signals (Fig. 5.25), but it has an important drawback.

a)

b)

c)

Figure 5.25. Locally adaptive linear minimum mean square error (LLMMSE) Algorithm applied to an RFI contaminated radiometric signal. a) RFI contaminated signal’s spectrogram, σ2RFI = σ2n − 5 dB; b) Estimation of the RFI present in the radiometric signal by denoising; c) Cleared radiometric data by RFI substraction.

For an optimal performance, it is necessary to accurately estimate in advance the power of the thermal noise (TA). Error in the estimation of the thermal noise power, introduces an error in the denoising process which leads to an error in the retrieved TA itself as it can be seen in Fig. 5.26. Thus, thermal noise power must be first estimated for a proper extraction of the RFI from the radiometric signal. In addition, it is observed that the error introduced by the LLMMSE algorithm is almost equal to the error of the estimated noise power itself.

The reason to perform a RFI extraction is to accurately estimate TA, which in fact is the thermal noise power. However, the LLMMSE algorithm does not improve the accuracy in the estimation of the thermal noise as it can be seen in Fig. 5.26, and therefore this algorithm is not considered suitable for RFI mitigation.

115

Chapter 5
60 10

INR=10dB
50 0

INR=5dB INR=0dB INR=-5dB

1

INR [dB] eTA – eTA | [K]

0.8

RFI = 0 dB

40

-10

eTA [K]

0.6

INR=-10dB

30

-20

0.4 INR=-20dB 0.2 INR=-25dB

INR=-15dB

20

-30

10

-40

0
0 -50

INR=-∞dB 0 10 20

INR=-30dB 30 40 50

0

10

20

30

40

50

60

es^ n [K]

es^ n [K]

a)

b)

Figure 5.26. Error in the estimation of the retrieved temperature using the LLMMSE denoising algorithmin as a function of the error in the a-priori estimation of the thermal noise power. Both errors are represented in [K]. Coloured lines represent RFI contaminated radiometric signals with an INR determined by its label. Black dotted line represents a radiometric signal in absence of RFI.

5.3.3 FIAT algorithm simulation results
In order to observe the performance of the FIAT algorithm compared to the Smoothing algorithm, the same RFI contaminated signals that have been tested with the Smoothing algorithm have been tested with the FIAT algorithm.

5.3.3.1 Chirp and sinusoidal RFI tests

In the first place, the sinusoidal and chirp RFI contaminated signals have been tested with the FIAT algorithm, emphasizing the most important advantage of the Smoothing algorithm which is the capacity of detecting and eliminating clusters of pixels instead of entire frequency or time segments. In Fig. 5.27, the comparison between the performances of the FIAT and the Smoothing algorithm is presented, showing the inability of the FIAT algorithm to correctly detect and eliminate chirp signals. In fact, if different thresholds are used in the FIAT algorithm, as it is represented in Fig. 5.27 it can be observed that a high Pfa threshold value (Pfa = 10-1 in Fig. 5.27c) eliminates a very high amount of RFI free pixels, and a low Pfa threshold value (Pfa = 10-3 in Fig. 5.27d) fails to detect properly the RFI signal. Hence, if chirp signals are present in the RFI, the FIAT algorithm is not recommended.

116

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis
0.5 -115

0.5 -120

Normalized Frequency

Normalized Frequency

0.4

-120 -125

0.4 0.3

-125 -130 -135

0.3

-130 -135 -140

0.2

0.2 0.1 0

-140 -145 -150 -155 0 200 400 600 800 1000

0.1 -145 0 0 200 400 600 800 1000 -150

Number of FFT segment
0.5

Number of FFT segment
-115 -120 0.5

a)

b)

-115 -120

0.4

Normalized Frequency

Normalized Frequency

-125 -130 -135

0.4

-125 -130 -135

0.3 0.2 0.1

0.3 0.2 0.1

-140 -145 -150 -155 200 400 600 800 1000

-140 -145 -150 -155 200 400 600 800 1000

0

0

Number of FFT segment

Number of FFT segment

c)

d)

Figure 5.27. Comparison between Smoothing and FIAT Algorithm performances when applied to a simulated chirp and sinusoidal RFI contaminated radiometric signal. a) RFI contaminated signal’s spectrogram, σ2RFI = σ2n − 5 dB; b) RFI-mitigated spectrogram with Smoothing algorithm using the most suitable threshold for this INR and a smoothing of a 15 ×15 Hann window; c) RFI-mitigated spectrogram with FIAT algorithm using a threshold with a Pfa = 10-1; d) RFI-mitigated spectrogram with FIAT algorithm using a threshold with a Pfa = 10-3;

On the other hand, if only sinusoidal RFI components are present in the radiometric signal, the FIAT algorithm will have a good behaviour, as it can be in the Fig. 5.28a, where the error introduced by the FIAT Algorithm in the retrieved antenna temperature εTA [K] is represented for different interfering signal powers is represented, in the same way as in Fig. 5.13. The right part of the figure (Fig. 5.28b) is included for comparison purposes, as it exist a noticeable difference in the behaviour of the Smoothing and the FIAT algorithms. In Table 5.5, the comparison between errors in the retrieved antenna temperatures, for the Smoothing (15 × 15 and 35 × 35 Hann windows) and the FIAT algorithms, with only sinusoidal interfering signals is presented.

117

Chapter 5

Table 5.5. Maximum retrieved TA error [K] for the best threshold Pfa for 2 different cases of Smoothing Algorithm filtering, and one case of FIAT algorithm filtering. Maximum retrieved TA error [K] in absence of RFI for the most suitable threshold is also shown.

RFI detection Smoothing with Smoothing with algorithm 35 × 35 Hann 15 × 15 Hann −3 Pfa 2.76 × 10 1.33 × 10−3 Threshold value 1.27·σ2n 1.67·σ2n Max εTA 1.07 K 1.27 K Max εTA 0.66 K 0.98 K for RFI = 0 dB
3

FIAT

1.5 × 10−3 0.16·σ2n 0.27 K 0.27 K

15 × 15
0.5 INR=-25dB 0.4 INR=0dB INR=5dB 0.3 INR=-10dB INR=-∞dB 2 2.5 INR=-20dB INR=-15dB INR=-10dB

ε T [K]

ε T [K]

A

0.2 INR=-5dB 0.1 INR=-20dB 0 -4 10 INR=-15dB INR=-30dB 10
-3

A

1.5

INR=-25dB

INR=5dB

1 INR=-5dB 0.5 INR=-∞dB 10
-3

INR=-30dB

INR=0dB 10
-2

10

-2

0 -4 10

Pfa

Pfa

a)

b)

Figure 5.28. Error obtained in the retrieved TA by a) the FIAT algorithm, and b) the Smoothing Algorithm using a 15×15 Hann window as smoothing filter. Error is represented in Kelvin (y-axis), threshold value in Pfa (x-axis). Colored lines represent RFI contaminated radiometric signals with INR value determined by its label. Black dotted line represents a radiometric signal in absence of RFI. RFI signal is composed only by frequency constant sinusoidal interfering signals in both cases.

5.3.3.2 Broadband PRN and OFDM RFI tests

Figure 5.29 presents the retrieved error in TA as a function of the threshold’s Pfa, and the power of the interfering signal, for a PRN interfering signal (Fig. 5.29a), and an OFDM interfering signal (Fig. 5.29b), after the application of the FIAT algorithm.

It can be observed that as it happens with the Smoothing algorithm, in order to detect PRN interfering signals it is needed a very high Pfa, associated with a very high threshold value (in fact, a interfering signal with a INR higher than 0 dB does not appear in the Fig. 5.29a as the error obtained in the RFI elimination was too high). As it has been stated in Section 5.3.1.2, PRN signal behaves like noise, so a FFT-based algorithm will not be the best way to detect this RFI signal.

118

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

12 INR=-5dB 10 INR=-10dB INR=0dB

5 4.5 4 3.5 INR=-20dB INR=-∞dB INR=-10dB INR=-15dB

8

ε T [K]

A

A

6 INR=-20dB

INR=-∞dB

ε T [K]

INR=-15dB

3 2.5 2 1.5 INR=5dB INR=0dB INR=-5dB

4

INR=-30dB

2

INR=-25dB

1 0.5 INR=-30dB INR=-25dB 10
-2

0 -3 10

10

-2

10

-1

0 -3 10

10

-1

Pfa

Pfa

a)

b)

Figure 5.29. Error obtained by the FIAT Algorithm in the retrieved TA. Error is represented in Kelvin (yaxis), threshold value in Pfa (x-axis). Colored lines represent RFI contaminated radiometric signals with INR value determined by its label. Black dotted line represents a radiometric signal in absence of RFI. RFI signal is: a) PRN interfering signal, and b) OFDM interfering signal.

On the other hand, and in the same way as in the Smoothing algorithm case, a Pfa|Opt exists for all INR parameter values for the OFDM RFI case, Pfa ~ 5.79 × 10−2, with a maximum RMS error value of 3.29 K. The threshold value associated to this Pfa is directly related to the erf function and the number of time and frequency segments (due to the central limit theorem as explained in Section 5.2.4). Table 5.6 compares these results with the results obtained with the Smoothing algorithm, showing that, as the OFDM interfering signal is composed merely by sinusoidal components, slightly better results are obtained using the FIAT algorithm (the same way than in the Section 5.3.3.1 with only sinusoidal components).
Table 5.6. Retrieved TA error for the threshold value used in Fig. 5.30 for the two different cases of Smoothing Algorithm filtering and the FIAT algorithm filtering, with a INR value of −5 dB (equivalent to 126.5 K of interference); for the OFDM interfering signal case

RFI detection Smoothing with Smoothing with FIAT algorithm 35 × 35 Hann 15 × 15 Hann 7.24 × 10−4 Pfa 5.79 × 10−2 2.16 × 10−2 2 2 1.72·σ n Threshold value ~0.1·σ2n 1.18·σ n εTA 2.7 K 2.33 K 2.25 K

119

Chapter 5
0.5 -115
0.5 -120

Normalized Frequency

Normalized Frequency

0.4

-120 -125

0.4 0.3 0.2 0.1 0

-125 -130 -135 -140 -145 -150 -155 0 200 400 600 800 1000

0.3 0.2

-130 -135 -140

0.1 -145 0 0 200 400 600 800 1000 -150

Number of FFT segment
0.5

Number of FFT segment
0.5
-120

a)
Normalized Frequency
-125 -130

b)

-115 -120

Normalized Frequency

0.4 0.3 0.2 0.1 0

0.4 0.3

-125 -130 -135

-135 -140 -145 -150 -155 0 200 400 600 800 1000

0.2 0.1

-140 -145 -150 -155 200 400 600 800 1000

0

Number of FFT segment

Number of FFT segment

c)

d)

Figure 5.30. Comparison between Smoothing and FIAT algorithm performances when applied to a simulated PRN RFI contaminated radiometric signal. a) RFI contaminated signal’s spectrogram, σ2RFI = σ2n − 5 dB; b) RFI-mitigated spectrogram with Smoothing algorithm with a 15×15 Hann window; c) RFI-mitigated spectrogram with Smoothing algorithm with a 35×35 Hann window; d) RFI-mitigated spectrogram with FIAT algorithm. The most suitable threshold for this INR (-5 dB) has been used in all the cases.
0.5 -115
0.5

-120 -125 -130

Normalized Frequency

Normalized Frequency

0.4

-120 -125

0.4 0.3

0.3

-130 -135 -140

-135 -140 -145

0.2

0.2 0.1

0.1 -145 0 0 200 400 600 800 1000 -150

-150 -155 0 200 400 600 800 1000

0

Number of FFT segment
0.5

Number of FFT segment
0.5
-120 -125 -130

a)
Normalized Frequency

b)

-115 -120

Normalized Frequency

0.4 0.3 0.2

0.4

-125 -130 -135

-135 -140 -145

0.3

0.2

-140 -145

0.1 0

-150 -155 0 200 400 600 800 1000

0.1 0

-150 -155 200 400 600 800 1000

Number of FFT segment

Number of FFT segment

c)

d)

Figure 5.31. Comparison between Smoothing and FIAT Algorithm performances when applied to a simulated OFDM RFI contaminated radiometric signal. a) RFI contaminated signal’s spectrogram, σ2RFI = σ2n − 5 dB; b) RFI-mitigated spectrogram with Smoothing algorithm with a 15×15 Hann window; c) RFI-mitigated spectrogram with Smoothing algorithm with a 35×35 Hann window; d) RFI-mitigated spectrogram with FIAT algorithm. The most suitable threshold for all INR has been used in all the cases.

120

RFI Detection and Mitigation Algorithms Based on Spectrogram Analysis

For comparison purposes, Figs. 5.30 and 5.31 include an RFI contaminated spectrogram (by a PRN interfering signal for Fig 5.30a and by an OFDM interfering signal for Fig 5.31a), the result of the Smoothing algorithm application with a 15 × 15 Hann window (Figs. 5.30b and 5.31b), a 35 × 35 Hann window (Figs. 5.30c and 5.31c), and the result of the FIAT algorithm application (Figs. 5.30d and 5.31d), using the most suitable threshold for any INR value in both figures.

It can be observed in Figs. 5.30 and 5.31 that the elimination of the whole frequency subbands is useful when the entire frequency subband is contaminated. Hence, the FIAT algorithm performs similarly to the Smoothing algorithm with a 35 × 35 Hann window, but computationally more efficient.

5.4 Conclusions
Four new RFI detection and mitigation algorithms have been presented. All of them are based on processing of the radiometric signal's spectrogram, and thus operate in the time and frequency domains simultaneously.

The Smoothing Algorithm is studied and its performance is estimated using Monte Carlo simulations. The threshold value in the Smoothing Algorithm is the most critical parameter, as it minimizes the retrieved TA error depending on the filter size and the RFI power, which is a priori unknown. For a determined filter size, the best threshold can be calculated varying the RFI signal power and keeping the noise power constant. In case that the interference is sinusoidal, it is found that there is an optimal threshold value which minimizes the retrieved TA error for any RFI power. This threshold value diminishes with the filter size used in the Smoothing Algorithm. For a simulation of a sinusoidal RFI, with a threshold value of 1.37·σ2n, the retrieved TA error is 2 K for a filter size of 25×25 pixels, and any INR value.

In case that the RFI is broadband, two cases have been studied, which are a PRN RFI, and an OFDM RFI. The PRN RFI behaves like noise and it is found that there is not an optimal threshold value, as increasing the RFI power always increases the

121

Chapter 5

retrieved TA error. The OFDM RFI behaves like a sinusoidal RFI, so there also exists an optimal threshold value, although the maximum retrieved error in the TA is higher than in the sinusoidal case. In addition, broadband RFI’s contaminate extense areas of the spectrogram, resulting in a poorer radiometric resolution as many more pixels of the spectrogram have been eliminated.

A 2D denoising filter based on the optimum Wiener filter (LLMMSE) is also studied to estimate the RFI in the signal’s spectrogram and to substract it from the contaminated one. However, it has been found that the Wiener filter has an acceptable performance applied to the spectrogram to detect RFI signals for subsequent substraction only if the noise power of the received signal is precisely known, which is actually the magnitude to be determined. Therefore although the Wiener filter is optimal for signal denoising in signal processing, the accuracy required in the estimation of the noise power is much higher in microwave radiometry than in typical communications applications, and therefore it is suitable for RFI mitigation. In addition, if the noise power is known with enough accuracy before applying this algorithm, it would not be necessary indeed.

A simpler algorithm called FIAT algorithm has been developed as a complement of the Smoothing algorithm; besides, it can be used isolated obtaining very good results in the elimination of the RFI present in the signal. However, this algorithm has two main drawbacks, it does not work properly if the interference varies its frequency in time (chirped RFI), and eliminates a high number of pixels.

These three algorithms have been tested only with sinusoidal, chirp, PRN and OFDM like signals, testing other types of RFI signals with these algorithms will be performed in the future as well as processing measured data.

Another algorithm has been presented, Edge Detection Algorithm. However, results of this algorithm with real data will be discussed in Chapter 8, in addition of results with real data of the Smoothing and the FIAT Algorithm.

122

RFI Mitigation in Microwave Radiometry using Wavelets

Chapter 6
6. RFI Mitigation in Microwave Radiometry using Wavelets

Current mitigation techniques are mostly based on blanking in the time and/or frequency domains of the periods of time and/or sub-bands where RFI has been detected. However, in some geographical areas, RFI is so persistent in time that prevents from acquiring any useful radiometric data. In this chapter a wavelet-based technique is proposed to mitigate RFI. The interfering signal is estimated by using the powerful de-noising capabilities of the wavelet transform. The estimated RFI signal is then subtracted from the received signal and a “cleaned” noise signal is obtained, from which the power is estimated later. The algorithm performance is presented as a function of the threshold type, and the threshold selection method, the decomposition level, the wavelet type and the interference-to-noise ratio.

123

Chapter 6

Most of the previous studies are focused in pulsed sinusoidal signals within the radiometer’s band. In radio-astronomy some techniques have been proposed to deal with GLONASS RFI [23], but since the RFI signal is much lower than the noise, some a priori knowledge of the interferent signal must be known. These are called “physical modeling” in communications’ terminology and a different model is required for each type of RFI.

Another type of analyses belong to the so-called “statistical-physical modeling” category, that provide universal model for natural and man-made RFI. The main models are the Middleton’s class A (narrowband RFI within receiver’s band), class B (broadband RFI wider than receiver’s band), and class C (mixture of class A and B) canonical models for which the mathematical form is independent of the physical environment [73]. The RFI mitigation approach is then based on the estimation of the model parameters, and apply a linear optimal filtering (Wiener filter) or optimal detection rules [74].

In this Ph. D. thesis a different approach to mitigate the effect of RFI in microwave radiometry is proposed. It is based on the use of the power of the wavelet transform to denoise (remove noise from a signal) so as to estimate the interfering

ˆ signal s(t ) (RFI) without any “a priori” knowledge of it. This signal is then subtracted

ˆ from the received signal x(t), to obtain a quasi RFI-free noise signal n(t ) from which
the power is detected (Fig. 6.1).

ˆ( Figure 6.1. RFI mitigation technique: an estimate of the RFI signal s t ) is subtracted from the received

ˆ( signal x(t) = s(t) + n(t), so as to obtain a quasi RFI-free noise signal n t ) .

In the following sections the principles of denoising are briefly reviewed. The optimum parameters to detect and mitigate four different types of RFI (sinusoidal, Doppler-like, chirp, and pseudo-random noise) are studied: type of thresholding,

124

RFI Mitigation in Microwave Radiometry using Wavelets

sequence length, decomposition level, and type of wavelet (among 75 different types [75, 76]). In order to make the study completely general and to make an homogeneous inter-comparison between these different signals:
•

The maximum instantaneous frequency has been set to be equal to 1 for all of

them, so that the sequence length corresponds to the number of samples per signal period, and
•

The amplitude has been properly scaled so that the interfering signal power is

also the same, and so the INR ratio.

The noise power is assumed to be equal to one, and the algorithm’s performance is expressed in terms of the error of the detected output power as a function of the INR.

6.1 Principles of Denoising
Consider the problem of denoising an unknown signal [77-80]:

x (t ) = s (t ) + n (t ) ,

(6.1)

from a set of samples xi = si + ni (i = 1,…,N) corrupted by a zero mean AGWN
ni. The Discrete Wavelet Transform (DWT) can be used for denoising a noisy signal. If W denotes a N by N orthonormal wavelet transformation matrix, the previous equation

can be expressed in the wavelet domain as:
X =S+N,

(6.2)

where X = W ⋅ x S = W ⋅ s and N = W ⋅ n are the wavelet transforms of x, s and n ˆ ˆ ˆ

[79, 80]. For a smooth function with AGWN, a theoretical threshold exists that completely removes the noise and successfully reproduces the original function [79, 81]:

ˆ s = W −1 ⋅ H ⋅W ⋅ x ,
where H is a filter characterized by eqn. 6.4

(6.3)

125

Chapter 6

H = diag[h(1), h(2),..., h( N )] .

(6.4)

Thresholding is one method for filtering. If a limited number of the highest coefficients of the DWT spectrum is retained (h(i)=0 for i=m...N), and an inverse transform W-1 with the same wavelet basis is applied, a denoised signal is obtained. There are a number of ways to decide which coefficients should be retained. The two simplest methods are the hard and soft thresholding. The hard threshold filter Hhard removes coefficients below a threshold value T, determined by the noise variance [77]:
Y = X ⋅( X > T ) .

(6.5)

The soft threshold filter Hsoft shrinks the wavelet coefficients above and below the threshold T, reducing the coefficients towards zero [79]:
Y = sign( X ) ⋅ ( X − T ) . (6.6)

If the resulting signal has to be smooth, it has been shown that the soft threshold filter must be used [80]. However, the hard threshold filter performs better.

The selection of the threshold value can also be difficult. In practice, if the noise-free signal s(t) is unknown, a smooth approximation of the signal is looked for. Small threshold values lead to noisy results, while large threshold values introduce bias. Experimental studies have demonstrated that for some applications, the optimal threshold is simply computed as a constant c times the noise variance [80]. Four approaches are used in this study: •

One approach utilizes a selection rule based on Stein's Unbiased Risk Estimate

or SURE (quadratic loss function). If the signal to noise ratio is very small, the SURE estimate is very noisy. • times

The Universal method assigns a threshold level equal to the noise variance
2 log ( N ) , where N is the sample size [78].

•

The heuristic approach is a mixture of the two previous ones, and if the signal to

noise ratio is detected to be very small, the fixed form threshold is used.

126

RFI Mitigation in Microwave Radiometry using Wavelets

•

The fourth method uses a fixed threshold selected to yield the minimax

performance for mean square error against an ideal procedure (minimum of the maximum mean square error).

It is known that the choice of the proper wavelet scaling function is always the most important thing. Generally, for the denoising, the wavelet scaling function must have mathematical properties (shape, continuity of the signal and its derivatives) similar to the original signal. For example, to denoise pulsed signals, the Haar wavelet (box scaling function) will perform well, but it will not perform as well to denoise sinusoidal signals. On the other hand, high order Daubechies wavelets (e.g. order 8) will perform well for sinusoidal signals. In the limit, if the level of decomposition increases any input signal could be reproduced with enough accuracy, so a trade-off exists between the decomposition level and the complexity to evaluate the wavelet transform. The number of different wavelets is very large. See [82] for a quite detailed list of wavelet types and their properties. The effect of the wavelet type (Haar wavelet) and the threshold selection method can be visually seen in Fig. 6.2 for the four different types of signals been analyzed in this work. In this case, the pseudo-random noise (PRN) signal (bottom panel) is much better reconstructed than the other three smooth and continuous signals.
10 SINE 0 -10 0 10 DOPPLER 0 -10 0 10 CHIRP 0 -10 0 10 PRN 0 -10 0

1

2

3

4

5

6

7

8

9

10

1

2

3

4

5

6

7

8

9

10

1

2

3

4

5

6

7

8

9

10

1

2

3

4

5
time

6

7

8

9

10

Figure 6.2. Original noise-free signal (black), original noisy signal (crosses), and reconstructed signals using the Haar wavelet transform, level = 12, 32 samples per period (total 320) and threshold: SURE (red), universal (green), heuristic SURE (blue) and minimax (cyan). INR = σ2interference/ σ2noise =10.

127

Chapter 6

6.2 Optimum Parameters Selection
In this research 75 different wavelet types have been tested [82], but simulation results (average of 100 realizations each) are presented first only for the simplest wavelet: Haar wavelet or Daubechies 1. Once the trends are understood and the optimum parameters are found for the Haar wavelet, the optimum performance for the each signal type and each wavelet type is presented.

6.2.1 Threshold selection and Sequence length
Figure 6.3 shows the performance of the proposed RFI mitigation technique using wavelets using the Haar wavelet, as a function of the threshold methods described in Section 6.1 and the sequence length, for the four different types of interferent signals (sine, Doppler, chirp, and PRN), decomposition level = 12, and INR = 100.

As it can be appreciated, as the sequence length increases, the estimation noise power error also decreases by the same factor, saturating around 3·10-4 for the PRN signal at sequence lengths longer than 213.

In general, the fixed threshold estimation provides the worst performances, except for the SURE thresholding and the chirp signal for lengths between 64 and 1024.

In general, for all signal types, the best performance is achieved for any sequence length using the heuristic SURE thresholding. Only for the PRN type of signal and below 256 samples, the minimax thresholding methods outperforms the heuristic SURE method.

128

RFI Mitigation in Microwave Radiometry using Wavelets
RFI mitigation performance - Haar transform
squares: fixed threshold with single level noise estimation circles: soft sure thresholding diamonds: soft heuristics sure thresholding 2 triangles: minimax threshold with multiple level estimation of σ noise solid line: sine RFI dashed line: Doppler RFI dotted line: Chirp RFI dash-dot line: PRN RFI

10 Interference-to-Noise Ratio = σ2 /σ2 = 100; σ2 =1 interference noise noise

3

10

2

Worst performance

10

1

Noise power error

10

0

10

-1

10

-2

10

-3

Best performance
-4

10

10

0

10

1

10 10 Sequence length (21...216)

2

3

10

4

10

5

Figure 6.3. RFI mitigation performance using the Haar transform as a function of the threshold method and sequence length for four different types of interferent signals, and decomposition level = 12. Threshold method: squares = fixed threshold with single level noise estimation, circles = soft SURE, diamonds = soft heuristics SURE, and triangles = minimax. RFI signal: solid line = sine, dashed line = Doppler, dotted = chirp, and dash-dot = pseudo-random noise.

6.2.2 Decomposition level
Figure 6.4 shows the performance of the proposed RFI mitigation technique using wavelets using the Haar wavelet, with 216 sequence length, as a function of the decomposition level. As intuitively expected, the more the decomposition levels, the better the reconstruction (slope ~-1/3 decade per unit level) and the smoother the function, the smaller the number of terms in the decomposition that have to be used to reconstruct the signal properly. The quality of the reconstruction of the sinusoidal signal (and later cancellation) saturates above 5 levels, the Doppler signal saturates above 6 levels, the chirp signal saturates above 7 levels, and only the PRN signals starts saturating at 11 levels. In all cases the heuristic SURE thresholding method is used, which is the one that achieves the best performance, only matched by the fixed thresholding for PRN signal.

129

Chapter 6
RFI mitigation performance - Haar transform
squares: fixed threshold with single level noise estimation diamonds: soft heuristics sure thresholding solid line: sine RFI dashed line: Doppler RFI dotted line: Chirp RFI dash-dot line: PRN RFI

10 Interference-to-Noise Ratio = σ2 /σ2 = 100; σ2 =1 interference noise noise

0

10

-1

Noise power error

10

-2

10

-3

10

-4

0

2

4

6 Decomposition level

8

10

12

Figure 6.4. RFI mitigation performance using the Haar transform as a function of the threshold method and decomposition level for four different types of interferent signals, and sequence length 216. Threshold method: squares = fixed threshold with single level noise estimation, circles = soft SURE, diamonds = soft heuristics SURE, and triangles = minimax. RFI signal: solid line = sine, dashed line = Doppler, dotted = chirp, and dash-dot = pseudo-random noise.

6.2.3 RFI mitigation performance vs. interference-tonoise ratio
Figures 6.5 and 6.6 show the RFI mitigation performance using the Haar transform or the optimum wavelet transform respectively, as a function of the threshold method and INR, for a sequence length of 216 and a decomposition level equal to 12. As it can be appreciated, using the soft heuristic SURE thresholding the rejection is very good (~40 dB) for high INR = 100, decreasing with decreasing INR. Below INR 2·10-4 the algorithm is no longer able to estimate the RFI signal (Fig. 6.5).

The optimum wavelets for each signal type have found to be the wavelt symlet 3 for the sinusoidal RFI, the reverse biorthogonal wavelets 1.5 for the Doppler signal, the discrete approximation of Meyer wavelet for the chirp signal, and the reverse biorthogonal wavelets 1.3 for the PRN signals. When using the optimum wavelet types for each RFI signal the performance is significantly improved for high INRs (Fig. 6.6), but remains stable as INR decreases so below INR~2·10-4, as in the case of the Haar wavelet, no improvement is seen.

130

RFI Mitigation in Microwave Radiometry using Wavelets

10

2

RFI mitigation performance - Haar transform
squares: fixed threshold with single level noise estimation circles: soft sure thresholding diamonds: soft heuristics sure thresholding

10

1

Noise power error; σ2 =1 noise

10

0

INTERFERENCE REJECTION Worst performance

10

-1

10

-2

10

-3

10

-4

10

-5

Best performance

10 -4 10

-6

10

-3

10 10 10 Interference-to-Noise Ratio = σ2 /σ2 ; σ2 =1 interference noise noise

-2

-1

0

10

1

10

2

Figure 6.5. RFI mitigation performance using the Haar transform as a function of the threshold method and INR for four different types of interferent signals, sequence length 216 and decomposition level = 12. Threshold method: squares = fixed threshold with single level noise estimation, circles = soft SURE, diamonds = soft heuristics SURE, and triangles = minimax. RFI signal: solid line = sine, dashed line = Doppler, dotted = chirp, and dash-dot = pseudo-random noise.
RFI mitigation performance - Optimum wavelet for each signal

10

2

10

1

diamonds: soft heuristics sure thresholding solid line: sine RFI + sym3 wavelet dashed line: Doppler RFI + rbio1.5 wavelet dotted line: Chirp RFI + dmey wavelet dash-dot line: PRN RFI + rbio1.3 wavelet

10 =1 Noise power error; σ2

0

noise

10

-1

10

-2

INTERFERENCE REJECTION

10

-3

10

-4

10

-5

10 -4 10

-6

10

-3

10 10 102 10 Interference-to-Noise Ratio = σ2 /σ ; σ2 =1 interference noise noise

-2

-1

0

1

10

2

Figure 6.6. RFI mitigation performance using the optimum wavelet type for each signal as a function of the INR for the heuristic SURE threshold method for four different types of interferent signals, sequence length 216 and decomposition level = 12. RFI signal: solid line = sine, dashed line = Doppler, dotted = chirp, and dash-dot = pseudo-random noise.

131

Chapter 6

6.3 Conclusions
In general, the soft heuristic SURE thresholding method is the one that best performs for any type of signal, except for very weak RFI (INR ~ 10-3 … 10-4) in which the fixed thresholding slightly outperforms.

Increasing the number of decomposition levels above 6-7 does not improve the quality of the RFI mitigation, except for the PRN (pulsed) signals.

The minimum RFI cancellation is ~40 dB for high INRs (INR~100) when using the Haar wavelet transform, but this value may increase up to ~60-70 dB if the optimum wavelet type for each signal is selected. This will allow make useful radiometric measurements in areas heavily corrupted by RFI.

RFI cancellation algorithm can be applied in scenarios where the error induced by RFI is equal or larger than the measurement uncertainty (∆T). For example, if the antenna temperature is 100 K, the receivers noise temperature is 300 K, and the measurement uncertainty is ∆T = 1 K, the minimum detectable power is 1/(100+300) = 2.5·10-3, which defines the minimum useful INR for which it makes sense to apply the RFI mitigation algorithm. In other applications, such as sea surface salinity retrieval, where the required ∆T is much smaller (∆T = 0.05 K), RFI mitigation can be useful down to INR of ~0.05/(100+300) ≈ 10-4, which is at the limit of the performance of the RFI cancellation algorithm, regardless of the wavelet type used.

The algorithm presented can be applied either at intermediate frequency or for the in-phase and quadrature components of the demodulated signal, which reduces by a factor of 2 the signal bandwidth and the computing requirements. If the computing requeriments need to be further reduced, subbanding can be applied and the proposed algorithms apply to each of the subbands.

132

MERITXELL

Chapter 7
7. MERITXELL: The Multifrequency Experimental Radiometer with Interference Tracking for Experiments over Land and Littoral
This chapter describes the design and implementation of a multiband radiometer covering the L, S, C, X, K, Ka and W bands, conceived for scientific studies and to do research on the presence of RFI at different bands and techniques to detect and mitigate it. To add flexibility and simplify the design, a spectrum analyzer is used as backend for all the bands. Moreover, the instrument includes a thermographic camera operating in the Thermal InfraRed (IR) range (8 - 14 µm), a multi-spectral camera with four spectral bands: Red, Green, Blue and Near InfraRed (NIR), a visible camera, and a GNSS reflectometer. The purpose of this radiometer is the study of 1) RFI detection and mitigation techniques, and 2) to test data fusion techniques to take advantage of all the simultaneous measurements performed.

133

Chapter 7

Despite their poor spatial resolution, passive microwave sensors have been intensively employed with great success in Earth remote sensing during the last decades due to their accuracy and large swath. As it has been previously explained in Chapter 1, the measured data is the so-called antenna temperature, which is related to the amount of power collected by the antenna. From it, a number of geophysical parameters from the ocean (sea surface salinity, sea surface wind speed, ice coverage…), the land (vegetation, soil moisture…), and the atmosphere (rain rate, temperature profiles, water vapour…) can be obtained.

For a given application, the antenna temperatures must be measured at particular frequency bands and/or polarizations. Furthermore, data obtained by other methods (e. g. radar and optical sensors) can also be included to develop models with improved accuracy. Therefore, in general, the larger the number frequencies and polarizations a radiometer can measure, the more geophysical parameters can be determined, and/or the better the retrieval accuracy that can be obtained. This is the reason why many airborne and spaceborne multi-frequency microwave radiometers are currently flown. Just to cite a few examples, the HUTRAD [83] and the PSR [84] (airborne), or the SSMI/S [85] and the AMSR-E [86] (spaceborne).

7.1 General Overview
This Chapter describes the design and implementation of a multiband dualpolarization TPR radiometer with frequent calibration which covers eight protected bands usually used in Earth remote sensing: L-band (1.400 GHz - 1.427 GHz), S-band (2.69 GHz - 2.70 GHz), C-band (7.14 GHz - 7.23 GHz), X-band (10.6 GHz - 10.7 GHz), K-band (18.6 GHz - 18.8 GHz and 23.6 GHz - 24.0 GHz), Ka-band (36 GHz - 37 GHz), and W-band (86 GHz - 92 GHz); developed in the RSLab of the Universitat Politècnica de Catalunya (UPC) (Fig. 7.1).

To add flexibility and simplify the design, a spectrum analyzer is used as a backend (IF stage, filter and power detector) for all the bands. In order to complement the radiometric measurements, the instrument includes three cameras to obtain data in the infrared and optical range: a thermographic camera operating in the range of 8 - 14 µm,

134

MERITXELL

a multi-spectral camera with four spectral bands: Red (670 nm and ~40 nm BW), Green (540 nm and ~40 nm BW), Blue (460 nm and ~45 nm BW), and Near InfraRed (800 nm and ~60 nm BW) and a camera in the visible part of the spectrum. In addition, a GNSS reflectometer has been included in order to take advantage of the GPS reflectometry remote sensing techniques developed by the PAU team [87].

This radiometer has been implemented to investigate the RFI detection and mitigation techniques. Temporal and spectral RFI mitigation techniques will be easily performed with the aid of the spectrum analyzer, as it can divide the signal spectrum in sub-bands so that the interference can be isolated more easily in a narrow band and it can be easily eliminated. Besides, the cameras that are included in the MERITXELL will be used to test data fusion techniques with the radiometric data.

The radiometer is designed as a TPR radiometer with frequent calibration [88], using a Rohde & Schwarz R&S-FSP40 spectrum analyzer as a back-end. Figure 7.1 represents the radiometer block diagram, where it is shown that every specific frequency band is measured by an independent reception module. The radiometric measurements performed by the all the modules, except the W-band module, are multiplexed according to its polarization in order to have only one D.C. to 40 GHz input per polarization. In the end, only one D.C. to 40 GHz input is connected to the spectrum analyzer, so a switch to select the polarization is needed. Using an harmonic mixer the W-band output is down-converted to IF at a frequency in the range of 16 GHz, and it is then introduced independently through the IF input of the spectrum analyzer since the main input of the Spectrum analyzer only can measure up to 40 GHz.

All the processes performed by the spectrum analyzer are controlled by a PC, which can also post-process (with the RFI detection and mitigation algorithms developed in Chapters 4, 5 and 6) and store all the data obtained by the radiometer. Besides, a temperature measurement and control system is included, and a power supply set is necessary to feed some DC components and the temperature control system, which will need a large amount of power to heat and maintain the temperature of whole instrument.

135

Chapter 7

Figure 7.1: MERITXELL schematic.

MERITXELL’s dimensions are approximately 84x82x176 cm and the total weight is larger than 250 kilograms. These weight and dimensions make the MERITXELL handling and management very complex, so a mobile unit has been designed and manufactured to transport and perform the measurements with both the MERITXELL and PAU-SA [89] instruments.

This mobile unit consists of a NISSAN ATLEON truck with a maximum weight of eight tons which can be observed in Fig. 7.2. This truck has an elevator tower mounted on it for measurement purposes (Fig. 7.3), which is capable to rise up to 8 meters, has an azimuth (φ) and elevation (θ) movements of -180º ≤ φ ≤ 135º and 0º ≤ θ ≤ 150º; and it is compatible with both instruments, but only one at a time meanwhile the other one is parked. The elevator tower has four positions: up or measuring, down or parked, calibration or looking to the internal absorber and change the radiometer. The commands of these movements are sent by an external computer, and finally controlled via a Programmable Logic Controller (PLC) located in the control panel inside the 136

MERITXELL

truck. Moreover, the mobile unit has four stabilization legs manually controlled covering the maximum surface allowing to work with an instrument at eight meters high withstanding winds of up to 100 km/h. Both the elevator tower and the stabilization legs work with an hydraulic unit.

Figure 7.2: NISSAN ATLEON truck

Figure 7.3: Mobile unit’s elevator tower

In order to have the instruments and the tower well protected, an enclosure is included to store and transport the instruments (Fig. 7.4). A microwave absorber area has been placed inside the mobile unit for hot load calibration purposes, one for each instrument (Fig. 7.5). In addition to these main four groups there is a diesel electricity generator set of 10 kVA in order to power the electronic parts of the mobile unit and the radiometer in operation.

137

Chapter 7

Figure 7.4: Mobile unit’s housing mounted over the truck

Figure 7.5: Microwave absorbers used for calibration purposes

7.2 Specific Description
To describe the MERITXELL is useful to differentiate five parts, in which all the components of the radiometer can be classified: •

Radio-frequency part, composed by the RF devices (antennas, amplifiers,

switches) which compose the front-ends of each radiometric frequency band, and the GPS reflectometer.

138

MERITXELL

•

Instrumental part, which includes other instruments such as the cameras, the

Spectrum Analyzer and the personal computer among others. •

Thermal control part, which includes the temperature measurement system by

means of Peltier cells and the control system by means of a PID (Proportional, Integral, and Derivative). •

Electrical part, which includes the power supplies control and the electronic

circuits that feed all the RF circuits. •

Structural part, including all the mechanical pieces which hold and settle all

circuits and components present in the MERITXELL, and the aluminium structure which gives shape and stability to the whole hardware.

7.2.1 Radio-Frequency part
This part is the main part of the MERITXELL as it includes the front-ends of all the radiometric measurement bands before down-converting. Since the downconversion process is performed by the Spectrum analyzer, in this part filters, multipliers, and local oscillators are not included.

The first hardware component of each channel is a dual polarized antenna. Antennas above 10 GHz are horn antennas, with Fresnel lenses at their apertures to achieve a Gaussian beam; these antennas have a corrugated horn in the focus in order to increase the bandwidth and decrease sidelobes and cross-polarization. An OMT in the 10.7 GHz antenna or a polarization grid in the rest of the antennas (18.7 to 89 GHz) separate the vertical and horizontal polarizations. Figure 7.6 shows the 36 GHz horn antenna. It can be observed that the outputs of this antenna are two waveguides, one for the vertical polarization, and the other for the horizontal polarization; the rest of the horn antennas are quite similar.

139

Chapter 7

a)

b)

Figure 7.6: 36 GHz lens antenna. a) front view; b) lateral view

For frequencies below 10 GHz the antennas are 4x4 dual polarized patch arrays. These arrays have been developed following the novel architecture used in the LAURA radiometer, described in [90], after the excellent performance achieved. In fact, the antenna used for the L-band measurements is a replica of the LAURA antenna, while the other two are almost scaled versions of the previous one. Patch arrays are dual-polarization coaxial-fed patches printed in a 0.6 mm fiberglass circuit board as it can be seen in Fig. 7.7. The signals of the 16 patches are combined with different weights depending on the position in the array by means of a RF microstrip power combiner circuit for each frequency band and each polarization, being 6 power combiners in total (Figs. 7.8 and 7.9). The signal output of each power combiner is guided by a SMA cable to the Dicke stage.

a)

b)

c)

Figure 7.7: Patch antennas used for the MERITXELL low frequency bands, a 25 cm rule is included in order to appreciate the size differences between antennas. a) 1.4 GHz antenna; b) 2.7 GHz antenna; c) 7.2 GHz antenna.

140

MERITXELL

a)

b)

c)

Figure 7.8: Power combiners used for the MERITXELL low frequency bands, a 25 cm rule is included in order to appreciate the size differences between antennas. a) 1.4 GHz power combiner, ; b) 2.7 GHz power combiner; c) 7.2 GHz power combiner.

Figure 7.9: 2.7 GHz power combiner housing box. Two combiners are stacked in this box, one for each polarization, hence the paired SMA connectors

The implementation of the frequently calibrated TPR radiometer architecture follows each polarization of each antenna. For frequencies above 10 GHz this architecture is formed by a latching circulator and a matched load; in Fig. 7.10 four of these latching circulators are shown. For frequencies below 10 GHz, they are formed by a switch that commutes between the input from the circulator and a matched load followed by a circulator performing as an isolator. In Fig. 7.11 the different circulators are shown.

141

Chapter 7

Figure 7.10: Latching circulators for the 18.7 to 89 GHz frequency bands.

Figure 7.11: Circulators used for the MERITXELL low frequencies.

Finally the tuned amplifiers complete each receiving channels, one for each polarization. All these tuned amplifiers have a gain of at least 60 dB in the band-pass frequency, with gain flatness ≤ 1.5 dB, noise figure ≤ 2.4 dB at the highest frequency band, and 1 dB compression point ≥ 5 dBm for the worse case. Figure 7.12 shows the complete mounted chain from the switch to the amplifier in two cases, the upper frequency bands except the W-band (10.6 GHz to 36.5 GHz) and the lower frequency bands (1.4 GHz to 7.2 GHz).

142

MERITXELL

a)

b)

Figure 7.12: RF components mounted inside the MERITXELL. a) 7.2 GHz band switch + matched load + circulator + amplifier, the two polarizations; b) 18.7 GHz band latching circulator + matched load + amplifier, horizontal poalrization.

The W-band signal is down-converted to an IF located at 16 GHz with a Rohde & Schwarz harmonic mixer since the spectrum analyzer only work up to 40 GHz. The harmonic mixer has two ports, a W-band waveguide that is connected directly to the 89 GHz amplifier (output port), and a SMA connector and that is connected to the LO/IF external mixer port of the spectrum analyzer. The spectrum analyzer provides the LO frequency through the output port to down-convert the 89 GHz input signal. Figure 7.13 shows the complete mounted chain from the latching circulator to the harmonic mixer.

Figure 7.13: W-band RF components for vertical polarization mounted inside the MERITXELL, composed by the latching circulator (left), matched load (behind the latching circulator), amplifier (behind the heat sink) and harmonic mixer (right).

143

Chapter 7

The output of each amplifier of the frequency bands of 1.4 GHz to 36.5 GHz are multiplexed in two different septuplexors, one for each polarization. Hence, all the signals ranging from D.C. to 40 GHz are multiplexed in only two K-connectors, (one for each polarization). Figure 7.14 shows one of the septuplexors used.

Figure 7.14: Septuplexor used to obtain only one D.C to 40 GHz input per polarization.

The next stage consists of two switches that select the polarization to be sent to the Spectrum analyzer. One of the switches operates from DC to 40 GHz, while the other one operates up to 18 GHz. This way, only two connections arrive to the spectrum analyzer, which comprises eight protected radiometric bands at horizontal and vertical polarizations. Figure 7.15 shows these two switches and Fig. 7.16 shows the integration of the septuplexors and switches in the MERITXELL radiometer.

Figure 7.15: Switches used for the polarization switching: (left) D.C. to 40 GHz switch and (right) 2 GHz to 18 GHz switch for the IF signal coming from the W-band.

144

MERITXELL

Figure 7.16: Septuplexors used for each polarization, stacked, and polarization switches under them.

In addition to all the RF hardware dedicated to the radiometric measurements, a GPS reflectometer is mounted. It consists of an array of 5 LHCP GPS l1 band (1575.42 MHz) ceramic patches, with a power combiner that assigns different weights to every patch depending on its position in the array, similar to the 1.4 GHz, 2.7 GHz and 7.2 GHz radiometers. The output of the power combined is connected to a GPS amplifier whose output is connected to a SIGE GN3S GPS sampler module co-developed by the GNSS-Lab at the University of Colorado and the SiGe company [91]. This sampled data is processed with several algorithms developed by the RSLab group from the UPC to obtain GPS reflectometry data [87]. This device is intended to obtain reflectometry data to be combined with all the radiometric data obtained by the MERITXELL. Figure 7.17 shows the front-end of the GPS reflectometer.

145

Chapter 7

a)

b)

Figure 7.17: Front-end of the GPS reflectometer included in the MERITXELL. a) Front view of the ground plane, the high permittivity of the ceramic patches make them quite small compared to the 1.4 GHz fiberglass patches; the holes in the ground plane are required as the three cameras present in the MERITXELL are housed sharing place with the GPS reflectometer; b) Rear view of the ground plane, showing the power combiner, cables are carefully bended to leave place to the different cameras.

7.2.2 Instrumental part
In this section, all the measurement instruments included in the MERITXELL radiometer are described.

7.2.2.1 Spectrum Analyzer
As explained in the previous sections, this radiometer is designed to have a common power detector stage in order to simplify the overall design. This stage is completely performed by a Rohde & Schwartz (FSP40) Spectrum Analyzer capable to measure up to 40 GHz radio-frequency signals. The advantage of using a Spectrum Analyzer as a detector is that there is only one back-end stage (filter, local oscillator, multiplier and detector) in front of the 8 different back-ends that would be necessary instead. Moreover, it can be programmed in terms of central frequency, span, filter bandwidth, number of frequency points, adding versatility to all the radiometric frequency bands.

Another important reason of this choice is the chance to employ the spectrum analyzer as a pre-processing stage, with ability to divide the signal into arbitrary subbands that can be weighted to equalize or mimic arbitrary frequency responses of

146

MERITXELL

different instruments, time intervals, and calculate histograms of the received signal for each sub-band.

Furthermore, the FPS40 spectrum analyzer includes an ADC module to sample and store the noise input voltage signal in the I/Q modulation, for a determined central frequency, and a determined bandwidth; allowing a later digital post-processing of this sampled radiometric data.

All these abilities can be used to apply the normality test algorithms explained in Chapter 4 and the time-frequency domain RFI detection and mitigation algorithms described in Chapter 5.

This instrument can be programmed and controlled remotely via an Ethernet connection. Figure 7.18 shows the front of the spectrum analyzer with the only two inputs detailed in the previous section. As the spectrum analyzer has not the wake-up LAN function implemented, a device called “fingertron” has been designed to remotely “push” the ON-button. The “fingertron” can be observed at the left lower part of the Fig. 7.18, where the spectrum analyzer ON-button is located.

Figure 7.18: Front view of the FSP40 Spectrum Analyzer, note the two RF inputs, left one corresponding to the down-converted W-band and right one corresponding from DC to 40 GHz.

147

Chapter 7

7.2.2.2 Cameras
The MERITXELL radiometer also includes three cameras in order to obtain additional data to be combined with the radiometric data, giving the chance for data fusion of both radiometric and optic data.

One of the cameras is a Flir InfraRed (IR) camera model A320 with a spectral range of 7.5 – 13 µm. With this camera, the physical temperature of the bodies measured by the multi-band radiometer can be accurately known, with a thermal sensitivity (radiometric resolution) of 70 mK, quantized in 16 bits, obtaining a valuable geophysical parameter of the medium being measured. The Field-of-View (FOV) of this camera is equal to 19ºx25º with 320x240 pixels, giving a spatial resolution of 1.36 mrad. A temporal resolution of 30 frames per second is achieved by this camera in case that a video recording is needed. Data obtained by this camera is sent to a personal computer outside the MERITXELL radiometer via Ethernet.

The second camera is a DuncanTech visible and Near InfraRed (NIR) multispectral camera model MS4100. The sensors of the camera can cover simultaneously three different spectral bands from the four spectral bands that are covered by the camera: Red (~0.62 µm), Green (~0.54 µm), Blue (~0.45 µm) and Near InfraRed (~0.80 µm), with a Signal-to-Noise Ratio (SNR) of 60 dB and 10 quatization bits. The Field-of-View (FOV) is equal to 60º but it can be zoomed in as a 14 mm, f/2.8 lens is included. The spectral images can have a maximum number of pixels of 1920x1080, with 10 frames per second; however, lower spatial resolutions can be configured, increasing the number of frames per second. To manage this camera, a PCI card has been installed in a PC located inside the MERITXELL; by this way, and by a LAN connection, this PC can be remotely controlled, thus the multispectral camera, in a similar way than the IR camera.

The third camera is a Q series IP camera also remotely controlled by an Ethernet connection. This camera adds real-time imagery to all the measured data.

148

MERITXELL

All the three cameras are placed behind the ground-plane of the GPS reflectometer, as it can be seen in Fig. 7.19.

Figure 7.19: Front view of the three cameras present in the MERITXELL radiometer. The camera at the top is the IP camera, the camera at the left is the multispectral camera, and the camera on the bottom is the IR camera.

7.2.2.3 Inclinometer
As this radiometer will be installed in a mast capable to orient it in azimuth and elevation, the antennas can be oriented in any direction. Consequently, there must be a way to measure the elevation of the radiometer, as the emissivity of the bodies strongly depends on the incidence angle of the measuring instrument. To solve this point, a Dual Axis DXL360S inclinometer has been installed inside the MERITXELL, with an accuracy of 0.08º, a resolution of 0.01º and a measuring range of 40º for the two axes (360º if only 1 axis is measured). It also has a short response time, lower than 0.4 seconds.

This inclinometer is controlled by the PC that also controls the multispectral camera.

7.2.2.4 Peripheal Interface Controller (PIC) devices
There are some tasks in the radiometer that are controlled remotely by using PIC microcontrollers, such as the temperature and the switches control.

149

Chapter 7

In order to periodically measure the temperatures of the matched loads and the amplifiers of every radiometric subband and polarization, 32 DS18B20 digital thermometers (one for each of the 16 amplifiers and 16 matched loads) are used. These electronic components are controlled by a C-programmed PIC microcontroller. This way, temperatures of the matched load and the amplifiers are monitored and may be used to calibrate and correct the radiometric measurements.

Switches and latching circulators select the polarization and the radiometric input (antenna or matched load) for the different measurement bands. To control the 6 switches and 10 latching circulators that select the radiometric input, and the 2 switches that select the polarization, another PIC microcontroller is used.

Both PIC’s are of the Microchip 16F877A family and are controlled via Ethernet by a Mikroelektronika circuit board. In Fig. 7.20 a PIC and the control board are shown.

Figure 7.20: Microchip 16F877A PIC and Ethernet controller board.

7.2.2.5 Computer devices
In order to perform a remote control of all the devices previously described, a PC and a router are placed inside the MERITXELL radiometer. The PC is required to house the PCI card that controls the multispectral camera and the inclinometer, and to store the data obtained by the GPS reflectometer.

150

MERITXELL

In addition, a 16 port Netgear switch model JFS516 is included in the radiometer as there are several electronic components that are controlled by an Ethernet connection, such as the spectrum analyzer, two cameras, the PC, and the two PIC’s. This way, only an Ethernet output is needed to control all the MERITXELL radiometer.

7.2.3 Temperature control part
The temperature inside any radiometer must be kept as constant as possible, since variations of the radiometer’s physical temperature will lead to variations in the radiometer’s measured antenna temperature. The origin of these variations is the dependence on the physical temperature inside the radiometer of the gain and noise figure of the amplifier, the circulators’ losses, and other physical parameters.

As a consequence, the radiometer must be thermally insulated, so the walls of the radiometer must be adiabatic, i. e. must prevent any heat interchange between inside and outside the radiometer. Hence, the selected walls of the radiometer are 20 mm dualside metallized foam boards (both for EMI and thermal insulation), and a 5.5 mm thick radome to cover the antennas, showing losses of approximately 0.25 dB at the 85-95 GHz band. The metallic part of the walls is composed by two sheets of 1 mm thickness aluminium.

Microwave radiometers usually work at a temperature higher than the outside so that the maximum antenna temperature will always be lower than the internal temperature, and because it is easier to stabilize at a slightly higher temperature. Hence, the MERITXELL radiometer is designed to work at a constant temperature around 45ºC. In order to maintain constant the temperature inside the radiometer, first a set of heater resistors controlled by a thermostat and disseminated around the structure heats the radiometer’s structure up to 45ºC, when the thermostat turns off the heater resistors. Afterwards, the radiometer’s temperature is maintained constant with the aid of 24 Peltier cells located at the walls of the radiometer, providing ~90W each.

151

Chapter 7

A Peltier cell is a thermoelectric device working on the Peltier phenomenon principle. Peltier cells are commonly constructed of larger amount of semiconductor elements, which are connected in series from the electrical point of view and in parallel from the point of view of the heat transfer. They are mostly used for the active cooling, since the cell allows transferring heat from the cold plate to the warm one. Even more, the direction of this heat transfer can be selected with the direction of the current, allowing the use of these Peltier cells either to cool or heat indistinctly; thus being suitable to compensate the temperature variations.

This way, the Peltier cells must be attached to the external aluminium sheet of the radiometer wall, in order to transfer heat outside or inside the radiometer when required. Peltier cells must be surrounded by aluminium heater sinks so that the heat or cold is transferred to the medium (inside or outside the radiometer) with the aid of a fan. Figure 7.21 shows the fan + heater sink + Peltier cell block mounted on a radiometer’s wall.

Figure 7.21. Thermal control structure composed by a Peltier cell (not seen), 2 heater sinks and 2 fans. MERITXELL radiometer has 24 of these structures. Left heater sink and fan correspond to the outside part, and right heater sink and fan correspond to the inside part, note the Peltier cell feeding cable in the inside part.

The temperature control inside the radiometer is performed by the control loop feedback mechanism, the 2216L Eurotherm Proportional Integral Derivative (PID) temperature controller. The temperature inside the radiometer is measured with a PT100 temperature dependant resistance. With this temperature measurement, the PID is able

152

MERITXELL

to maintain the temperature inside the radiometer at a determined constant value with the control of the Peltier cells.

Peltier cells are feeded with a voltage of 12 V and a current of 7.3 A in order to get a correct performance. Obviously, this amount of current cannot be handled by the PID controller, so several MD03 H-bridge motor drives are used to handle this current from the 24 V switching power supplies needed to feed the thermal control system to the Peltier cells. These circuits can handle up to 50 V and 20 A so it has been decided to use 6 of these circuits, as 6 switching power supplies are used (in order to avoid interconnection between different power supplies). The MD03 device is able to reverse the current sent to the Peltier cells, thus allowing to heat or to cool when needed.

To summarize, first of all, the radiometer is heated by heating resistors up to 45ºC when the thermostat disconnects the heating resistors. Afterwards, the temperature is controlled by the PID, which manages the control of the MD03 motor drivers that behave as a forward and reverse relay between the switching power supplies and the Peltier cells.

7.2.4 Electrical part
Due to its dimensions and the large amount of electronic devices, this radiometer needs a significant amount of electric power. The maximum amount of power that this radiometer requires, assuming all the devices are connected, including the temperature control, is 5.4 kW. Table 7.1 describes more detailed the power consumption of the different components.

153

Chapter 7

Table 7.1: Power consumption of the different components of the MERITXELL radiometer

Components Linear power supplies (amplifiers and switches) Switched power supplies (Peltier cells, fans, cameras and PICs) Heater resistors Spectrum Analyzer Personal Computer Switch Total

Power consumption (W) 505

2885 1000 300 600 10 5400

The radiometer only has an electrical input at 220V 50Hz AC, in order to be possible to connect directly to the electrical grid. Usually, the MERITXELL radiometer will be feeded by a diesel generator located in a trailer pulled by the truck. In this aspect, it has been tried to isolate power supplies from different devices in order to avoid current peaks in the most sensible ones. Therefore, all components can be classified in four different groups, every one separated of the rest with a ResidualCurrent Circuit Breaker: •

RF active devices (amplifiers and switches) need a very stable voltage input, so

linear power supplies have been chosen to feed these components, as ripple present in the switching power supplies could produce gain fluctuations in the amplifiers. •

Electronic instruments (cameras, spectrum analyzer, PC, switch and two PICs)

are delicate electronic devices and must be isolated from high current peaks that can be produced by Peltier cells and heating resistors. Some of them work connected directly to the electrical grid (Spectrum analyzer, PC and switch), and others are connected to switching power supplies as they are not affected by the ripple present in this kind of power supplies. •

Fans and Peltier cells operate with DC, but consume very high current values, so

independent switching power supplies have been used for these components.

154

MERITXELL

•

Heating resistors are feed directly with 220V AC, so no extra power supplies are

needed.

Finally, 6 linear power supplies and 9 switched power supplies are used to feed all the DC components. In Fig. 7.22 several switched power supplies (Fig. 7.22a) and linear power supplies (Fig. 7.22b) mounted on the MERITXELL structure are shown.

a)

b)

Figure 7.22: Power supplies housed in the MERITXELL radiometer. a) Switching power supplies; b) Linear power supplies.

Another complexity that has to deal the MERITXELL electrical power supply system is that some of the amplifiers need negative voltage values. Table 7.2 details the voltage and maximum current of these amplifiers. Consequently, electronic circuits have been developed in order to feed the proper voltage and enough current to every amplifier, and additionally redundant voltage rectifiers are included in these circuits in order to ensure a constant feed to the amplifiers. Figures 7.23, 7.24, and 7.25 show the electronic schematics of the three different circuits used to feed all the different amplifiers.

Table 7.2: Suitable voltage and current for the different amplifiers of the MERITXELL radiometer.

Frequency (GHz) 1.4 2.7 7.2 10.6 18.7 23.8 36.5 89

Voltage (V) 15 15 15 15 15 15/-15 15/-15 8

Current (mA) 200 200 250 275 300 300/-50 275/-50 300

155

Chapter 7

Figure 7.23: Electronic schematic of the circuit feeding the 23.8 GHz and 36.5 GHz amplifiers. Four of these circuits are needed as there are two amplifiers for each frequency band.

Figure 7.24: Electronic schematic of the circuit feeding the 89 GHz amplifiers. Two of these circuits are needed as there are two 89 GHz amplifiers.

Figure 7.25: Electronic schematic of the circuit feeding the 1.4 GHz, 2.7 GHz, 7.2 GHz, 10.6 GHz and 18.7 GHz amplifiers. Ten of these circuits are needed as there are two amplifiers for each frequency band.

Furthermore, electrical circuits that drive all the switches and latching circulators present in the radiometer have been designed. As explained in the instrumental part, the latching circulators and the switches are managed by a PIC, but a feeding circuit is required for these RF devices. As the latching circulators switches its circulation direction with a high current peak, it is recommended that these RF devices are isolated from the other RF devices; in order to handle this situation, an optocoupler is used in

156

MERITXELL

these circuits. Figures 7.26, 7.27, 7.28, and 7.29 show the electronic schematics of the four different circuits used to feed all the different switches and latching circulators.

Figure 7.26. Electronic schematic of the circuit that controls the antenna/load latching circulators. Five of these circuits are needed as each circuit can switch the two latching circulators of each frequency band.

Figure 7.27. Electronic schematic of the circuit that controls the antenna/load switches of the 1.4 GHz, 2.7 GHz and 7.2 GHz bands. Six of these circuits are needed as there are two switches for each frequency band.

157

Chapter 7

Figure 7.28. Electronic schematic of the circuit that controls the 89 GHz band polarization switch.

Figure 7.29. Electronic schematic of the circuit that controls the D.C. to 40 GHz polarization switch.

7.2.5 Structural part
In all radiometric devices of considerably high dimensions, the structure must be designed carefully to ensure the properly operation and long duration of the radiometer. In our case, as the entire radiometer has an approximated dimensions of 180x90x90 cm, and it will be mounted in an elevator mast which can move it up to 8 meters above the ground, the structure must be as solid as possible. This elevator mast, the cabinet where MERITXELL is placed and the MERITXELL aluminium structure itself have been

158

MERITXELL

designed by the GUTMAR company to stand in adverse climatological conditions (strong winds up to 100 km/h, humidity, rain, and temperatures between -10ºC – 50ºC) .

The main structure is composed by 40x40 mm and 80x40 mm aluminium ITEM bars, as this kind of bars allows a modular design. In Fig. 7.30 the MERITXELL structure is shown.

Figure 7.30: Aluminium internal structure of the MERITXELL radiometer with the horn and the patch antennas.

In addition to the main structure, several ITEM bars and aluminium pieces have been added afterwards in order to hold and settle all the different electronic instruments, RF components and all the components forming the MERITXELL radiometer.

159

Chapter 7

The structural part also includes the cover walls of the radiometer, which are 20 mm dual-side metallized foam boards (for electric and thermal insulation) and a 5 mm radome to cover the antennas. In these walls several Peltier cells are distributed in order to control the temperature inside the radiometer. One of these walls is shown in Fig. 7.31.

Figure 7.31: Cover wall of the MERITXELL radiometer.

7.3 Conclusions
A detailed description of the MERITXELL radiometer has been presented in this Chapter. With this hardware radiometric measurements in different frequency bands can be acquired to test different RFI detection algorithms at different frequency bands; results of these algorithms will be presented in Chapter 8.

160

MERITXELL

However, all the capabilities of the MERITXELL radiometer have not fully been exploited in the development of this Thesis. This radiometer jointly with the PAU-SA radiometer will be used for future field experiments.

161

RFI detection algorithms applied to radiometric data

Chapter 8
8. RFI detection algorithms applied to radiometric data

This chapter presents the results obtained by the RFI detection algorithms described in Chapters 4 and 5 applied to the radiometric data obtained with the MERITXELL radiometer described in Chapter 7. Normality tests will be applied to radiometric data measured on the 1.4 GHz, 2.7 GHz, and 10.65 GHz and GPS L1 bands, in addition, spectrogram analysis will be applied to radiometric data measured at 1.4 GHz.

163

Chapter 8

The RFI detection algorithms described and simulated in Chapters 4 and 5 have been tested to determine their performance for RFI detection in microwave radiometry. Normality tests and spectrogram analysis are first tested with RFI-free radiometric noise to ensure that the tables with threshold values used on the simulations also are valid with real radiometric data. After this proof, both RFI detection methods will be tested to determine in which case which method is best, and the best combination of methods described in Chapters 4 and 5 will be looked for.

8.1 Description of the method to retrieve the radiometric measurements
The MERITXELL radiometer described in Chapter 7 has been used to obtain all the radiometric data used in this Chapter.

In Chapter 7 Section 7.2.2.1 the Spectrum Analyzer programmed to obtain the different measurements was described. The Spectrum Analyzer includes an ADC module that can sample the input data. In addition, the measured Spectrum Analyzer data can be stored in the frequency and time domains as data vectors of different time traces can form a matrix equivalent to a spectrogram. Thereby, the radiometric data has been retrieved by these two ways to prove the RFI detection and mitigation algorithms developed in this Ph. D. Thesis.

8.1.1 Using the ADC to retrieve data
Firstly, radiometric measurements have been acquired using the ADC from the Spectrum Analyzer, in order to have voltage I/Q data. This I/Q data is imperative to apply the normality tests based RFI detection algorithms (as this radiometric I/Q data is Gaussian in the absence of RFI), and to apply the Smoothing Algorithm, since the spectrogram used in this algorithm is obtained by means of the application of the STFT to this I/Q voltage data.

164

RFI detection algorithms applied to radiometric data

The maximum number of consecutive I/Q data samples that can be collected by the ADC is 65536 samples, thus the time interval and the bandwidth of the radiometric signal will be quite limited in this case.

As an example, the measurement bandwidth used for the L-band is limited to 2 MHz, using 21 adjacent measurements to cover the protected L-band and a nonprotected adjacent bandwidth around this band. Therefore, the time duration of the radiometric measurement is limited to 32.77 ms. After this period, the measurement must be interrupted. Furthermore, these 65536 samples data units occupy an amount of approximately 1 Megabyte (MB), so long time measurements will need a huge data storage capacity.

Thereby, this method of data acquisition may be appropriate to test and evaluate the RFI methods developed in this thesis, but obviously it is not the optimal one for real time radiometric measurements.

8.1.2 Using the Spectrum Analyzer to acquire data
On the other hand additional radiometric measurements have been taken by storing several temporal traces of a determined span in order to form a time-frequency matrix of power values, each one corresponding to the integrated power along the selected resolution bandwidth during the selected sweep time. This way, the spectrogram is obtained by storing 500 temporal traces as columns each one separated in 501 frequencies; thus, the spectrogram is formed by a 501x500 matrix of independent power measurements.

However, the Spectrum analyzer has an important drawback in the measurement of each time trace as introduces a delay equivalent to 41ms, independently of the sweep time value. Hence, the time sweep should have a value at least longer than 41 ms in order to have enough efficiency, but this time cannot be too long in order to be to detect RFI temporal pulses.

165

Chapter 8

Nevertheless, the radiometric measurements obtained have longer time duration, wider frequency bandwidth, and occupy much less space in disk.

8.2 RFI detection algorithms with I/Q data
First of all, RFI-free radiometric data must be captured by the MERITXELL radiometer in order to prove that in RFI free conditions the data is actually thermal noise without any trace of RFI. This way, the RFI free radiometric data must accomplish the condition that the probability of false alarm (Pfa) must be equal to the probability of detection (Pdet), no matter if they are normality test-based, spectrogram analysis-based, or wavelet-based.

8.2.1 Normality test-based algorithms
For the normality test-based algorithms, the RFI-free measurements have been captured by the MERITXELL setting the circulator of every measured band to the reference load, so the spectrum analyzer measures only the matched load thermal noise, which corresponds to a zero-mean Gaussian signal. Therefore, if an algorithm detects any interferences it will be clearly a false alarm.

It is worth noting that in real systems the isolations of the latching circulators the circulators and the switches are never ideal (isolation is approximately 30 dB), hence the acquired measures may not be totally interference free.

8.2.1.1 Normality tests in RFI free radiometric data
To prove the correct performance of the normality tests, matched load measurements have been sampled. Then, all the normality tests described in Chapter 4 have been applied to this data. So, the detection ratio values (obtained Pdet), which is the ratio of the total number of interference detections generated by a test (both phase, I and quadrature, Q components), and the total number of measurements is calculated.

166

RFI detection algorithms applied to radiometric data

Ideally, without the presence of interferences and approximation errors, the detection ratio is equal to the fixed Pfa. On the contrary, in real cases, the obtained values differ slightly from the ideal one, as the RFI-free noise input signal may have RFI attenuated by the circulator, but still present. In addition, if the frequency response of the radiometer is not completely flat, the Gaussian white noise may get ‘coloured’, affecting its Gaussianity. The radiometer’s part which affects more its frequency response is the R&S Spectrum Analyzer signal processing block presented in Fig. 8.1.

Figure 8.1: Block diagram illustrating the R&S FSP signal processing [92].

As it can be seen in Fig. 8.1, the radiometric signal arriving to the R&S Spectrum Analyzer signal processing block passes through an IF filter first, whose Resolution Bandwidth (RBW) corresponds to a Gaussian filter. Then, data is sampled at 32 MHz, and I/Q demodulated. After the demodulation process, data is decimated to the sampling rate selected by the user, and a decimation filter is applied. The bandwidth of the acquired signal is determined by the decimation filter, not by the RBW.

The RBW filter is Gaussian, with 3 dB attenuation at the selected RBW frequency, while the decimation filter has a more flat frequency response instead. Therefore, is interesting that the RBW be as high as possible (the maximum is 10 MHz) in order to have a less ‘coloured’ white noise, thus less affecting to the normality tests. In Fig. 8.2 the frequency response of the 1.4 GHz band of the MERITXELL is presented for two different values of the RBW (1 MHz in Fig. 8.2a, and 10 MHz in Fig. 8.2b) and only one value of the decimation filter (1.6 MHz). The RBW parameter is

167

Chapter 8

configured to 10 MHz for all the measurements taken with the ADC present in the spectrum analyzer.
0 -2 -4 Attenuation [dB]
Attenuation [dB] 0 -0.5 -1 -1.5 -2 -2.5 -3 -3.5 1399

-6 -8 -10 -12 -14 1399

1399.5

1400 Frequency [MHz]

1400.5

1401

1399.5

1400 Frequency [MHz]

1400.5

1401

a)

b)

Figure 8.2. Frequency response of the 1.4 GHz band MERITXELL radiometer with 2 Mega samples per second ADC’s frequency sampling and: a) 1 MHz RBW; b) 10 MHz RBW

According to the previous results, the best choice to have the flattest frequency response of the noise is to have the maximum value of RBW (10 MHz), and a decimation filter bandwidth value so low that the Gaussian shape of the RBW does not affect the shape of the noise spectrum. To verify the normality tests performance in absence of RFI, a set of 1000 RFI-free measurements has been performed. The Pdet has been calculated with the results of the normality tests, which must be Pdet = Pfa in absence of RFI. The parameters’ values selected to perform these measurements are: • A set of probability of false alarm of: Pfa = [0.001 0.005 0.01 0.05 0.1]. • A set of number of samples of: N = [216 215 214 213 212 211 210]. • Resolution bandwidth of RBW = 10 MHz. • Sampling frequency of fs = 2 MHz for the L-band (1.395 – 1.437 GHz), the Sband (2.685 – 2.707 GHz) and the X-band (10.659 – 10.701 GHz); and sampling frequency of fs = 0.5 MHz for the GPS band (1.57275 – 1.57825 GHz).

168

RFI detection algorithms applied to radiometric data

With these parameters, the white noise behaves well when passed through the normality tests as shown in tables 8.1 to 8.8; which exhibit a good overall performance (measured Pdet very close to specified Pfa). However, there are individual cases with slightly differences between Pdet and Pfa, as the S, K and CM tests at L-band, but having a correct performance in the rest of the bands.

Table 8.1: Normality test performance (Pdet) on L-band RFI free radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,000 0,001 0,005 0,055 0,118

S 0,003 0,007 0,014 0,038 0,085

JB 0,003 0,006 0,009 0,047 0,097

K2 0,003 0,005 0,009 0,042 0,093

L 0,003 0,006 0,009 0,043 0,088

AD 0,003 0,009 0,011 0,032 0,083

CM 0,002 0,009 0,010 0,035 0,079

SW 0,000 0,005 0,010 0,063 0,125

LM CHI2 0,002 0,002 0,005 0,006 0,007 0,010 0,043 0,063 0,096 0,114

Table 8.2: Normality test performance (Pdet) on L-band RFI free radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
216 215 214 213 212 211 210

K 0.118 0.115 0.116 0.117 0.113 0.111 0.095

S 0.085 0.093 0.089 0.088 0.098 0.089 0.074

JB 0.097 0.107 0.098 0.102 0.092 0.090 0.080

K2 0.093 0.103 0.092 0.096 0.083 0.077 0.068

L 0.088 0.096 0.101 0.089 0.096 0.091 0.091

AD 0.083 0.096 0.099 0.093 0.101 0.096 0.099

CM 0.079 0.092 0.097 0.090 0.098 0.092 0.093

SW 0.125 0.119 0.108 0.116 0.086 0.097 0.097

LM CHI2 0.096 0.114 0.098 0.107 0.109 0.103 0.103 0.104 0.114 0.105 0.093 0.111 0.088 0.108

Table 8.3: Normality test performance (Pdet) on S-band RFI free radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,001 0,006 0,015 0,053 0,104

S 0,002 0,005 0,008 0,056 0,103

JB 0,002 0,003 0,01 0,058 0,103

K2 0,002 0,003 0,009 0,053 0,100

L 0,002 0,003 0,011 0,052 0,108

AD 0,000 0,000 0,005 0,045 0,099

CM 0,000 0,001 0,005 0,040 0,103

SW 0,002 0,006 0,011 0,053 0,113

LM CHI2 0,000 0,001 0,003 0,004 0,007 0,008 0,049 0,045 0,091 0,098

169

Chapter 8

Table 8.4: Normality test performance (Pdet) on S-band RFI free radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
2 215 214 213 212 211 210
16

K 0.104 0.088 0.114 0.095 0.091 0.095 0.104

S 0.103 0.102 0.114 0.100 0.091 0.102 0.108

JB 0.103 0.105 0.120 0.103 0.090 0.087 0.110

K2 0.100 0.101 0.117 0.102 0.079 0.075 0.098

L 0.108 0.083 0.100 0.095 0.096 0.082 0.113

AD 0.099 0.088 0.106 0.095 0.109 0.078 0.104

CM 0.103 0.086 0.105 0.110 0.122 0.085 0.112

SW 0.113 0.126 0.108 0.106 0.097 0.088 0.107

LM CHI2 0.091 0.098 0.091 0.096 0.095 0.089 0.094 0.115 0.084 0.104 0.096 0.111 0.099 0.119

Table 8.5: Normality test performance (Pdet) on X-band RFI free radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,001 0,007 0,015 0,050 0,097

S 0,002 0,004 0,006 0,040 0,090

JB 0,002 0,008 0,011 0,051 0,092

K2 0,001 0,006 0,011 0,046 0,092

L 0,001 0,008 0,012 0,046 0,097

AD 0,001 0,005 0,014 0,054 0,102

CM 0,000 0,006 0,013 0,057 0,100

SW 0,001 0,005 0,010 0,050 0,105

LM CHI2 0,001 0,000 0,004 0,007 0,007 0,01 0,047 0,051 0,101 0,107

Table 8.6: Normality test performance (Pdet) on X-band RFI free radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
216 215 214 213 212 211 210

K 0.097 0.110 0.086 0.118 0.099 0.108 0.107

S 0.090 0.096 0.083 0.109 0.119 0.102 0.109

JB 0.092 0.112 0.087 0.097 0.106 0.101 0.097

K2 0.092 0.106 0.080 0.093 0.103 0.089 0.085

L 0.097 0.098 0.112 0.111 0.108 0.100 0.097

AD 0.102 0.113 0.107 0.112 0.109 0.099 0.101

CM 0.100 0.113 0.106 0.114 0.112 0.097 0.105

SW 0.105 0.103 0.098 0.105 0.105 0.103 0.114

LM CHI2 0.101 0.107 0.104 0.117 0.109 0.115 0.109 0.104 0.127 0.111 0.110 0.117 0.106 0.107

Table 8.7: Normality test performance (Pdet) on GPS L1 band RFI free radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 500 kHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,001 0,005 0,009 0,051 0,106

S 0,003 0,005 0,008 0,044 0,095

JB 0,001 0,007 0,011 0,050 0,106

K2 0,001 0,005 0,011 0,048 0,104

L 0,005 0,008 0,011 0,047 0,104

AD 0,003 0,007 0,011 0,049 0,108

CM 0,003 0,008 0,011 0,050 0,099

SW 0,000 0,004 0,015 0,053 0,106

LM CHI2 0,000 0,001 0,002 0,001 0,011 0,006 0,049 0,049 0,112 0,095

170

RFI detection algorithms applied to radiometric data

Table 8.8: Normality test performance (Pdet) on GPS L1 band RFI free radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 500 kHz).

0.1
2 215 214 213 212 211 210
16

K 0.106 0.103 0.114 0.093 0.093 0.098 0.099

S 0.095 0.105 0.114 0.090 0.090 0.085 0.085

JB 0.106 0.105 0.105 0.093 0.093 0.084 0.080

K2 0.104 0.102 0.098 0.090 0.084 0.077 0.073

L 0.104 0.114 0.117 0.091 0.086 0.104 0.116

AD 0.108 0.099 0.114 0.098 0.085 0.096 0.102

CM 0.099 0.103 0.113 0.105 0.087 0.097 0.105

SW 0.106 0.102 0.101 0.102 0.089 0.095 0.094

LM CHI2 0.112 0.095 0.109 0.115 0.098 0.111 0.102 0.103 0.103 0.092 0.088 0.098 0.074 0.105

8.2.1.2 Normality tests in RFI contaminated radiometric data
As stated in the previous chapter, in absence of RFI the Pdet is approximately equal to the Pfa, and as the measured radiometric data are real radiometric measurements, i. e. the RFI contamination level is not known; the best normality test will be the one that detects more RFI contaminated data, thus, the one with the highest RFI detection ratio for the same input RFI contaminated radiometric data. The RFI detection ratio is the ratio of the number of measurements flagged as RFI, divided by the total number of measurements.

Thus, a set of 630 RFI contaminated radiometric data measurements in the Lband and X-band cases, and 330 measurements in the S-band and GPS L1 band cases have been performed. RFI detection ratio as a function of different Pfa has been calculated with the results of these data applied to the normality tests. The parameters’ values selected to perform these measurements are the same used in the Section 8.2.1.1.

Results of RFI contaminated data measurements are represented in Tables 8.9 to 8.16 :

Table 8.9: Normality test performance (RFI detection ratio) on L-band RFI contaminated radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,559 0,608 0,629 0,675 0,722

S 0,003 0,010 0,016 0,071 0,127

JB 0,548 0,586 0,610 0,662 0,692

K2 0,532 0,578 0,603 0,659 0,692

L 0,451 0,489 0,522 0,589 0,632

AD 0,524 0,538 0,560 0,627 0,675

CM 0,511 0,538 0,552 0,614 0,654

SW 0,430 0,481 0,500 0,600 0,646

LM CHI2 0,003 0,186 0,006 0,241 0,017 0,263 0,060 0,349 0,113 0,406

171

Chapter 8

Table 8.10: Normality test performance (RFI detection ratio) on L-band RFI contaminated radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
2 215 214 213 212 211 210
16

K 0.722 0.651 0.579 0.524 0.448 0.383 0.305

S 0.127 0.114 0.127 0.100 0.138 0.122 0.108

JB 0.692 0.635 0.570 0.494 0.449 0.362 0.273

K2 0.692 0.633 0.562 0.481 0.440 0.340 0.233

L 0.632 0.567 0.538 0.406 0.311 0.244 0.213

AD 0.675 0.613 0.575 0.476 0.397 0.286 0.233

CM 0.654 0.590 0.554 0.448 0.363 0.283 0.229

SW 0.646 0.568 0.513 0.454 0.398 0.316 0.267

LM CHI2 0.113 0.406 0.116 0.362 0.127 0.325 0.121 0.292 0.117 0.271 0.117 0.240 0.108 0.208

Table 8.11: Normality test performance (RFI detection ratio) on S-band RFI contaminated radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,636 0,636 0,639 0,667 0,682

S 0,006 0,018 0,024 0,073 0,115

JB 0,636 0,636 0,636 0,664 0,679

K2 0,636 0,636 0,636 0,664 0,676

L 0,633 0,633 0,639 0,645 0,661

AD 0,636 0,636 0,636 0,645 0,658

CM 0,636 0,636 0,636 0,645 0,655

SW 0,636 0,636 0,636 0,661 0,697

LM CHI2 0,006 0,473 0,015 0,527 0,018 0,548 0,052 0,603 0,106 0,636

Table 8.12: Normality test performance (RFI detection ratio) on S-band RFI contaminated radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
216 215 214 213 212 211 210

K 0.682 0.670 0.658 0.661 0.645 0.548 0.442

S 0.115 0.124 0.142 0.118 0.136 0.139 0.148

JB 0.679 0.652 0.670 0.682 0.633 0.524 0.406

K2 0.676 0.652 0.670 0.679 0.624 0.509 0.394

L 0.661 0.658 0.633 0.594 0.439 0.342 0.273

AD 0.658 0.667 0.667 0.639 0.576 0.421 0.336

CM 0.655 0.658 0.667 0.615 0.533 0.391 0.330

SW 0.697 0.676 0.664 0.609 0.579 0.467 0.367

LM CHI2 0.106 0.636 0.106 0.621 0.115 0.564 0.127 0.464 0.121 0.367 0.103 0.318 0.115 0.300

Table 8.13: Normality test performance (RFI detection ratio) on X-band RFI contaminated radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 0,238 0,252 0,262 0,322 0,367

S 0,010 0,014 0,024 0,068 0,117

JB 0,238 0,246 0,254 0,313 0,360

K2 0,237 0,244 0,254 0,306 0,354

L 0,068 0,111 0,135 0,219 0,265

AD 0,167 0,186 0,195 0,238 0,292

CM 0,138 0,159 0,183 0,227 0,276

SW 0,208 0,224 0,235 0,294 0,340

LM CHI2 0,008 0,005 0,011 0,021 0,022 0,029 0,063 0,081 0,121 0,133

172

RFI detection algorithms applied to radiometric data

Table 8.14: Normality test performance (RFI detection ratio) on X-band RFI contaminated radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
2^16 2^15 2^14 2^13 2^12 2^11 2^10

K 0.367 0.343 0.300 0.287 0.248 0.198 0.178

S 0.117 0.117 0.127 0.122 0.162 0.124 0.130

JB 0.360 0.335 0.308 0.279 0.244 0.190 0.179

K2 0.354 0.333 0.306 0.278 0.233 0.176 0.167

L 0.265 0.221 0.187 0.110 0.110 0.117 0.105

AD 0.292 0.267 0.227 0.148 0.113 0.105 0.098

CM 0.276 0.251 0.202 0.137 0.111 0.110 0.103

SW 0.340 0.330 0.308 0.271 0.221 0.179 0.157

LM CHI2 0.121 0.133 0.114 0.127 0.119 0.110 0.124 0.110 0.113 0.103 0.129 0.108 0.143 0.114

Table 8.15: Normality test performance (RFI detection ratio) on GPS band RFI contaminated radiometric data as a function of the Pfa with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

216
0.001 0.005 0.01 0.05 0.1

K 1,000 1,000 1,000 1,000 1,000

S 0,006 0,021 0,027 0,109 0,161

JB 1,000 1,000 1,000 1,000 1,000

K2 1,000 1,000 1,000 1,000 1,000

L 1,000 1,000 1,000 1,000 1,000

AD 1,000 1,000 1,000 1,000 1,000

CM 1,000 1,000 1,000 1,000 1,000

SW 1,000 1,000 1,000 1,000 1,000

LM CHI2 0,000 0,915 0,003 0,945 0,006 0,964 0,067 0,982 0,139 0,988

Table 8.16: Normality test performance (RFI detection ratio) on GPS band RFI contaminated radiometric data as a function of the sample size with a Pfa = 0.1 (RBW = 10 MHz, fs = 2 MHz).

0.1
216 215 214 213 212 211 210

K 1,000 1,000 1,000 0.997 0.970 0.827 0.558

S 0.161 0.158 0.152 0.224 0.200 0.218 0.185

JB 1,000 1,000 1,000 0.997 0.961 0.797 0.518

K2 1,000 1,000 1,000 0.997 0.955 0.773 0.494

L 1,000 1,000 0.985 0.897 0.633 0.379 0.245

AD 1,000 1,000 0.997 0.979 0.812 0.567 0.327

CM 1,000 1,000 0.997 0.961 0.758 0.512 0.291

SW 1,000 1,000 0.994 0.970 0.858 0.733 0.482

LM CHI2 0.139 0.988 0.130 0.976 0.118 0.891 0.152 0.758 0.161 0.497 0.155 0.397 0.142 0.324

By comparing Tables 8.9 to 8.16, it is easy to arrange the different normality tests according to their RFI detection ratio, with a fixed Pfa, sample size, and frequency band. As it can be seen in the previous tables, the RFI detection ratio of every normality test increases with the sample size and the Pfa (if the contribution of false alarms, equal to the Pfa value, is excluded), except for the Skewness and Lin-Mudholkar tests, which do not work properly for RFI detection on the four different bands. On the other hand, the Kurtosis test and the kurtosis-based tests (Jarque-Bera and K-squared tests) are the ones that perform better, but as the Skewness test is not suitable to detect RFI, Kurtosis test work better than the kurtosis-based tests. The ECDF based tests perform similarly, being the two best of this kind the Shapiro-Wilk test for low number of samples (212 or lower) and both the Shapiro-Wilk and the Anderson-Darling tests for a large number of

173

Chapter 8

samples (213 or higher). Finally, the Chi square test performs properly, but with worse performance than the Kurtosis-based tests and the ECDF-based tests.

An important conclusion that can be extracted from the Tables 8.9, 8.11, 8.13, and 8.15 is that when Pfa is increased from 0.01 to 0.05 or from 0.05 to 0.1 the RFI detection ratio increases almost the same value as the Pfa increases, thus, the main part of the RFI is detected with a Pfa of 0.01, and an increase of Pfa just translates into an increase of the false alarms without increasing substantially the detected RFI’s. With this information it can be concluded that a reasonable good Pfa value for the normality tests with these measurements is Pfa = 0.01, in order to minimize the false alarm rate, and keeping a good compromise with the probability of RFI detection. However, if a high probability of RFI detection is mandatory, high Pfa values must be used, as it will be better to have lesser data that RFI contaminated data.

Also, it is observed that as the number of samples increases, the RFI detection ratio also increases, due to the fact that the normality tests become more accurate with longer data sets, as explained in Chapter 4 Section 4.3. However, in case that the RFI is present only in a few samples (an impulsive RFI); a higher number of samples will only lead to an elimination of a higher number of RFI-free samples. Moreover, a false alarm will lead to discard a longer data set.

As it is observed, at L- and S-bands the probability of being affected by RFI is quite high while at X-band, detected RFI the probability of RFI detection is lower, and in the GPS L1 band, RFI is present in the whole measured spectrum.

The problem of RFI detection with normality tests is that if RFI is detected, the whole measurement must be discarded. This situation will leave a lower part of the radiometric data as valid. In order to see the RFI detection more precisely, these measurements can be performed separatedly in different sub-bands.

Tables from 8.17 to 8.20 show the RFI detection on a radiometric measurement for the different normality tests separated in sub-bands, for a Pfa = 0.01, and a number of samples of N = 216, for the L-, S-, X-, and GPS L1 bands. In these tables, the value of 1 represents that the normality test has detected a RFI in any of the in phase or quadrature 174

RFI detection algorithms applied to radiometric data

components, and a 0 represents that the normality test has not detected any RFI. This way, only the contaminated sub-bands are discarded, and the remaining radiometric signal can be used as an RFI-free measurement.

For the different measurement bands, the bandwidth of the sub-bands corresponds to the sampling frequency, according to Nyquist, so, the L-band, S-band and X-band sub-bands have a bandwidth of 2 MHz, and the GPS L1 band sub-bands a bandwidth of 0.5 MHz.

Table 8.17: Normality test RFI detection results in the L-band divided in 2 MHz sub-bands, with sample size of 216 and Pfa = 0.01 (RBW = 10 MHz, fs = 2 MHz).

K S JB K2 L AD CM SW LM CHI2

0 0 0 0 0 1 1 1 1 0 0

2 0 0 0 0 0 0 0 0 0 0

4 1 0 1 1 1 1 1 1 0 1

6 1 0 1 1 1 1 1 0 0 1

8 0 0 0 0 0 0 0 0 0 0

10 0 0 0 0 0 0 0 0 0 0

1400 MHz + 12 14 16 18 0 1 1 1 0 0 0 0 0 1 1 1 0 1 1 1 0 0 1 0 0 1 1 0 0 1 1 0 0 1 1 0 0 0 0 0 0 0 1 0

20 1 0 1 1 0 0 0 0 0 0

22 1 0 1 1 0 0 0 1 0 0

24 1 0 1 1 0 1 1 0 0 0

26 1 0 1 1 1 1 1 1 0 1

28 1 0 1 1 1 1 1 1 0 1

As it can be seen in Table 8.17, Kurtosis and kurtosis-based tests detect RFI in the same sub-bands; ECDF tests also detect practically on the same sub-bands, but slightly different from kurtosis based tests. In fact, an RFI is detected by the four ECDF based tests (low probability of false alarm) in the 1400 MHz sub-band (left column Table 8.17), while it is not detected by the kurtosis-based tests; On the other hand, kurtosis-based tests detect RFI in the 1418, 1420 and 1422 MHz bands while ECDF based tests do not clearly detect RFI in these bands, therefore both normality-tests groups complement themselves. Surprisingly chi-square test has a poor performance as it detects only a part of the cases detected by the other normality tests, and Skewness and Lin-Mudholkar test do not work at all with these parameters (all elements in the row are zero).

The main conclusion that can be obtained from Table 8.17 is that the best chance to detect RFI with the normality tests is by combining the Kurtosis test (the

175

Chapter 8

kurtosis-based test which performs better), with one of the ECDF based tests, e.g. the Anderson-Darling test (however all four perform similarly). This way, the sub-band in which both tests are satisfied is accepted. Therefore, RFI that cannot be detected by the Kurtosis test, will be detected by the other one (e.g. the case of the 1400 MHz subband).

In order to have more results of the normality tests in other measurement bands, S-band radiometric data has been tested with the same parameters as the L-band. Results of this test can be observed in Table 8.18, in which the kurtosis-based tests and the ECDF-based tests have the same performance for this case, this is due to the fact that the RFI has a high power compared to the radiometric signal’s power.
Table 8.18: Normality test RFI detection results in the S-band divided in 2 MHz sub-bands, with sample size of 216 and Pfa = 0.01 (RBW = 10 MHz, fs = 2 MHz).

K S JB K2 L AD CM SW LM CHI2

0 0 0 0 0 0 0 0 0 0 0

2 1 0 1 1 1 1 1 1 0 1

4 1 0 1 1 1 1 1 1 0 1

6 1 0 1 1 1 1 1 1 0 1

2686 MHz + 8 10 12 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1

14 1 0 1 1 1 1 1 1 0 1

16 0 0 0 0 0 0 0 0 0 0

18 0 0 0 0 0 0 0 0 0 0

20 0 0 0 0 0 0 0 0 0 0

The number of samples used to detect RFI in the S-band radiometric data is changed to 213 samples, in order to decrease the Pdet for all the normality tests to a level which only the best normality tests will detect the RFI. Results of this new test can be observed in Table 8.19. These results confirm the results obtained in Table 8.17, where kurtosis-based tests detect RFI with more probability than the rest of the tests, and best ECDF based test is SW. L test has a lower probability of detection for this low probability of false alarm (Pfa = 0.01), as L test tends to be more sensitive near the center of the distribution not in the tails, as AD and CM test are more sensitive in the tails, they perform better than the L test (see Chapter 4, Sections 4.1.5, 4.1.6 and 4.1.8).

176

RFI detection algorithms applied to radiometric data

Table 8.19: Normality test RFI detection results in the S-band divided in 2 MHz sub-bands, with sample size of 213 and Pfa = 0.01 (RBW = 10 MHz, fs = 2 MHz).

K S JB K2 L AD CM SW LM CHI2

0 0 0 0 0 0 0 0 0 0 0

2 1 0 1 1 0 0 0 1 0 0

4 1 0 1 1 0 1 1 1 0 0

6 1 0 1 1 1 1 1 1 0 0

2686 MHz + 8 10 12 1 1 1 0 0 0 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 0 0 0 1 0

14 1 0 1 1 1 1 1 1 0 1

16 0 0 0 0 0 0 0 0 0 0

18 0 0 0 0 0 0 0 0 0 0

20 0 0 0 0 0 0 0 0 0 0

X-band RFI measurements have also acquired. In this case, with the initial values of sample size of 216 and Pfa = 0.01 fewer detected RFI are obtained as compared to the case of sample size of 216 and Pfa = 0.1. Results of the normality tests for both cases are shown in Tables 8.20 and 8.21. For the X-band measurements, RFI is best detected with higher Pfa values because of the reduced power of the RFI present in this band, as it can be observed in the spectrograms of the 2nd, 8th and 14th sub-bands shown in Fig. 8.3. Although it seems that any RFI is present in these three sub-bands, applying the FIAT algorithm (a spectrogram based algorithm explained in Chapter 5 Section 5.2.4) shows that the error in the retrieved brightness temperature is higher than in the RFI-free case. Comparing Figs. 8.3 and 8.4 where spectrograms of three sub-bands from X-band (Fig. 8.3) and L-band (Fig. 8.4) measurements are presented; low RFI power in the X-band case is obvious (RFI in the L-band case is easily “seen” while in the X-band case not); hence, a higher Pfa value is needed to detect RFI with normality tests for the X-band than for the L-band.

177

Chapter 8

Table 8.20: Normality test RFI detection results in the X-band divided in 2 MHz sub-bands, with sample size of 216 and Pfa = 0. 01 (RBW = 10 MHz, fs = 2 MHz). 2nd (16602 MHz), 8th (16614 MHz) and 14th (16626 MHz) sub-bands are high-lighted.

K S JB K2 L AD CM SW LM CHI2

0 1 0 1 1 1 1 1 1 0 1

2 1 0 1 1 0 1 1 1 0 0

4 1 0 1 1 1 1 1 1 0 0

6 1 0 1 1 1 1 1 1 0 0

8 1 0 1 1 0 0 0 1 0 0

10660 MHz + 10 12 14 16 18 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

20 1 0 0 0 0 0 0 0 0 0

22 0 0 0 0 0 0 0 0 0 0

24 0 1 0 0 0 0 0 0 0 0

26 0 0 0 0 0 0 0 0 0 0

28 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Table 8.21: Normality test RFI detection results in the X-band divided in 2 MHz sub-bands, with sample size of 216 and Pfa of 0. 1 (RBW = 10 MHz, fs = 2 MHz). 2nd (16602 MHz), 8th (16614 MHz) and 14th (16626 MHz) sub-bands are high-lighted.

K S JB K2 L AD CM SW LM CHI2

0 1 0 1 1 1 1 1 1 0 1

2 1 0 1 1 1 1 1 1 1 0

4 1 0 1 1 1 1 1 1 0 1

6 1 1 1 1 1 1 1 1 0 0

8 1 0 1 1 1 1 1 1 0 0

10660 MHz + 10 12 14 16 18 1 1 1 1 0 0 0 0 0 0 1 1 1 0 0 1 1 1 0 0 0 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0

20 1 0 1 1 0 1 0 0 0 1

22 0 0 0 0 0 1 0 0 0 0

24 0 1 1 1 0 1 1 0 0 0

26 1 0 1 1 1 1 1 0 0 0

28 30 0 1 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 1 0 1

178

RFI detection algorithms applied to radiometric data

10.661

10.6615 Frequency [GHz]

10.662

10.6625

10.663

0

5

10

15 20 Time [ms]

25

30

a)
10.673 10.685

10.6735 Frequency [GHz] Frequency [GHz] 0 5 10 15 20 Time [ms] 25 30

10.6855

10.674

10.686

10.6745

10.6865

10.675

10.687 0

5

10

15 20 Time [ms]

25

30

b)
nd th

c)

Figure 8.3: Spectrograms of the a) 2 (10.662 GHz), b) 8 (10.674 GHz) and, c) 14th (10.686 GHz) sub-bands of the X-band measurement.

RFI measurements in the GPS L1 band have also taken in order to see if RFI could be detected in case of developing a radiometer in this band [87]. In Tables 8.15 and 8.16 is obvious that RFI exists in the entire measured GPS L1 band, so using normality tests to eliminate RFI present in this bandwidth will be useless, as all the bandwidth would be eliminated.

The fact that the normality tests based RFI detection algorithm eliminate a high amount of data, leads to a recommendation of combining these algorithms with spectrogram based algorithms. Even more, normality tests can fail to detect RFI compared to the spectrogram based algorithms as it can be seen in Fig. 8.4, where spectrograms of three 2 MHz sub-bands of the L-band centered at 1.4 GHz, 1.402 GHz and 1.412 GHz respectively are shown. For these three sub-bands, the Kurtosis based algorithms has failed to detect any RFI, and the ECDF based algorithms only have detected RFI in the sub-band centered at 1.4 GHz (Fig. 8.4a). However, the spectrogram analysis shows that there are RFI tones present in these sub-bands.

179

Chapter 8

1399

1399.5 Frequency [MHz]

1400

1400.5

1401

0

5

10

15 20 Time [s]

25

30

a)
1401 1411

1401.5 Frequency [MHz] 0 5 10 15 20 Time [s] 25 30 Frequency [MHz]

1411.5

1402

1412

1402.5

1412.5

1403

1413

0

5

10

15 20 Time [s]

25

30

b)
st nd

c)
th

Figure 8.4: Spectrograms of the a)1 (1.400 GHz), b)2 (1.402 GHz) and c)7 (1.412 GHz) sub-bands of the L-band measurement.

In order to show the correct performance of the normality tests, if Pfa is increased from Pfa = 0.01 to Pfa = 0.05, the 2nd (1402 MHz) and 7th (1412 MHz) sub-bands appear to be detected at least by the Kurtosis, and by the ECDF based tests (AD test). Results are shown in Table 8.22.

Table 8.22: Normality test RFI detection results in the L-band divided in 2 MHz sub-bands, with sample size of 216 and Pfa = 0.05 (RBW = 10 MHz, fs = 2 MHz). 2nd (1402 MHz) and 7th (1412 MHz) sub-bands are high-lighted.

K S JB K2 L AD CM SW LM CHI2

0 0 0 0 0 1 1 1 1 0 1

2 1 0 1 1 0 0 0 1 0 0

4 1 0 1 1 1 1 1 1 1 1

6 1 0 1 1 1 1 1 1 0 1

8 0 0 0 0 0 0 0 0 0 0

1400 MHz + 10 12 14 16 18 0 0 1 1 1 0 0 0 0 0 0 0 1 1 1 0 0 1 1 1 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 1 1 0

20 1 0 1 1 0 1 1 0 0 0

22 1 0 1 1 1 0 1 1 0 0

24 1 0 1 1 1 1 1 1 0 1

26 1 0 1 1 1 1 1 1 0 1

28 1 0 1 1 1 1 1 1 0 1

180

RFI detection algorithms applied to radiometric data

Consequently, these spectrogram-based algorithms must be studied.

8.2.2 Spectrogram-based algorithms
I/Q sampled data has been used to test the spectrogram-based algorithms, in the same way than the normality test based algorithms. However, at it has been commented on Section 8.1.2, spectrogram-based algorithms can be applied with power data instead of I/Q data.

As it has been performed previously with the normality test based algorithms, the RFI free measurements have been obtained measuring directly to the reference load.

8.2.2.1 Spectrogram-based algorithms with RFI-free radiometric data
In the case of the Spectrogram-based RFI detection, Pfa can also be selected to reduce the false RFI detections thus avoiding a high clipping level in the PDF of the radiometric data. In this case, as Spectrogram-based algorithms treat the spectrogram of the signal as an image, only the contaminated pixels are flagged as RFI and eliminated, thus the quantity of radiometric data eliminated is much lower than in the normality tests based RFI detection.

Regarding the Spectrogram analysis, the method to ensure that the algorithm is performing correctly in the absence of RFI is testing a RFI-free radiometric measurement applying a determined Pfa; hence if the number of pixels flagged as RFI (Pdet) is equal to this Pfa, the algorithm works properly. On the other hand, it is useful to calculate the retrieved antenna temperature (TA), before and after the application of the RFI algorithms in order to obtain the error produced by the pixel elimination in the retrieved TA in RFI free data. Therefore, configuration parameters of these algorithms must be selected in order to have a compromise between the TA error in absence of RFI and the elimination of the maximum number of RFI contaminated pixels.

181

Chapter 8

The TA error should be lower than the desired radiometric resolution. The TA error of the different algorithms must be compared with the radiometric resolution of the MERITXELL radiometer in order to put it into scale.

After a preliminary calibration of the MERITXELL radiometer, the radiometer temperature has been measured as TREC = 41 K with a reference load temperature of TREF = 301 K; thus, the radiometric resolution of this band when the radiometer works as an ADC with 65536 samples is defined as:

∆TL-band =

TREF + TREC Bτ

=

TREF + TREC f sTs 65536

=

301 + 41 ≈ 1.34K 65536

(8.1)

where fs is the sampling frequency of the ADC (which is equal to the ADC bandwidth) and Ts is the sampling period. The first algorithm to be tested is the Smoothing Algorithm.

8.2.2.1.1 Smoothing Algorithm behaviour with RFI-free data

After testing the Smoothing Algorithm the same RFI-free data as the normality tests, the RFI Pdet as a function of Pfa for the L-band are represented in Table 8.23. In this case the number of samples of the radiometric data is set to 216 in order to have a sufficiently large spectrogram, with a size of 512x509 pixels in our case of study. Each row in Table 8.23 represents the Pdet as a function of the Smoothing algorithm Pfa (Pfa|S) for a determined size of the bi-dimensional filter used to smooth the spectrogram (see Chapter 5, Section 5.2.1). These results have been obtained by the averaging of 500 RFI-free measurements.

182

RFI detection algorithms applied to radiometric data

Table 8.23: Smoothing Algorithm performance (Pdet) on L-band RFI free radiometric data as a function of the Pfa|S and the 2-D filter size with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

1x1 3x3 5x5 7x7 9x9 11x11 13x13 15x15 17x17 19x19 21x21 23x23 25x25 27x27 29x29 31x31 33x33 35x35

0.1 0.1001 0.0996 0.0994 0.0991 0.0986 0.0982 0.0976 0.0971 0.0966 0.0961 0.0956 0.0951 0.0945 0.0939 0.0934 0.0928 0.0923 0.0917

0.05 0.0501 0.0498 0.0497 0.0496 0.0493 0.0491 0.0488 0.0486 0.0483 0.0480 0.0477 0.0475 0.0472 0.0469 0.0467 0.0464 0.0461 0.0459

0.01 0.0100 0.0099 0.0100 0.0099 0.0099 0.0098 0.0098 0.0097 0.0097 0.0096 0.0096 0.0095 0.0095 0.0095 0.0094 0.0094 0.0094 0.0094

0.005 0.0050 0.0050 0.0050 0.0050 0.0049 0.0049 0.0049 0.0049 0.0049 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048

0.001 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0010 0.0011 0.0011

The Pdet observed in Table 8.23 is almost equal to the Pfa|S having a higher error value in case of enlarging the smoothing filter; which can be a result of the first approximation of the mean value of the RFI free power. However, the Smoothing Algorithm behaviour with RFI-free radiometric data can be considered quite acceptable.

The advantage of the Smoothing Algorithm is that, while Pdet remains almost constant for the different sizes of the smoothing filter, the error in the retrieved TA diminishes with the 2-D filter size of the smoothing algorithm, as it can be seen in the Table 8.24, where the error in the retrieved TA due to the radiometric data elimination is represented. The reason of this is all the pixels are averaged with the neighbouring pixels, therefore, some pixels with low power are flagged as RFI if neighbouring pixels have high power value; besides, some pixels with high power value will not be flagged avoiding a pronounced clipping of the PDF of the radiometric signal.

Results of Table 8.24 are essential to select the suitable 2-D filter size and Pfa|S parameters of the RFI detection algorithm as this retrieved TA error will be always present in our measurements (as they are produced by false alarms), and this error must be selected taking into account our radiometric resolution objective.

183

Chapter 8

Table 8.24: Error [K] in the retrieved TA produced by the application of the Smoothing algorithm on Lband RFI free radiometric data, as a function of the Pfa|S and the 2-D filter size with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

1x1 3x3 5x5 7x7 9x9 11x11 13x13 15x15 17x17 19x19 21x21 23x23 25x25 27x27 29x29 31x31 33x33 35x35

0.1 89,40 78,55 63,84 51,18 41,70 34,73 29,50 25,51 22,39 19,89 17,84 16,13 14,70 13,48 12,43 11,51 10,71 10,01

0.05 55,14 49,21 39,95 31,65 25,48 20,99 17,67 15,17 13,23 11,68 10,42 9,40 8,54 7,81 7,19 6,65 6,18 5,76

0.01 16,29 14,93 12,29 9,59 7,55 6,09 5,04 4,27 3,67 3,21 2,85 2,55 2,30 2,10 1,93 1,78 1,66 1,54

0.005 9,34 8,62 7,14 5,55 4,35 3,48 2,87 2,42 2,07 1,81 1,60 1,43 1,29 1,18 1,08 1,00 0,93 0,87

0.001 2,42 2,24 1,91 1,49 1,15 0,92 0,75 0,62 0,53 0,47 0,41 0,37 0,34 0,31 0,29 0,26 0,25 0,23

8.2.2.1.2 FIAT Algorithm behaviour with RFI free data

This algorithm is also tested with the same L-band RFI-free data as the Smoothing Algorithm, and also with the same number of samples (65536). The RFI Pdet as a function of the Pfa for the L-band is presented in Table 8.25, and the error in the retrieved TA due to the radiometric data elimination is presented in Table 8.26. In both Tables the FIAT algorithm performance (Table 8.25) and error (Table 8.26) are shown in the first row; while second and third rows show the performance and error of the independent frequency and time thresholding respectively (Section 5.2.4).

Table 8.25 shows that the Pdet coincides with the FIAT algorithm Pfa (Pfa|FIAT) in the absence of RFI, thus proving that the FIAT algorithm works properly with RFI-free radiometric data. If Tables 8.24 and 8.26 are compared, it can be seen that the error in the retrieved TA in the FIAT algorithm is much smaller than the one in the Smoothing Algorithm. The reason is the FIAT algorithm is similar to the Smoothing Algorithm with a large 2-D filter size (but with time and frequency intervals); hence, segments

184

RFI detection algorithms applied to radiometric data

eliminated contain mixed high and low power values, avoiding a hard clipping of the Gaussian noise PDF. This clipping would lead to a decrease in the mean retrieved TA value, and therefore an increase in the retrieved TA error.
Table 8.25: FIAT Algorithm performance (Pdet) on L-band RFI free radiometric data as a function of the Pfa|FIAT with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

0.1 Combined Frequency and Time thresholding (FIAT) Pdet Frequency thresholding Pdet Time thresholding Pdet 0.1093 0.0567 0.0558

0.05 0.0589 0.0301 0.0297

0.01 0.0142 0.0072 0.0070

0.005 0.0078 0.0040 0.0039

0.001 0.0021 0.0011 0.0010

Table 8.26: Error in the retrieved TA produced by the application of the FIAT algorithm on L-band RFI free radiometric data, as a function of the Pfa|FIAT with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

0.1 Combined Frequency and Time thresholding (FIAT) [K] Frequency thresholding [K] Time thresholding [K] 5,29 2,70 2,64

0.05 3,12 1,58 1,55

0.01 0,90 0,46 0,44

0.005 0,53 0,27 0,26

0.001 0,16 0,08 0,08

8.2.2.1.3 Combined Smoothing and FIAT algorithms behaviour with RFI-free data

As stated in Chapter 5, the combination of both the FIAT and the Smoothing algorithms gives a better performance than these algorithms separated, so the same data used in the previous cases has been tested with both combined algorithms, and Pdet and the error in the retrieved TA for this case are presented in Tables 8.27 and 8.28. In this case, the 2-D filter size of the Smoothing algorithm is presented only for four values due to the fact of the high quantity of obtained data, constructing a Table with four separated quadrants to represent it. Table 8.27 represents the Pdet after applying to the RFI free data the Smoothing Algorithm and the FIAT Algorithm; hence, the Pdet depends on three parameters, the 2-D filter size, which is represented in the upper-left part of every quadrant, the Pfa|S (columns in each quadrant), and the Pfa|FIAT (rows in each quadrant).

185

Chapter 8

It can be noted that after applying two different algorithms, the Pdet is not equal to any of the two Pfa‘s as in the two algorithms applied separately, because both algorithms eliminate different data, hence after the Smoothing algorithm RFI detection and elimination, the FIAT algorithm will present a lower rate of RFI detection.

Table 8.27: Combined performance of the Smoothing Algorithm and the FIAT Algorithm (Pdet) on L-band RFI free radiometric data as a function of the Pfa of the Smoothing Algorithm (columns), the Pfa of the FIAT Algorithm (rows) and size of the 2-D filter with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

1x1
0.1 0.05 0.01 0.005 0.001 Pfa|FIAT

0.1

0.05

Pfa|S 0.01 0.005 0.001

5x5
0.1 0.05 0.01 0.005 0.001 Pfa|FIAT

0.1

0.05

Pfa|S 0.01 0.005 0.001

0,1008 0,0597 0,0679 0,0799 0,0982 0,1001 0,0521 0,0344 0,0394 0,0507 0,1001 0,0501 0,0132 0,0109 0,0118 0,1001 0,0501 0,0114 0,0077 0,0066 0,1001 0,0501 0,0102 0,0056 0,0024

0,1120 0,0766 0,0799 0,0871 0,1007 0,1048 0,0592 0,0413 0,0447 0,0527 0,1012 0,0509 0,0149 0,0124 0,0124 0,1006 0,0503 0,0124 0,0087 0,0070 0,0999 0,0499 0,0104 0,0058 0,0024

15x15
0.1 0.05 0.01 0.005 0.001 Pfa|FIAT

0.1

0.05

0,1444 0,1133 0,1176 0,0777 0,1008 0,0537 0,0991 0,0510 0,0977 0,0491

Pfa|S 31x31 0.01 0.005 0.001 0,1023 0,1037 0,1068 0.1 0,0566 0,0562 0,0575 0.05 0,0196 0,0161 0,0142 0.01 0,0148 0,0108 0,0081 0.005 0,0109 0,0063 0,0028 0.001 Pfa|FIAT

0.1

0.05

Pfa|S 0.01 0.005 0.001

0,1607 0,1290 0,1100 0,1088 0,1088 0,1254 0,0870 0,0621 0,0600 0,0589 0,0992 0,0548 0,0215 0,0176 0,0148 0,0960 0,0505 0,0157 0,0116 0,0085 0,0935 0,0474 0,0110 0,0065 0,0030

Table 8.28: Total error [K] in the retrieved TA produced by the application of the Smoothing Algorithm and the FIAT Algorithm on L-band RFI free radiometric data as a function of the Pfa of the Smoothing Algorithm (columns), the Pfa of the FIAT Algorithm (rows) and size of the 2-D filter with sample size of 216 samples (RBW = 10 MHz, fs = 2 MHz).

1x1
0.1 0.05 0.01 0.005 0.001 Pfa|FIAT

0.1
89,43 89,40 89,40 89,40 89,40

0.05
55,56 55,24 55,14 55,14 55,14

Pfa|S 0.01 0.005 0.001
18,89 17,52 16,48 16,38 16,30 12,78 11,09 9,71 9,52 9,38 7,04 3,10 2,80 2,53 5,01

5x5
0.1 0.05 0.01 0.005 0.001 Pfa|FIAT

0.1
64,52 64,18 63,98 63,94 63,89

0.05
41,21 40,46 40,03 40,00 39,97

Pfa|S 0.01 0.005 0.001
15,51 13,89 12,60 12,46 12,32 10,97 9,18 7,61 7,39 7,20 6,67 4,61 2,63 2,32 2,02

15x15
0.1 0.05 0.01 0.005 0.001 Pfa|FIAT

0.1
28,05 26,74 25,78 25,66 25,56

0.05
18,39 16,79 15,52 15,35 15,22

Pfa|S 31x31 0.01 0.005 0.001 8,71 7,16 5,74 0.1 6,74 5,12 3,61 0.05 4,90 3,13 1,46 0.01 4,61 2,82 1,11 0.005 4,36 2,53 0,77 0.001 Pfa|FIAT

0.1
16,50 14,59 12,96 12,72 12,51

0.05
11,77 9,68 7,81 7,52 7,28

Pfa|S 0.01 0.005 0.001
7,23 4,97 2,77 2,40 2,06 6,55 4,26 1,97 1,59 1,23 5,93 3,59 1,23 0,84 0,45

186

RFI detection algorithms applied to radiometric data

Table 8.28 represents the error in the retrieved TA due to the radiometric data elimination; and in this table is more clear how to tune the three different parameters in order to have an acceptable error in the radiometric measurements in a RFI-free situation. As in the Smoothing Algorithm alone case, configuration parameters must be selected as the TA error is lower than our radiometric resolution goal. The Pdet parameter represented in Tables 8.23, 8.25 and 8.27 also gives information on the fraction of the signal that is eliminated, thus losing radiometric data and decreasing the radiometric resolution; following the eq. (5.17) defined in Chapter 5. Therefore, the Pdet parameter must be taken into account as the error in the retrieved TA.

8.2.2.2 Spectrogram-based algorithms with contaminated data
The same algorithms tested with RFI free data in the previous section will be used with real radiometric data with RFI present on it. This way, the sampled data segments have a size of 65536 samples, obtaining a spectrogram with a size of 512x509 pixels (equivalent to an overlapping of 75% in the STFT calculation)

The L-band radiometric measurements used with the normality tests-based algorithms have been used with the spectrogram-based algorithms. However, measured antenna TA in the absence of RFI has a higher value than the matched load TA, (approximately TA = 327 K for a Tmatched load = 301 K) probably due to the fact that the measurements have been performed in the UPC Remote Sensing Laboratory (RSLab) and there is RFI coming from the surroundings.

The L-band data is separated in sub-bands some of them are highly RFI contaminated, some of them are slightly RFI contaminated and a few ones are not RFI contaminated at all. One radiometric measurement of each of these three cases will be studied in order to study the algorithms performance.

First of all the Pfa|S, the 2-D filter size and the Pfa|FIAT parameters must be selected, as these parameters characterize the error value produced by false alarm detections. In this work, a maximum error value in the absence of RFI is considered to

187

Chapter 8

be 5 K, thus the values selected for the three different parameters are: 2-D filter size = 15x15, Pfa|S = 0.01 and Pfa|FIAT = 0.01 (Table 8.28). The maximum error in absence of RFI corresponds to the combined Smoothing and FIAT algorithms, while both algorithms working separately will have lower error value (4.3 K for the Smoothing algorithm and 0.9 K for the FIAT algorithm, as shown in Tables 8.24 and 8.26 respectively).

8.2.2.2.1

Spectrogram-based

algorithms

behaviour

with

low

RFI

contaminated radiometric data

The selected low RFI contaminated sub-band has the centre frequency equal to 1.408 GHz. Normality tests applied to this sub-band failed to detect any RFI, as it can be seen in the 5th column of the Table 8.17 which corresponds to the sub-band centred around 1.408 GHz. In addition, a quick examination of this sub-band spectrogram (Fig. 8.5a) does not reveal any RFI present in this radiometric data. However, if an average is applied to the Spectrogram’s rows (to obtain a frequency domain average vector) and columns (to obtain a time domain average vector), it can be easier to “see” the RFI in a figure, as it can be seen in Figs. 8.5b (average of the rows to obtain only one frequency column) and 8.5c (average of the columns to obtain only one time row), where a tone 1.5 dB above the noise can be observed.

188

RFI detection algorithms applied to radiometric data

1407

-30 -40

1407.5 Frequency [MHz] -50 -60 1408 -70 -80 -90 1409

1408.5

0

5

10

15 20 Time [ms]

25

30

a)
-39 -40.5

-40 Power [dBm] Power [dBm] 1407.5 1408 1408.5 Frequency [MHz] 1409

-41

-41

-41.5

-42

-42

-43 1407

-42.5

0

10

20 Time [ms]

30

b)
th

c)

Figure 8.5: Data of the 5 sub-band of the L-band measurement (1.408 GHz) before application of the RFI algorithms, RBW = 10MHz. a) Spectrogram of the data, formed by 509 time points per 512 frequency points. b) Frequency domain average data (512 points). c) Time domain average data (509 points).

First of all the Smoothing algorithm is tested, with the configuration parameters exposed in the previous section. In Fig. 8.6 is represented the result of the application of the Smoothing algorithm to the selected data. Figure 8.6a is equal to the Fig. 8.5a, and is included for comparison purposes with the Fig. 8.6b which corresponds to the previous spectrogram with the pixels marked after the application of the RFI detection algorithm (Smoothing algorithm in this case); Fig. 8.6c and 8.6e correspond to the average of the different frequency and time segments of the spectrogram before the application of the Smoothing algorithm (and also are equal to the Figs. 8.5b and 8.5c), and Figs. 8.6d and 8.6f correspond to the average of the different frequency and time segments of the spectrogram after the application of the RFI detection algorithm (Smoothing algorithm).

189

Chapter 8

The randomly eliminated clusters of pixels observed in the Fig. 8.6b probably correspond to false alarms. After the application of the Smoothing algorithm, a tone 1.5 dB over the noise power can be observed in both Figs. 8.6c and 8.6d, stating that the Smoothing algorithm has not been able to eliminate this tone, however, the algorithm has attenuated its power.

In Table 8.29 the most important results of the application of the RFI detection algorithms to this data are presented. These results are the TA after the RFI detection algorithm application (final TA [K]), the difference between the TA initially measured by the antenna and the TA after the RFI detection algorithm application (initial TA − final TA [K]), the proportion of eliminated pixels of the spectrogram (flagged pixels [%]), and the proportion of increase in the radiometric resolution due to this pixel elimination (∆T increase [%]).
Table 8.29: Most relevant results obtained after the RFI detection algorithms application to the 5th sub-band of the L-band measurement. The TA initially measured by the antenna is TA = 327.82 K

Algorithm Smoothing FIAT Smoothing and FIAT

Final TA [K] 322.89 326.48 321.97

initial TA − final TA [K] 4.93 1.34 5.85

flagged pixels [%] 1.11 1.37 2.24

∆T increase [%] 0.56 0.69 1.14

As seen in Table 8.29, the error in the TA for the Smoothing algorithm is 4.93 K, similar to the error obtained with RFI free data (4.29 K). The worsening of the radiometric resolution is negligible.

190

RFI detection algorithms applied to radiometric data

1407

1407

-30 -40

1407.5 Frequency [MHz]

1407.5 Frequency [MHz] -50 -60 1408 -70 -80 -90

1408

1408.5

1408.5

1409

0

5

10

15 20 Time [ms]

25

30

1409

0

5

10

15 20 Time [ms]

25

30

a)
-39

b)
-40 -40.5 Power [dBm] -41 -41.5 -42 -42.5 1407

-40 Power [dBm]

-41

-42

-43 1407

1407.5 1408 1408.5 Frequency [MHz]

1409

1407.5 1408 1408.5 Frequency [MHz]

1409

c)
-40.5

d)
-40.5

-41 Power [dBm]

-41 Power [dBm]
0 10 20 Time [ms] 30

-41.5

-41.5

-42

-42

-42.5

-42.5

0

10

20 Time [ms]

30

e)

f)

Figure 8.6: Results of the application of the Smoothing Algorithm with the parameters Pfa|S = 0.01 and 2-D filter size = 15 to the 5th sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing algorithm: a) Spectrogram;, c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

The next tested algorithm is the FIAT algorithm; in this case, the tone located at 1.408 GHz has been detected and eliminated, in addition to other tones with a power higher to the mean power estimation. In Fig. 8.7 results obtained by the application of the FIAT algorithm are presented. This figure follows the same structure than the Fig. 8.6, only substituting the Smoothing algorithm by the FIAT algorithm.

191

Chapter 8

1407

1407

-30 -40

1407.5 Frequency [MHz]

1407.5 Frequency [MHz] -50 -60 1408 -70 -80 -90

1408

1408.5

1408.5

1409

0

5

10

15 20 Time [ms]

25

30

1409

0

5

10

15 20 Time [ms]

25

30

a)
-39 -40.5

b)

-40 Power [dBm] Power [dBm] 1407.5 1408 1408.5 Frequency [MHz] 1409

-41

-41

-41.5

-42

-42

-43 1407

-42.5 1407

1407.5 1408 1408.5 Frequency [MHz]

1409

c)
-40.5

d)
-40.5

-41 Power [dBm]

-41 Power [dBm]
0 10 20 Time [ms] 30

-41.5

-41.5

-42

-42

-42.5

-42.5

0

10

20 Time [ms]

30

e)

f)

Figure 8.7: Results of the application of the FIAT Algorithm with the parameters Pfa|FIAT = 0.01 to the 5th sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

In the FIAT algorithm case, the obtained TA error (1.34 K) is also very close to the error obtained with RFI free data (0.9 K). The increase in radiometric resolution is also negligible.

192

RFI detection algorithms applied to radiometric data

Finally, both algorithms are tested jointly, first the Smoothing algorithm and then the FIAT algorithm. The data presented in Fig. 8.8 corresponds to the original spectrogram before the RFI analysis (Fig. 8.8a), the spectrogram with marked pixels as RFI after the Smoothing algorithm application (Fig. 8.8b), and the spectrogram with marked pixels as RFI after the Smoothing and the FIAT algorithms application (Fig. 8.8c). Following these three captions, Figs. 8.8d and 8.8g correspond to the average of the different frequency and time segments of the spectrogram before the application of both RFI detection algorithms; Figs. 8.8e and 8.6h correspond to the average of the different frequency and time segments of the spectrogram after the application of the Smoothing algorithm and Figs. 8.8f and 8.8i correspond to the average of the different frequency and time segments of the spectrogram after the application of the both Smoothing and FIAT algorithms.

The 1.408 GHz tone has been eliminated, (Fig. 8.8f), but all the Smoothing algorithms false alarm spots are present (Figs. 8.8b and 8.8c). In the combined algorithms case, the TA error (5.85 K) is also close to the TA error obtained with RFI free data (4.9 K). Again, the radiometric resolution increase due to the pixel elimination is negligible.

193

Chapter 8
1407
1407 1407

-30 -40

Frequency [MHz]

Frequency [MHz]

Frequency [MHz]

1407.5

1407.5

1407.5

-50 -60 -70

1408

1408

1408

1408.5

1408.5

1408.5

-80 -90

1409

0

10

20 Time [ms]

30

1409

0

10

20 Time [ms]

30

1409

0

10

20 Time [ms]

30

a)
-39 -40 -40.5 Power [dBm] -41 -41.5 -42 -42.5 1407 -40 Power [dBm]

b)
-40.5 -41 Power [dBm]

c)

-41

-41.5

-42

-42

-43 1407

1407.5 1408 1408.5 Frequency [MHz]

1409

1407.5 1408 1408.5 Frequency [MHz]

1409

-42.5 1407

1407.5 1408 1408.5 Frequency [MHz]

1409

d)
-40.5 -40.5 -41 Power [dBm] Power [dBm] -41

e)
-41 Power [dBm]

f)

-41.5

-41.5

-41.5

-42

-42

-42

-42.5

0

10

20 Time [ms]

30

-42.5

0

10

20 Time [ms]

30

-42.5

0

10

20 Time [ms]

30

g)

h)

i)

Figure 8.8: Results of the application of the Smoothing and FIAT algorithms combined with the parameters Pfa|S = 0.01, 2-D filter size = 15 and Pfa|FIAT = 0.01 to the 5th sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the Smoothing and FIAT algorithms: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data.

8.2.2.2.2 Spectrogram-based algorithms behaviour with medium RFI contaminated radiometric data For this section, the sub-band centred around 1.418 GHz (10th column) has been selected for the Kurtosis-based algorithms to detect RFI, although the ECDF based algorithms fail to detect it (Table 8.17), hence an spectrogram analysis can be useful.

In this case, the spectrogram of this sub-band (Fig. 8.9a) shows some RFI present in the measurement, a tone at a frequency near 1.4175 GHz, and two pulsed

194

RFI detection algorithms applied to radiometric data

tones at 1.417 GHz and 1.419 GHz respectively. In Fig. 8.9b interferences can be observed more clearly. Figure 8.9 follows the same distribution as the Fig. 8.5.

Table 8.30 presents the same values as the Table 8.29, but in this case with the 10th sub-band of the L-band measurement, centered at 1.418 GHz.
Table 8.30: Most relevant results obtained after the RFI detection algorithms application to the 10th sub-band of the L-band measurement. The TA initially measured by the antenna is TA = 390.75 K

Algorithm Smoothing FIAT Smoothing and FIAT

Final TA [K] 358.8 358.2 353.77

initial TA − final TA [K] 31.95 32.55 36.98

flagged pixels [%] 4.92 6.6 8.41

∆T increase [%] 2.55 3.47 4.49

Analysing the Table 8.30, the FIAT algorithm flags a higher proportion of pixels than the Smoothing algorithm may be due to the complete elimination of the tones as it can be observed in Fig. 8.11d. On the other hand, while, the retrieved final TA is equal for both algorithms, but lower in the combined case that may be due to the sum of false alarms of both algorithms which are different as it can be seen in Figs. 8.10b, 8.11b and 8.12c.

195

Chapter 8

1417 -30 -40 1417.5 Frequency [MHz] -50 -60 1418 -70 1418.5 -80 -90 1419 -100

0

5

10

15 20 Time [ms]

25

30

a)
-30 -32 -40.5 Power [dBm] -34 -36 -38 -40 -42 1417 1417.5 1418 1418.5 Frequency [MHz] 1419 -42 0 10 20 Time [ms] 30 Power [dBm] -40

-41

-41.5

b)
th

c)

Figure 8.9: Data of the 10 sub-band of the L-band measurement (1.418 GHz) before application of the RFI algorithms, RBW = 10MHz. a) Spectrogram of the data, formed by 509 time points per 512 frequency points. b) Frequency domain average data (512 points). c) Time domain average data (509 points).

First the Smoothing algorithm is applied, with the same configuration parameters as in the previous case. In this case the Smoothing algorithm detects several of these tones and pulsed tones, however, in Fig. 8.10d several undetected tones can still be observed.

196

RFI detection algorithms applied to radiometric data

1417

1417 -30 -40

1417.5 Frequency [MHz] Frequency [MHz]

1417.5 -50 -60 1418 -70 1418.5 -80 -90

1418

1418.5

1419

0

5

10

15 20 Time [ms]

25

30

1419

0

5

10

15 20 Time [ms]

25

30

-100

a)
-30 -32 -39 Power [dBm] -34 -36 -38 -40 -42 1417 1417.5 1418 1418.5 Frequency [MHz] 1419 -42 1417 Power [dBm] -38

b)

-40

-41

1417.5 1418 1418.5 Frequency [MHz]

1419

c)
-40 -40.5

d)

-40.5 Power [dBm] Power [dBm] 0 10 20 Time [ms] 30 -41

-41

-41.5

-41.5

-42

-42

0

10

20 Time [ms]

30

e)

f)

Figure 8.10: Results of the application of the Smoothing Algorithm with the parameters Pfa|S = 0.01 and 2-D filter size = 15 to the 10th sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

The application of the FIAT algorithm shows a better detection of the tones, however, the drawback is that the frequencies where the pulsed tones are located, are completely flagged.

197

Chapter 8

1417

1417 -30 -40

1417.5 Frequency [MHz] Frequency [MHz]

1417.5 -50 -60 1418 -70 1418.5 -80 -90

1418

1418.5

1419

0

5

10

15 20 Time [ms]

25

30

1419

0

5

10

15 20 Time [ms]

25

30

-100

a)
-30 -32

b)
-40

-40.5
Power [dBm] -34 -36 -38 -40 -42 1417 1417.5 1418 1418.5 Frequency [MHz] 1419

Power [dBm]

-41

-41.5

-42 1417

1417.5 1418 1418.5 Frequency [MHz]

1419

c)
-40
-40.5

d)

-40.5 Power [dBm]
Power [dBm] -41

-41

-41.5

-41.5

-42

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

e)

f)

Figure 8.11: Results of the application of the FIAT Algorithm with the parameters Pfa|FIAT = 0.01 to the 10th sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

The combination of both algorithms seem to share good points of both algorithms as all the tones have been flagged (Fig. 8.12f), and frequencies with pulsed tones have not completely eliminated (Fig. 8.12c).

198

RFI detection algorithms applied to radiometric data
1417
1417

1417 -40 Frequency [MHz] 1417.5 -60

Frequency [MHz]

Frequency [MHz]

1417.5

1417.5

1418

1418

1418

1418.5

1418.5

1418.5

-80

1419

0

10

20 Time [ms]

30

1419

0

10

20 Time [ms]

30

1419

0

10

20 Time [ms]

30

-100

a)
-30 -32

b)
-38
-40.5

c)
-41 Power [dBm]

-39
Power [dBm] -34 -36 -38 -40 -42 1417 1417.5 1418 1418.5 Frequency [MHz] 1419

Power [dBm]

-40

-41.5

-41

-42

-42 1417

1417.5 1418 1418.5 Frequency [MHz]

1419

-42.5 1417

1417.5 1418 1418.5 Frequency [MHz]

1419

d)
-40 -40.5 -40.5 Power [dBm] Power [dBm] -41

e)
-40.5 Power [dBm]

f)

-41

-41

-41.5

-41.5

-41.5

-42

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

g)

h)

i)

Figure 8.12: Results of the application of the Smoothing and FIAT algorithms combined with the parameters Pfa|S = 0.01, 2-D filter size = 15 and Pfa|FIAT = 0.01 to the 10th sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the Smoothing and FIAT algorithms: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data.

8.2.2.2.3

Spectrogram-based

algorithms

behaviour

with

high

RFI

contaminated radiometric data

Sub-bands with high RFI values can have a large part of the time-frequency domain contaminated (Fig. 8.13a). In this case, the 3rd sub-band of the L-band (Table 8.17) with a centre frequency of 1.404 GHz has an important RFI component concentrated between 1.4035 and 1.404 GHz (Fig. 8.13a and 8.13b), in addition, RFI varies in time (Fig. 8.13c) having more power during the first 30 ms.

199

Chapter 8 Table 8.31: Most relevant results obtained after the RFI detection algorithms application to the 3rd sub-band of the L-band measurement. The TA initially measured by the antenna is TA = 799.05 K

Algorithm Smoothing FIAT Smoothing and FIAT

Final TA [K] 392.56 383.51 380.66
1403

initial TA − final TA [K] 406.49 415.54 418.39

flagged pixels [%] 24.09 27.33 28.19

∆T increase [%] 14.78 17.31 18.01

-20 -30

1403.5 Frequency [MHz]

-40 -50

1404 -60 -70 1404.5 -80 -90 0 5 10 15 20 Time [ms] 25 30

1405

a)
-25 -37 -37.5 Power [dBm] 1403.5 1404 1404.5 Frequency [MHz] 1405 -38 -38.5 -39 -39.5

-30 Power [dBm]

-35

-40

-45 1403

0

10

20 Time [ms]

30

b)
rd

c)

Figure 8.13: Data of the 3 sub-band of the L-band measurement (1.404 GHz) before application of the RFI algorithms, RBW = 10MHz. a) Spectrogram of the data, formed by 509 time points per 512 frequency points. b) Frequency domain average data (512 points). c) Time domain average data (509 points).

The application of the two RFI detection algorithms (Smoothing and FIAT), separately and jointly, has not provided very good results, as still some residual RFI has not been eliminated from the data (Figs. 8.14, 8.15 and 8.16). This is specially present in the frequency domain representations (Figs. 8.14d, 8.15d and 8.16f). Table 8.31 presents the same data as the Tables presented in the previous sections. However, as the RFI detection algorithms seem not to perform correctly, it would be better to slightly

200

RFI detection algorithms applied to radiometric data

change the design of the RFI detection algorithms and apply them on the highly RFI contaminated data again in order to have better results.
1403 1403

-20 -30

1403.5 Frequency [MHz] Frequency [MHz]

1403.5

-40 -50

1404

1404 -60 -70 1404.5 -80 -90 0 5 10 15 20 Time [ms] 25 30

1404.5

1405

0

5

10

15 20 Time [ms]

25

30

1405

a)
-25
-36 -37

b)

-30 Power [dBm]
Power [dBm] -38 -39 -40 -41

-35

-40

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

-42 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

c)
-37 -37.5 Power [dBm] -38 -38.5 -39 -39.5 Power [dBm] -39.5 -40 -40.5 -41 -41.5 -42

d)

0

10

20 Time [ms]

30

0

10

20 Time [ms]

30

e)

f)

Figure 8.14: Results of the application of the Smoothing Algorithm with the parameters Pfa|S = 0.01 and 2-D filter size = 15 to the 3rd sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

201

Chapter 8

1403

1403

-20 -30

1403.5
Frequency [MHz]

1403.5

-40 -50

Frequency [MHz]

1404

1404 -60 -70 1404.5 -80 -90 0 5 10 15 20 Time [ms] 25 30

1404.5

1405

0

5

10

15 20 Time [ms]

25

30

1405

a)
-25
-39 -39.5

b)

-30 Power [dBm]
Power [dBm] -40 -40.5 -41 -41.5

-35

-40

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

-42 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

c)
-37 -37.5 Power [dBm] -38 -38.5 -39 -39.5 Power [dBm] -40

d)

-40.5

-41

-41.5

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

e)

f)

Figure 8.15: Results of the application of the FIAT Algorithm with the parameters Pfa|FIAT = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

202

RFI detection algorithms applied to radiometric data
1403
1403

1403

-20

Frequency [MHz]

Frequency [MHz]

Frequency [MHz]

1403.5

1403.5

1403.5

-40

1404

1404

1404

-60

1404.5

1404.5

1404.5 -80

1405

0

10

20 Time [ms]

30

1405

0

10

20 Time [ms]

30

1405

0

10

20 Time [ms]

30

a)
-25
-36 -37

b)
-40 -40.5
-38 -39 -40 -41

c)
Power [dBm]

-30 Power [dBm]
Power [dBm]

-35

-41

-40

-41.5

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

-42 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

-42 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

d)
-37 -37.5 Power [dBm] Power [dBm] -38 -38.5 -39 -39.5 -39.5 -40 -40.5 -41 -41.5 -42

e)
-40 -40.5 Power [dBm]

f)

-41

-41.5

0

10

20 Time [ms]

30

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

g)

h)

i)

Figure 8.16: Results of the application of the Smoothing and FIAT algorithms combined with the parameters Pfa|S = 0.01, 2-D filter size = 15 and Pfa|FIAT = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the Smoothing and FIAT algorithms: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data.

With all these data it can be stated that the Smoothing algorithms seem not to work properly in case of a high level of RFI contamination. Then, something in the algorithms is not performing correctly. The most important issue is the approximation of the standard deviation of the noise power, which is strongly contaminated by RFI. The way to approximate it has been the elimination of the 25% of the pixels with higher values (in a similar way as in [25]), assuming that the RFI will not affect to the rest of the 75% of the pixels, which is not the case when RFI is strong.

8.2.2.2.4 Improvements of the Spectrogram-based algorithms for large RFI contaminated radiometric data

203

Chapter 8

In order to address the problem, a recurrent algorithm has been developed. First, the approximated standard deviation is approximated by the average in the absence of RFI, as the noise power follows an exponential distribution, thus the approximated mean of the noise power is calculated and taken as the approximated standard deviation. This standard deviation is used to calculate the thresholds used in the RFI detection algorithms. This standard deviation will be larger than in the RFI-free case, but will be enough to detect the most RFI contaminated pixels.

After these pixels have been detected, the mean is computed again not taking them into account, obtaining a lower value which is used as the new threshold. RFI detection algorithms are applied to the original data, increasing the performance as more RFI contaminated pixels are flagged, thus being able to calculate a more accurate RFI free mean/standard deviation value. This process is stopped when difference between the ith and the (i−1)th computed mean have a difference in the brightness temperature value lower than a predefined threshold, for example, 1 K. This iterative process can increase the TA error produced by false alarms, but for the Pfa value of 0.01 this increase has been found to be negligible. This increase is equal to 20% for Pfa = 0.05. In addition to this recurrent design in both algorithms (and in the combined version), it has been decided to use a 2-D filter size of 31x31 pixels, thus eliminating more pixels, but achieving a better performance. The other two parameters Pfa|S and Pfa|FIAT have not been changed. This way, the error in the retrieved TA for the Smoothing algorithm decreases to 1.78 K, and in the combined case, error in the retrieved TA decreases to 2.77 K (almost equal to the sum of errors of both RFI detection algorithms). Hence, the modified Smoothing and FIAT algorithms are applied to the same segment of data belonging to the 3rd sub-band of the L-band, at 1.404 GHz.

204

RFI detection algorithms applied to radiometric data

1403

1403

-20 -30

1403.5 Frequency [MHz] Frequency [MHz]

1403.5

-40 -50

1404

1404 -60 -70 1404.5 -80 -90 0 5 10 15 20 Time [ms] 25 30

1404.5

1405

0

5

10

15 20 Time [ms]

25

30

1405

a)
-25
-39.5 -40 Power [dBm] -40.5 -41 -41.5 -42 1403

b)

-30 Power [dBm]

-35

-40

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

1403.5 1404 1404.5 Frequency [MHz]

1405

c)
-37 -37.5 Power [dBm] -38 -38.5 -39 -39.5 Power [dBm] -40

d)

-40.5

-41

-41.5

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

e)

f)

Figure 8.17: Results of the application of the iterative Smoothing Algorithm with the parameters Pfa|S = 0.01 and 2-D filter size = 31 to the 3rd sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the last iteration of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

205

Chapter 8

1403

1403

-20 -30

1403.5 Frequency [MHz] Frequency [MHz]

1403.5

-40 -50

1404

1404 -60 -70 1404.5 -80 -90 0 5 10 15 20 Time [ms] 25 30

1404.5

1405

0

5

10

15 20 Time [ms]

25

30

1405

a)
-25
-40

b)

-30 Power [dBm]
Power [dBm]

-40.5

-35

-41

-40

-41.5

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

-42 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

c)
-37 -37.5 Power [dBm] -38 -38.5 -39 -39.5 Power [dBm] -40

d)

-40.5

-41

-41.5

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

e)

f)

Figure 8.18: Results of the application of the iterative FIAT Algorithm with the parameters Pfa|FIAT = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after application of the last iteration of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

206

RFI detection algorithms applied to radiometric data
1403
1403

1403

-20

Frequency [MHz]

Frequency [MHz]

Frequency [MHz]

1403.5

1403.5

1403.5

-40

1404

1404

1404

-60

1404.5

1404.5

1404.5 -80

1405

0

10

20 Time [ms]

30

1405

0

10

20 Time [ms]

30

1405

0

10

20 Time [ms]

30

a)
-25 -39.5 -40 -30 Power [dBm] Power [dBm]

b)
-40.5 Power [dBm]

c)

-41

-40.5 -41 -41.5 -42 1403

-35

-41.5

-40

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

1403.5 1404 1404.5 Frequency [MHz]

1405

-42 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

d)
-37 -37.5 Power [dBm] Power [dBm] -38 -38.5 -39 -39.5 -40 -40.5

e)
-40 -40.5 Power [dBm]

f)

-41

-41

-41.5

-41.5

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

-42

0

10

20 Time [ms]

30

g)

h)

i)

Figure 8.19: Results of the application of the iterative Smoothing and FIAT algorithms combined with the parameters Pfa|S = 0.01, 2-D filter size = 31 and Pfa|FIAT = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10MHz. Data before the application of the Smoothing and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the last iteration of the Smoothing algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the last iteration of the Smoothing and FIAT algorithm: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data.

Figures 8.17 to 8.19 represent the results obtained by the application of this enhanced algorithms, and Table 8.32 presents the same data as Table, 8.31 but using the enhanced algorithms, in order to compare the Smoothing and FIAT algorithms, combined and their iterative versions.

Table 8.32: Most relevant results obtained after the iterative RFI detection algorithms application to the 3rd sub-band of the L-band measurement. The TA initially measured by the antenna is TA = 799.05 K

Algorithm Smoothing FIAT Smoothing and FIAT

Final TA [K] 369.06 369.80 365.19

initial TA − final TA [K] 429.99 429.25 433.86

flagged pixels [%] 34.67 35.35 36.64

∆T increase [%] 23.72 24.66 25.63

207

Chapter 8

Comparing the data presented in Table 8.32 with the data presented in Table 8.31, it can be observed that the Smoothing algorithm performs similar to the FIAT algorithm with the enhancement described, which is not the case for the original Smoothing algorithm that had a higher retrieved TA value. In both Tables, the combined algorithms have a lower retrieved TA value, but most probably due to the sum of false alarms. The proportion of flagged pixels is obviously higher in all cases of Table 8.32 than the same cases of Table 8.31, as a more accurate and lower approximated mean power has been obtained, flagging more pixels. However, the difference in the radiometric resolution increase is around a 10%, which is acceptable if a more accurate TA is obtained. It can be seen that with the recurrence of the RFI detection algorithms, the RFI present in the frequency responses of the results of the three different RFI detection algorithms seems to be completely eliminated. However, the resulting TA has a very high value in the three cases (Table 8.32). With the observation of the retrieved TA (Table 8.32) it can be stated that both Smoothing and FIAT algorithms have a similar behaviour, thus it will be better to use to simplest one (FIAT algorithm) in order to implement it in future hardware developments. In order to compare the results of the three algorithms, the retrieved TA after the algorithm application and the received TA before the algorithm application of the three different algorithms are presented in Table 8.33, for the 14 different 2MHz sub-bands that compose the radiometric L-band. In this table, results of the three algorithms are quite similar except in bands from 1420 to 1424 GHz, which is due to several tones present in these sub-bands (Fig. 8.20), tones that the Smoothing algorithm is not able to detect.

208

RFI detection algorithms applied to radiometric data

Table 8.33: Brightness temperature values before and after the application of the three RFI detection algorithms for all the protected L-band, separated in 2 MHz wide sub-bands.

Center Frequency [GHz] 1400 1402 1404 1406 1408 1410 1412 1414 1416 1418 1420 1422 1424 1426
1421

Original TA [K] 1230.33 364.38 799.05 392.83 331.98 345.5 382.38 374.68 979.24 388.82 661.13 410.75 620.32 475.82

TA after Smoothing algorithm [K] 432.56 351.2 369.06 329.51 329.67 342.86 352.9 372.18 379.81 356.61 404.84 407.49 414.4 392.89

TA after FIAT algorithm [K] 417.1 350.86 369.8 329.99 330.27 342.6 349.53 370.69 380.1 355.19 357 360.5 385.64 392.71
1421

TA after Smoothing and FIAT algorithms combined [K] 414.73 346.76 365.19 328.75 327.88 340.39 350.62 369.3 377.01 348.85 323.64 312.75 368.8 387.62
-30 -40

1421.5 Frequency [MHz] Frequency [MHz]

1421.5 -50 -60 1422 -70 1422.5 -80 -90

1422

1422.5

1423

0

5

10

15 20 Time [ms]

25

30

1423

0

5

10

15 20 Time [ms]

25

30

a)
1421 1421

b)
-30 -40

1421.5 Frequency [MHz] Frequency [MHz]

1421.5 -50 -60 1422 -70 1422.5 -80 -90

1422

1422.5

1423

0

5

10

15 20 Time [ms]

25

30

1423

0

5

10

15 20 Time [ms]

25

30

c)
th

d)

Figure 8.20: Spectrograms analysis to the 13 sub-band of the protected L-band centered at 1.424 GHz. a) Original spectrogram; b) Spectrogram with flagged pixels after the application of the iterative Smoothing algorithm; c) Spectrogram with flagged pixels the after application of the iterative FIAT algorithm; d) Spectrogram with flagged pixels after the application of the iterative Smoothing and FIAT algorithms combined.

209

Chapter 8

The advantage that the three recurrent algorithms have is that it is not necessary to sort all the samples of the data to calculate the approximated mean with the 75% samples with lower value. Sorting is a very complicated algorithm to develop in real time processing.

8.3 RFI detection algorithms with the power spectrograms
As in the case of the ADC data, RFI-free radiometric data must be first tested to ensure that the RFI detection algorithms perform correctly, i.e. the probability of false alarm is equal to the probability of detection RFI-free conditions. Thus, RFI-free thermal noise from a matched load is first measured and tested.

A fact that must be taken into account is that the two algorithms developed for the Spectrum Analyzer data are Spectrogram based, as the data is directly acquired as a power value.

In order to eliminate possible distortion effects introduced by the frequency response of the radiometer, a matched load has been measured to equalize the radiometer’s frequency response.

The advantage of using the data obtained from the Spectrum Analyzer instead of the ADC sampled data is that the time duration of the spectrogram can be much longer, and the measured bandwidth can be increased. The drawback of increasing the bandwidth is the smaller spectrogram’s frequency resolution; as the number of frequencies in every measurement is fixed to 501 frequencies. In addition, a higher resolution bandwidth (RBW) of the Spectrum analyzer is required, as it must accomplish the eqn. (8.2) [92]:

Span < 500 RBW

(8.2)

This fact, it will not be a problem in RFI-free measurements, but if any RFI is present, it will be dispersed through a bandwidth equivalent to the resolution bandwidth.

210

RFI detection algorithms applied to radiometric data

8.3.1 Algorithms using power spectrograms with RFI-free radiometric data
As previously described in Chapter 5 Section 5.2.3, the Edge Detection Algorithm is used with data obtained by means of ordering the different time traces obtained by the Spectrum Analyzer of the MERITXELL radiometer in columns.

The FIAT algorithm is also implemented to be used with spectrum analyzer data. In fact, the Smoothing algorithm could also be used. Three different measurements are performed, each one with different configurations of the spectrum analyzer. The first measurement comprises the entire protected L-band. In order to try configurations with higher resolution, a new measurement is performed, but in this case with a span of only 2 MHz, same as the 3rd sub-band of the L-band used in Sections 8.2.2.2.3 and 8.2.2.2.4. An additional measurement is performed with a sweep time of 500 ms, longer than the recommended in Section 8.1.2 (50 ms). The configuration of the spectrum analyzer for these three different configurations is detailed in Table 8.34.
Table 8.34: Spectrum analyzer configuration parameters to obtain perform the three different measurements for this Section.

Configuration parameter Center Frequency Span Resolution Bandwidth Video Bandwidth Time Sweep Detector Frequency points Time traces

1st Config. 1413.5 27 100 10 50 Sample 501 500

2nd Config. 1404 2 10 10 50 Sample 501 500

3rd Config. 1413.5 27 100 10 500 Sample 501 500

Units [MHz] [MHz] [kHz] [MHz] [ms] [-] [-] [-]

8.3.1.1 Edge Detector algorithm with RFI-free radiometric data
Data tested is different from algorithms of Section 8.2 as the data is directly power, obtained by different means, but the representation of the results is quite similar. First, the Edge Detector algorithm performance is tested with RFI-free data as the other RFI detection algorithms. The Pdet as a function of the Pfa for the L-band is represented in Table 8.35. In this case spectrogram is formed by 500 consecutive traces composed

211

Chapter 8

by 501 power values, corresponding to 501 frequencies. This way, a 501x500 pixels spectrogram is obtained. Table 8.35 is very simple and only indicates the Pdet as a function of the Edge Detection algorithm Pfa (Pfa|ED), Table 8.36 indicates the Edge Detection error in TA as a function of the Pfa|ED,. These results have been obtained by the averaging of 50 RFI-free measurements.
Table 8.35: Edge Detection Algorithm performance (Pdet) on L-band RFI free radiometric data as a function of the Pfa|ED (Configuration data from Table 8.34).

Edge Detection Pdet

0.1 0.1124

0.05 0.0492

0.01 0.0083

0.005 0.0040

0.001 0.0008

Table 8.36: Error in the retrieved TA produced by the application of the Edge Detection algorithm on L-band RFI free radiometric data, as a function of the Pfa|ED (Configuration data from Table 8.34).

Edge Detection error in TA [K]

0.1 20,02

0.05 10,13

0.01 2,15

0.005 1,13

0.001 0,27

8.3.1.2 FIAT Algorithm behaviour with RFI-free radiometric data
This algorithm is also tested with the same L-band RFI-free data as the Edge Detection Algorithm is Section 8.3.1.1. The Pdet as a function of the Pfa for the L-band is represented in Table 8.37, and the error in the retrieved TA due to the radiometric data elimination is represented in Table 8.38.

Results are very similar as the obtained with the I/Q voltage data, described in Section 8.2.2.1.2. For the algorithm performance this case and the Section 8.2.2.1.2 case have almost the same results; however, for the error in the retrieved TA, in this case is a bit lower than in the Section 8.2.2.1.2 case but they are of the same order.
Table 8.37: FIAT Algorithm performance (Pdet) on L-band RFI free radiometric data as a function of the Pfa|FIAT.

0.1 Combined Frequency and Time thresholding Pdet Frequency thresholding Pdet Time thresholding Pdet

0.05

0.01

0.005

0.001

0.1044 0.0509 0.0617

0.0543 0.0253 0.0307

0.0126 0.0060 0.0068

0.0065 0.0033 0.0033

0.0014 0.0009 0.0005

212

RFI detection algorithms applied to radiometric data

Table 8.38: Error [K] in the retrieved TA produced by the application of the FIAT algorithm on L-band RFI free radiometric data, as a function of the Pfa|FIAT.

0.1 Combined Frequency and Time thresholding [K] Frequency thresholding [K] Time thresholding [K]

0.05

0.01

0.005

0.001

4,22 1,95 2,38

2,36 1,08 1,30

0,65 0,31 0,35

0,36 0,18 0,18

0,09 0,06 0,03

8.3.1.3 Behaviour of the combined Edge Detection and FIAT algorithms with RFI-free radiometric data
This section is similar to 8.2.2.1.3 where the performance of the combined RFI detection algorithms is evaluated. The Pdet and the error in the retrieved TA for this case are presented in Tables 8.39 and 8.40.

Table 8.39 represents the Pdet after applying the Edge Detection and the FIAT Algorithms to the RFI free data, having two parameters that affect to the Pdet, the Pfa|ED (columns), and the Pfa|FIAT (rows). As in Section 8.2.2.1.3, Pdet of the combined algorithms is not equal to any of the two Pfa‘s as in the two algorithms applied separately, as both algorithms eliminate different pieces of data.
Table 8.39: Performance of the combined Edge Detection and FIAT algorithms (Pdet) applied to L-band RFI-free radiometric data as a function of the Pfa of the Edge Detection algorithm (columns) and the Pfa of the FIAT Algorithm (rows).

Pfa|ED
0.1 0.05 0.01 0.005 0.001 0.1 0.1499 0.1271 0.1139 0.1131 0.1124 0.05 0.1107 0.0772 0.0538 0.0509 0.0496 0.01 0.1010 0.0550 0.0167 0.0126 0.0091 0.005 0.0994 0.0528 0.0143 0.0093 0.0051 0.001 0.1016 0.0528 0.0131 0.0068 0.0021

Pfa|FIAT

Table 8.40 represents the error in the retrieved TA due to the radiometric data elimination; and in this table is more clear how to tune both Pfa in order to have an acceptable error in the radiometric measurements in a RFI-free situation. Like in the rest

213

Chapter 8

of the cases, Pfa’s must be selected as the TA error is lower than our radiometric resolution aim.
Table 8.40: Total error in the retrieved TA produced by the application of the Edge Detection algorithm and the FIAT algorithm (Pdet) on L-band RFI free radiometric data as a function of the Pfa of the Edge Detection algorithm (columns) and the Pfa of the FIAT Algorithm (rows).

[K] 0.1 0.05 0.01 0.005 0.001

Pfa|ED
0.1 21.73 20.77 20.12 20.07 20.02 0.05 12.68 11.41 10.39 10.23 10.16 0.01 5.88 4.18 2.60 2.39 2.21 0.005 4.96 3.24 1.66 1.42 1.20 0.001 4.34 2.54 0.91 0.61 0.36

Pfa|FIAT

8.3.2 Algorithms using power spectrograms with RFI contaminated radiometric data
Now, the Edge Detection and FIAT algorithms are tested with real radiometric data in the presence of RFI.

8.3.2.1 Behaviour of the power spectrograms based algorithms with an entire L-band measurement
Figure 8.21 shows a measurement performed from 1.400 GHz to 1.427 GHz covering the whole protected L-band and presenting a high level of RFI contamination. The frequency resolution of this measurement is equal to 54 kHz, thus RFI frequency sharp tones present in the L-band can be more difficult to detect as they will be averaged with the thermal noise. However, the RFI is high as the measurements have been taken in the city of Barcelona.

Also, as in the spectrograms computed from I/Q data, the measured antenna TA in the absence of RFI was higher than the matched load Tph, (approximately TA = 315 K for a Tmatched
load

= 301 K), and also probably because the measurements have been

performed in the UPC RSLab, which is plenty of electronic devices.

214

RFI detection algorithms applied to radiometric data

In order to compare results obtained with the power spectrograms and the ones computed from the I/Q data, the Edge Detection Pfa (Pfa|ED) is selected to have a similar
TA error value as with the Smoothing algorithm (and not so different from the FIAT

algorithm). For Pfa|ED = 0.01, the retrieved TA error value is 2.15 K, which is quite similar to the 1.78 K for the Smoothing algorithm with a 2-D filter size of 31x31. Hence, Pfa|ED has been selected to have the value Pfa|ED = 0.01. On the other hand, Pfa|FIAT is set to the same value (Pfa|FIAT = 0.01).
1400 -30 1405 -40 Frequency [MHz] 1410 -50 -60 -70 1420 -80 1425 5 10 15 20 25 Time [s] 30 35 40 45 -90

1415

a)
-28 -33

-30 Power [dBm] Power [dBm] 1405 1410 1415 1420 Frequency [MHz] 1425

-33.5

-32

-34

-34

-34.5

-36 1400

-35

0

10

20 30 Time [s]

40

b)

c)

Figure 8.21: Data of the entire protected L-band (1.400-1.427 GHz) before the application of the RFI detection and mitigation algorithms, RBW = 100 kHz. a) Spectrogram of the data formed by 500 time points per 501 frequency points; b) Frequency domain average data (501 points); c) Time domain average data (500 points).

215

Chapter 8

1400

1400 -30

1405 Frequency [MHz] Frequency [MHz]

1405 -40

1410

1410

-50 -60 -70

1415

1415

1420

1420 -80

1425 5 10 15 20 25 Time [s] 30 35 40 45

1425 5 10 15 20 25 Time [s] 30 35 40 45

-90

a)
-28 -32 -32.5 -30 Power [dBm] Power [dBm] -33 -33.5 -34 -34.5 -36 1400 1405 1410 1415 1420 Frequency [MHz] 1425 -35 1400 1405

b)

-32

-34

1410 1415 1420 Frequency [MHz]

1425

c)
-33 -33

d)

-33.5 Power [dBm] Power [dBm] 0 10 20 30 Time [s] 40

-33.5

-34

-34

-34.5

-34.5

-35

-35

0

10

20 30 Time [s]

40

e)

f)

Figure 8.22: Results of the application of the Edge Detection algorithm with the parameter Pfa|ED = 0.01 to the entire L-band measurement, RBW = 100 kHz. Data before the application of the Edge Detection algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the Edge Detection algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

The result of the application of the Edge Detection, the FIAT, and both algorithms combined is shown in Figs. 8.22 (Edge Detecion), 8.23 (FIAT), and 8.24 (combined algorithms). In Table 8.42 the main values obtained by the application of the RFI detection algorithms are presented, in the same way as the previous sections.

216

RFI detection algorithms applied to radiometric data

1400

1400 -30

1405 Frequency [MHz] Frequency [MHz]

1405 -40

1410

1410

-50 -60 -70

1415

1415

1420

1420 -80

1425 5 10 15 20 25 Time [s] 30 35 40 45

1425 5 10 15 20 25 Time [s] 30 35 40 45

-90

a)
-28 -33.5

b)

-30 Power [dBm] Power [dBm] 1405 1410 1415 1420 Frequency [MHz] 1425 -34

-32

-34.5

-34

-36 1400

-35 1400

1405

1410 1415 1420 Frequency [MHz]

1425

c)
-33 -33

d)

-33.5 Power [dBm] Power [dBm] 0 10 20 30 Time [s] 40

-33.5

-34

-34

-34.5

-34.5

-35

-35

0

10

20 30 Time [s]

40

e)

f)

Figure 8.23: Results of the application of the FIAT algorithm with the parameter Pfa|FIAT = 0.01 to the entire L-band measurement, RBW = 100 kHz. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

217

Chapter 8
1400 1405 Frequency [MHz]
Frequency [MHz] 1400 1405

1400 -30 1405 Frequency [MHz] -40 1410 1415 1420 -80 1425
10 20 30 Time [s] 40

1410 1415 1420 1425 10 20 30 Time [s] 40

1410 1415 1420 1425

-50 -60 -70

-90 10 20 30 Time [s] 40

a)
-28
-31

b)
-33.5 -32 Power [dBm] Power [dBm] -34

c)

-30 Power [dBm]

-32

-33

-34.5

-34

-34

-36 1400

1405

1410 1415 1420 Frequency [MHz]

1425

-35 1400

1405

1410 1415 1420 Frequency [MHz]

1425

-35 1400

1405

1410 1415 1420 Frequency [MHz]

1425

d)
-33
-33

e)
-33 -33.5 Power [dBm] Power [dBm] -33.5

f)

-33.5 Power [dBm]

-34

-34

-34

-34.5

-34.5

-34.5

-35

0

10

20 30 Time [s]

40

-35

0

10

20 30 Time [s]

40

-35

0

10

20 30 Time [s]

40

g)

h)

i)

Figure 8.24: Results of the application of the Edge Detection and FIAT algorithms combined with the parameters Pfa|ED = 0.01 and Pfa|FIAT = 0.01 to the entire L-band measurement, RBW = 100 kHz. Data before the application of the Edge Detection and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the last iteration of the Edge Detection algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the last iteration of the Edge Detection and FIAT algorithm: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data. Table 8.41: Most relevant results obtained after the RFI detection algorithms application to the entire L-band measurement. The TA initially measured by the antenna is TA = 342.68 K

Algorithm ED FIAT ED and FIAT

Final TA [K]

329.46 318.7 315.92

initial TA − final TA [K] 13.22 23.98

flagged pixels [%] 2.38 10.16

∆T increase [%] 1.21 5.5

26.76

11.45

6.27

Observing Figs. 8.22 to 8.24, and Table 8.41, again the FIAT algorithm has the best performance. The frequency domain average data of the Edge Detection algorithm (Fig. 8.22d) reveals that although it eliminates the major part of the RFI, there is some residual RFI resulting in a poor behaviour of this algorithm.

218

RFI detection algorithms applied to radiometric data

On the other hand, the FIAT algorithm eliminates all tones, although the frequency response is not perfectly flat. The FIAT algorithm and joint Edge Detection and FIAT algorithms have a very similar performance, taking into account the difference between the errors due to false alarms. Therefore, it is more efficient to use only the FIAT algorithm, as the best RFI detection algorithm for the power spectrograms obtained from the spectrum analyzer.

8.3.2.2 Behaviour of the power spectrogram based algorithms with a 2 MHz sub-band of the L-band (3rd sub-band)
The next measurement comprises the 2 MHz sub-band centered at 1.404 GHz. The advantage of using a more stretch Span is having a higher frequency resolution (4 kHz). RFI present in the measurement is quite similar to the I/Q data case (Fig. 8.25). Same Pfa parameters have been selected for the application of the RFI detection algorithms in this measurement.

The result of the application of the different RFI detection algorithms can be consulted in the Figs. 8.26 (Edge Detection), 8.27 (FIAT) and 8.28 (combined algorithms) as in the previous section. Table 8.42 presents the main values obtained by the application of the RFI detection algorithms.

219

Chapter 8

1403

-30 -40

1403.5 Frequency [MHz] -50 -60 -70 1404.5 -80 -90 1405 5 10 15 20 25 Time [s] 30 35 40 45

1404

a)
-36 -38 Power [dBm] -40 -42 -44 -46 1403 Power [dBm] 1403.5 1404 1404.5 Frequency [MHz] 1405 -42.5

-43

-43.5

-44

-44.5

0

10

20 30 Time [s]

40

b)
rd

c)

Figure 8.25: Data of the 3 sub-band of the L-band measurement before the application of the RFI algorithms, RBW = 10 kHz. a) Spectrogram of the data formed by 500 time points per 501 frequency points. b) Frequency domain average data (501 points). c) Time domain average data (500 points).

220

RFI detection algorithms applied to radiometric data

1403 1403

-30 -40

1403.5 1403.5 Frequency [MHz] Frequency [MHz] -50 -60 -70 1404.5 1404.5 -90 1405 1405 5 10 15 20 25 Time [s] 30 35 40 45 5 10 15 20 25 Time [s] 30 35 40 45 -80

1404

1404

a)
-36 -38 Power [dBm] -40 -42 -44 -46 1403 Power [dBm] -41

b)

-42

-43

-44

1403.5 1404 1404.5 Frequency [MHz]

1405

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

c)
-42.5 -42.5

d)

-43 Power [dBm] Power [dBm] 0 10 20 30 Time [s] 40

-43

-43.5

-43.5

-44

-44

-44.5

-44.5

0

10

20 30 Time [s]

40

e)

f)

Figure 8.26: Results of the application of the Edge Detection algorithm with the parameter Pfa|ED = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10 kHz. Data before the application of the Edge Detection algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the Edge Detection algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

221

Chapter 8

1403

1403

-30 -40

1403.5 Frequency [MHz] Frequency [MHz]

1403.5 -50 -60 -70 1404.5 -80 -90

1404

1404

1404.5

1405

5

10

15

20 25 Time [s]

30

35

40

45

1405

5

10

15

20 25 Time [s]

30

35

40

45

a)
-36 -38 Power [dBm] -40 -42 -44 -46 1403 Power [dBm] -43

b)

-43.5

-44

-44.5

1403.5 1404 1404.5 Frequency [MHz]

1405

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

c)
-42.5 -43

d)

-43 Power [dBm] Power [dBm] 0 10 20 30 Time [s] 40

-43.5

-43.5

-44

-44

-44.5

-44.5

-45

0

10

20 30 Time [s]

40

e)

f)

Figure 8.27: Results of the application of the FIAT algorithm with the parameter Pfa|FIAT = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10 kHz. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

222

RFI detection algorithms applied to radiometric data
1403
1403

1403

-30 -40

Frequency [MHz]

Frequency [MHz]

Frequency [MHz]

1403.5

1403.5

1403.5 -50 1404 -60 -70 1404.5 -80 -90

1404

1404

1404.5

1404.5

1405

10

20 30 Time [s]

40

1405

10

20 30 Time [s]

40

1405

10

20 30 Time [s]

40

a)
-36 -38 Power [dBm]
Power [dBm] -41 -42

b)
-43 -43.5 Power [dBm]

c)

-40 -42 -44 -46 1403

-43

-44

-44

-44.5

1403.5 1404 1404.5 Frequency [MHz]

1405

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

-45 1403

1403.5 1404 1404.5 Frequency [MHz]

1405

d)
-42.5
-42.5

e)
-43 -43 Power [dBm] Power [dBm] -43.5

f)

-43 Power [dBm]

-43.5

-43.5

-44

-44

-44

-44.5

-44.5

0

10

20 30 Time [s]

40

-44.5

0

10

20 30 Time [s]

40

-45

0

10

20 30 Time [s]

40

g)

h)

i)

Figure 8.28: Results of the application of the Edge Detection and FIAT algorithms cobined with the parameters Pfa|ED = 0.01 and Pfa|FIAT = 0.01 to the 3rd sub-band of the L-band measurement, RBW = 10 kHz. Data before the application of the Edge Detection and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the last iteration of the Edge Detection algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the last iteration of the Edge Detection and FIAT algorithm: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data. Table 8.42: Most relevant results obtained after the RFI detection algorithms application to the 3rd sub-band of the L-band measurement. The TA initially measured by the antenna is TA = 375.55 K

Algorithm ED FIAT ED and FIAT

Final TA [K]

352.35 315.82 313.89

initial TA − final TA [K] 23.2 59.72

flagged pixels [%] 3.6 25.9

∆T increase [%] 1.85 16.17

61.65

26.32

16.5

223

Chapter 8

8.3.2.3 Behaviour of the power spectrogram based algorithms over an entire L-band measurement with a long sweep time
Another measurement which covers the entire protected L-band is performed (Fig. 8.29), but in this case with a longer sweep time (Table 8.34) to have a larger fraction of the time measuring vs. the time the spectrum analyzer waits for the next measurement (~ 41 ms), thus increasing the efficiency. This measurement has a similar level of RFI than the 1st measurement studied in the Section 8.3.2.1, although, more pixels are flagged as several RFI tones are present between 1.400 and 1.410 GHz. It also presents similar final TA values.
1400

1405 Frequency [MHz]

-30

1410

-40

-50 1415 -60 1420 -70 1425 50 100 150 Time [s] 200 250 -80

a)
-29 -30 -33.5 Power [dBm] -31 -32 -33 -34 -35 1400 1405 1410 1415 1420 Frequency [MHz] 1425 -35 0 50 100 150 Time [s] 200 250 Power [dBm] -33

-34

-34.5

b)

c)

Figure 8.29: Data of the entire L-band measurement before RFI algorithms application, RBW = 100 kHz. a) Spectrogram of the data formed by 500 time points per 501 frequency points. b) Frequency domain average data (501 points). c) Time domain average data (500 points).

However, the time duration of the entire measurement is a bit long, 275 s. vs. 47.5 s. of the 1st measurement, thus temperature drifts will affect more. These drifts are 224

RFI detection algorithms applied to radiometric data

noticeably when comparing Fig. 8.21f (belonging to the 1st measurement after RFI elimination) with Fig. 8.29f (belonging to the 3rd measurement after RFI elimination), where these drifts present in the retrieved power can be observed in the 3rd measurement, which are not present in the 1st measurement. On the other hand, in an airborne or space-borne instrument, radiometric measurements will have changed during 47.5 s. or 5 min., so this problem will affect in the same way for both time durations.

Finally, the results (Figs. 8.30 to 8.32, and Table 8.43) are similar to the two previous measurements, and the FIAT algorithm has the best performance again. This way, it is strongly recommended to use the FIAT algorithm despite of its higher number of eliminated pixels compared to the rest of the algorithms. In fact, as many RFI consists of tones that are present for long periods of time, or short time duration broadband RFI, RFI flagged pixels usually contain RFI indeed.

225

Chapter 8

1400

1400

1405 Frequency [MHz] Frequency [MHz]

1405

-30

1410

1410

-40

-50 1415 -60 1420 -70

1415

1420

1425 50 100 150 Time [s] 200 250

1425 50 100 150 Time [s] 200 250 -80

a)
-29 -30

b)
-31

-32
Power [dBm] -31 -32 -33 -34 -35 1400 1405 1410 1415 1420 Frequency [MHz] 1425

Power [dBm]

-33

-34

-35 1400

1405

1410 1415 1420 Frequency [MHz]

1425

c)
-33

d)
-33

-33.5 Power [dBm]

-33.5 Power [dBm]
0 50 100 150 Time [s] 200 250

-34

-34

-34.5

-34.5

-35

-35

0

50

100 150 Time [s]

200

250

e)

f)

Figure 8.30: Results of the application of the Edge Detection algorithm with the parameter Pfa|ED = 0.01 to the entire L-band measurement, RBW = 100 kHz. Data before the application of the Edge Detection algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the Edge Detection algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

226

RFI detection algorithms applied to radiometric data

1400

1400

1405 Frequency [MHz] Frequency [MHz]

1405

-30

1410

1410

-40

-50 1415 -60 1420 -70

1415

1420

1425 50 100 150 Time [s] 200 250

1425 50 100 150 Time [s] 200 250 -80

a)
-29 -30 Power [dBm] -31 -32 -33 -34 -35 1400 1405 1410 1415 1420 Frequency [MHz] 1425 -35 1400 1405 Power [dBm] -34 -33.5

b)

-34.5

1410 1415 1420 Frequency [MHz]

1425

c)
-33 -33.6 -33.8 Power [dBm] -34 -34.2 -34.4 -34.6

d)

-33.5 Power [dBm]

-34

-34.5

-35

0

50

100 150 Time [s]

200

250

0

50

100 150 Time [s]

200

250

e)

f)

Figure 8.31: Results of the application of the FIAT algorithm with the parameter Pfa|FIAT = 0.01 to the entire L-band measuremen, RBW = 100 kHz t. Data before the application of the FIAT algorithm: a) Spectrogram; c) Frequency domain average data; e) Time domain average data. Resulting data after the application of the FIAT algorithm: b) Spectrogram with flagged pixels detected as RFI; d) Frequency domain average data; f) Time domain average data.

227

Chapter 8
1400 1405

1400 1405 Frequency [MHz]

1400 1405 Frequency [MHz] 1410 1415 1420 -70 1425
50 100 150 Time [s] 200 250

-30 -40 -50 -60

Frequency [MHz]

1410 1415 1420 1425 50 100 150 Time [s] 200 250

1410 1415 1420 1425

50

100 150 Time [s]

200

250

-80

a)
-29 -30 -32 Power [dBm] Power [dBm] -31 -32 -33 -34 -35 1400 1405 1410 1415 1420 Frequency [MHz] 1425 -35 1400 1405 -31

b)
-33.5 Power [dBm]

c)

-34

-33

-34.5

-34

1410 1415 1420 Frequency [MHz]

1425

-35 1400

1405

1410 1415 1420 Frequency [MHz]

1425

d)
-33 -33 -33.5 Power [dBm] Power [dBm] -33.5

e)
-33 -33.5 Power [dBm]

f)

-34

-34

-34

-34.5

-34.5

-34.5

-35

0

50

100 150 Time [s]

200

250

-35

0

50

100 150 Time [s]

200

250

-35

0

50

100 150 Time [s]

200

250

g)

h)

i)

Figure 8.32: Results of the application of the Edge Detection and FIAT algorithms combined with the parameters Pfa|ED = 0.01 and Pfa|FIAT = 0.01 to the entire L-band measurement, RBW = 100 kHz. Data before the application of the Edge Detection and FIAT algorithms: a) Spectrogram; d) Frequency domain average data; g) Time domain average data. Resulting data after the application of the last iteration of the Edge Detection algorithm: b) Spectrogram with flagged pixels detected as RFI; e) Frequency domain average data; h) Time domain average data. Resulting data after the application of the last iteration of the Edge Detection and FIAT algorithm: c) Spectrogram with flagged pixels detected as RFI; f) Frequency domain average data; i) Time domain average data. Table 8.43: Most relevant results obtained after the RFI detection algorithms application to the entire L-band measurement. The TA initially measured by the antenna is TA = 344.23 K

Algorithm ED FIAT ED and FIAT

Final TA [K]

333.92 319.58 316.69

initial TA − final TA [K] 10.31 24.64

flagged pixels [%] 2.34 15.71

∆T increase [%] 1.19 8.92

27.53

16.57

9.48

228

RFI detection algorithms applied to radiometric data

8.4

Optimal

Pfa

for

the

FIAT

algorithm

performance
As it has been observed in Sections 8.3 and 8.2.2, the FIAT algorithm has the best performance among the different RFI detection algorithms for spectrograms. It is the recommended algorithm for future works regarding RFI detection and mitigation. However, the Pfa|FIAT which has been taken as a constant value (Pfa|FIAT = 0.01) may be modified in order to observe the variations introduced in the detection of RFI.

Consequently, four different values of this parameter have been used to observe the TA after RFI elimination, and compare these results with the values of TA error in RFI free measurements. This way, assuming errors present with RFI free data, the TA in case of RFI present could be more accurate.

As the analysis of the different spectrogram based RFI detection algorithms is differentiated between the spectrograms computed from I/Q data and those from power spectrograms, two Tables analyzing data from both cases are computed.

In the first table (Table 8.44), the TA after the RFI detection algorithm application, the proportion of increase in the radiometric resolution, and the TA error with RFI free data, for 4 different values of Pfa|FIAT between 0.005 and 0.05, and for the three different measures studied in the Section 8.2.2 is presented. Higher values of Pfa|FIAT have not been taken into account because of the recurrence of the FIAT algorithm, which can lead to errors in the TA produced by false alarms and the Pfa|FIAT value too. On the other hand, the TA error produced by false alarms has been recalculated (RFI free TA error column in Tables 8.26 and 8.36) taking into account recurrence of the FIAT algorithm; in fact, TA increases a 20% for a Pfa|FIAT = 0.05, but for lower values of Pfa|FIAT this increase is negligible.

229

Chapter 8

Table 8.44: Most relevant results obtained after the application of the FIAT algorithm to three 2 MHz sub-bands of the L-band measurement previously analyzed in Section 8.2.2.2 for different values of the Pfa|FIAT value.

Pfa|FIAT 3rd sub-band fc = 1.404 GHz BW = 2MHz Initial TA = 799.05 K 5th sub-band fc = 1.408 GHz BW = 2MHz Initial TA = 327.82K 10th sub-band fc = 1.418 GHz BW = 2MHz Initial TA = 390.75 K

Final TA [K]

0.05 0.02 0.01 0.005 0.05 0.02 0.01 0.005 0.05 0.02 0.01 0.005

364.65 368.5 369.8 370.62 323.61 325.51 326.47 326.6 348.49 353 355.19 355.88

RFI free TA error [K] 3,86 1.77 0,98 0,54 3,86 1.77 0,98 0,54 3,86 1.77 0,98 0,54

∆T increase [%]

30.08 25.71 24.37 23.63 3.67 1.59 0.69 0.59 10.73 5.99 4.12 3.59

Differences between Final TA values for different Pfa|FIAT values on the 5th subband (Table 8.44) are equal than differences between RFI free TA, because with a Pfa|FIAT = 0.005 all RFI can be eliminated. On the other hand, in the other two cases (3rd and 10th sub-bands), important differences are observed for Pfa|FIAT = 0.05 respect the rest of Pfa|FIAT values, (approximately 4 K difference between Pfa|FIAT = 0.05 and Pfa|FIAT = 0.02 when in RFI free difference is 2 K), this fact indicates that more RFI is detected when Pfa|FIAT value is increased. However it must be taken into account that data clipping produced by the false alarm elimination decreases the final TA value too, due to the fact that the time/frequency segments with larger power values will be eliminated and the ones with lower power values will remain. On the other hand, it must be taken into account that 3.86 K can be a very high value for determined applications. Figures 8.33 and 8.34 present the frequency response of the 3rd and 10th subband measurements respectively for different Pfa|FIAT values in order to observe the impact of the selection of this parameter. In the case of the 3rd sub-band, (Fig. 8.33) the borders of the main RFI component between 1403.4 MHz and 1404.2 MHz tend to be completely eliminated as the Pfa|FIAT value increases, additionally increasing the elimination of higher power components of the thermal noise; however, possible RFI contribution located between 1404.6 MHz and 1404.8 MHz could not be detected as the frecuency response is not perfectly flat. The main conclusion obtained from Table 8.44 and Fig. 8.33 for the 3rd sub-band is that while the Pfa|FIAT value increases, more low

230

RFI detection algorithms applied to radiometric data

level RFI components are eliminated, however, almost same RFI components are eliminated with a Pfa|FIAT = 0.02 (Fig. 8.33d) and Pfa|FIAT = 0.05 (Fig. 8.33e); so it is better to use Pfa|FIAT = 0.02, since better results are obtained in RFI free conditions. Figure 8.34 represents the 10th sub-band of the L-band, and shows the presence of several RFI contributions with a low level value, being eliminated as Pfa|FIAT value increases. For values higher than Pfa|FIAT = 0.01 (Figs. 8.34c, 8.34d and 8.34e) seems that all RFI components are eliminated, although it is very difficult to differentiate a small RFI tone from a high power value of thermal noise.

From Figs. 8.33 and 8.34, and Table 8.44 it can be concluded that the appropriate Pfa|FIAT value depends on the RFI present in the signal. Therefore, Pfa|FIAT value can only be selected by the TA error that introduces in the RFI free measurements, as stated in Chapters 4 and 5.

231

Chapter 8

-40.5 Power [dBm]

-41

-41.5

-42 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

a)
-40.5 Power [dBm]

-41

-41.5

-42 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

b)
-40.5 Power [dBm]

-41

-41.5

-42 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

c)
-40.5 Power [dBm]

-41

-41.5

-42 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

d)
-40.5 Power [dBm]

-41

-41.5

-42 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

e)
Figure 8.33: ADC based frequency domain average data after the application of the FIAT algorithm on the 3rd sub-band of the L-band (1.404 GHz) for different Pfa|FIAT values, RBW = 10 MHz: a) Pfa|FIAT = 0 (before the application of the FIAT algorithm); b) Pfa|FIAT = 0.005; c) Pfa|FIAT = 0.01; d) Pfa|FIAT = 0.02; e) Pfa|FIAT = 0.05.

232

RFI detection algorithms applied to radiometric data

-40.5 Power [dBm]

-41

-41.5

-42 1417

1417.2

1417.4

1417.6

1417.8 1418 1418.2 Frequency [MHz]

1418.4

1418.6

1418.8

1419

a)
-40.5 Power [dBm]

-41

-41.5

-42 1417

1417.2

1417.4

1417.6

1417.8 1418 1418.2 Frequency [MHz]

1418.4

1418.6

1418.8

1419

b)
-40.5 Power [dBm]

-41

-41.5

-42 1417

1417.2

1417.4

1417.6

1417.8 1418 1418.2 Frequency [MHz]

1418.4

1418.6

1418.8

1419

c)
-40.5 Power [dBm]

-41

-41.5

-42 1417

1417.2

1417.4

1417.6

1417.8 1418 1418.2 Frequency [MHz]

1418.4

1418.6

1418.8

1419

d)
-40.5 Power [dBm]

-41

-41.5

-42 1417

1417.2

1417.4

1417.6

1417.8 1418 1418.2 Frequency [MHz]

1418.4

1418.6

1418.8

1419

e)
Figure 8.34: ADC based frequency domain average data after the application of the FIAT algorithm on the 10th sub-band of the L-band (1.418 GHz) for different Pfa|FIAT values, RBW = 10 MHz: a) Pfa|FIAT = 0 (before the application of the FIAT algorithm), b) Pfa|FIAT = 0.005; c) Pfa|FIAT = 0.01; d) Pfa|FIAT = 0.02; e) Pfa|FIAT = 0.05.

233

Chapter 8

Table 8.45 is based in the spectrum analyzer data, and it has the same structure as in the previous Table, but with the three different measurements used in Section 8.3.
Table 8.45: Most relevant results obtained after the application of the FIAT algorithm to three spectrum analyzer measurements previously analyzed in Section 8.3 for different values of the Pfa|FIAT value.

Pfa|FIAT 1st Measurement BW = 27MHz Sweep time = 50ms Initial TA = 342.68 K 2nd Measurement BW = 2MHz Sweep time = 50ms Initial TA = 375.55 K 3rd Measurement BW = 27MHz Sweep time = 500ms Initial TA = 344.23 K

Final TA [K]

0.05 0.02 0.01 0.005 0.05 0.02 0.01 0.005 0.05 0.02 0.01 0.005

315.58 317.38 318.7 319.14 312.03 314.38 315.82 316.08 314.49 318.21 319.58 320.27

RFI free TA error [K] 2,5 1.04 0,59 0,34 2,5 1.04 0,59 0,34 2,5 1.04 0,59 0,34

∆T increase [%]

9.42 7.06 5.5 5.04 21.53 18 16.17 15.86 15.84 10.61 8.92 8.16

In this case, differences between Final TA values for different Pfa|FIAT for the three measurements (Table 8.45) are very similar between them. Again, using Pfa|FIAT = 0.05 will eliminate more RFI than the rest, having a higher decrease in the final TA value than the one produced by the false alarms only. Comparing Tables 8.44 and 8.45, RFI free TA error is lower for the same Pfa|FIAT value in the Spectrum analyzer measurements than in the spectrogram computed from I/Q measurements. Hence, taking measurements with the spectrum analyzer has an additional advantage. Figures 8.35 and 8.36 present the frequency response of the 1st and 2nd measurements respectively in order to observe the impact of the Pfa|FIAT value chosen, in the same way as Figs. 8.33 and 8.34. In Fig. 8.35 high values of Pfa|FIAT eliminate low level of RFI adjacent to the main RFI contributions at the edges of the protected band (1.400 GHz and 1.427 GHz); but on the other hand, the number of false alarms increases. Again, the main RFI peaks are eliminated with a Pfa|FIAT of 0.01, and higher values eliminate equally RFI peaks, and thermal noise peaks.

Figure 8.36 represents the same 2 MHz band as Fig. 8.33, and results are very similar, where low level RFI components between 1403.4 MHz and 1404.2 MHz tend

234

RFI detection algorithms applied to radiometric data

to be completely eliminated as the Pfa|FIAT value increases, additionally increasing the removal of higher power components of the thermal noise. In this case, the Pfa|FIAT value that best fits is the highest, Pfa|FIAT = 0.05, as this value eliminates the largest part of the low level RFI components at both sides of the main RFI (1403.4 MHz and 1404.2 MHz).

Again, the conclusion that can be extracted from Figs. 8.35 and 8.36, and Table 8.45, is that the best Pfa|FIAT value will depend on the RFI level present on the measurement, so, this value must be selected depending on the TA error that introduce in RFI free measurements. However, in this case the RFI-free TA error for Pfa|FIAT = 0.02 for spectrum analyzer data is almost equal than the RFI-free TA error for Pfa|FIAT = 0.01 for the spectrograms computed from I/Q data, both around 1 K.

235

Chapter 8

-33.6 -33.8 Power [dBm] -34 -34.2 -34.4 -34.6 1400 1405 1410 1415 Frequency [MHz] 1420 1425

a)
-33.6 -33.8 Power [dBm] -34 -34.2 -34.4 -34.6 1400 1405 1410 1415 Frequency [MHz] 1420 1425

b)
-33.6 -33.8 Power [dBm] -34 -34.2 -34.4 -34.6 1400 1405 1410 1415 Frequency [MHz] 1420 1425

c)
-33.6 -33.8 Power [dBm] -34 -34.2 -34.4 -34.6 1400 1405 1410 1415 Frequency [MHz] 1420 1425

d)
-33.6 -33.8 Power [dBm] -34 -34.2 -34.4 -34.6 1400 1405 1410 1415 Frequency [MHz] 1420 1425

e)
Figure 8.35: Spectrum analyzer based frequency domain average data after the application of the FIAT algorithm on the data obtained from the 1st configuration (Table 8.45) for different Pfa|FIAT values, RBW = 10 kHz: a) Pfa|FIAT = 0 (before the application of the FIAT algorithm); b) Pfa|FIAT = 0.005; c) Pfa|FIAT = 0.01; d) Pfa|FIAT = 0.02; e) Pfa|FIAT = 0.05.

236

RFI detection algorithms applied to radiometric data
-43.5

Power [dBm]

-44

-44.5 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

a)
-43.5

Power [dBm]

-44

-44.5 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

b)
-43.5

Power [dBm]

-44

-44.5 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

c)
-43.5

Power [dBm]

-44

-44.5 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

d)
-43.5

Power [dBm]

-44

-44.5 1403

1403.2

1403.4

1403.6

1403.8 1404 1404.2 Frequency [MHz]

1404.4

1404.6

1404.8

1405

e)
Figure 8.36: Spectrum analyzer based frequency domain average data after the application of the FIAT algorithm on the data obtained from the 2nd configuration (Table 8.45) for different Pfa|FIAT values, RBW = 10 kHz: a) Pfa|FIAT = 0 (before the application of the FIAT algorithm); b) Pfa|FIAT = 0.005; c) Pfa|FIAT = 0.01; d) Pfa|FIAT = 0.02; e) Pfa|FIAT = 0.05.

237

Chapter 8

8.5 Conclusions
Different RFI detection algorithms have been tested and results obtained.

From all the 10 normality test-based algorithms only 7 work properly, the Kurtosis-based algorithms (K, JB and K2), and the ECDF-based algorithms (AD, CM, SW and L). The CHI2 test had a Pdet in case of RFI contaminated signal noticeably lower than the previous seven algorithms; and the LM and S tests had a Pdet = Pfa in case of RFI contaminated signal; thus these three normality test-based algorithms are not suitable for RFI detection.

Since the algorithms’ performance from these two groups (Kurtosis-based and ECDF-based algorithms) is similar, the best algorithm of each of these two groups is enough to properly detect RFI by means of normality analysis. Hence the Kurtosis (K) test and the Anderson Darling (AD) or the Shapiro-Wilk (SW) tests are the most appropriate normality tests algorithms. In addition, all the normality tests perform better for data segments with a larger number of samples, hence the number of samples of 216 is recommended. For this case, the AD test works better than the SW test.

Kurtosis is simpler than the AD and test, as it does not require sorting the sampled data, thus it should be the first option. However, the best option would be a combination of both algorithms, as each one can detect RFI that cannot be detected by the other one.

However, normality tests eliminate all the data analyzed if RFI is detected, and let all the RFI present if they fail to detect it. On the other hand, the spectrogram based algorithms converts the radiometric data in STFT components (pixels of the spectrogram), detect and eliminate RFI contaminated pixels present in the spectrograms but leaving RFI-free pixels apart.

In addition, data used in spectrogram-based algorithms can be obtained by sampling process like the normality tests or by stacking temporal power traces obtained by a spectrum analyzer, which gives versatility to this kind of algorithms.

238

RFI detection algorithms applied to radiometric data

Three different spectrogram-based algorithms have been compared, the Smoothing and the FIAT algorithms for I/Q sampled data, and the FIAT and the Edge Detection algorithms for the spectrum analyzer data. In both cases the algorithm with the best performance is the FIAT one, so this algorithm is recommended to be used in front of the other two. In addition, the FIAT algorithm is the simplest one to be implemented, so for FPGA-based RFI detection systems this is the most suitable algorithm.

Among the two ways to obtain the data, I/Q sampled data has the advantage of letting us apply normality tests and spectrogram-based algorithms, while the temporal and frequency resolutions are much greater than the spectrum analyzer data. On the other hand, the spectrum analyzer data can cover a wider frequency span and time lapses, and these ones can be easily configured. In addition, the spectrogram-based data has the issue that a ~41 ms delay is introduced after each temporal trace. The error in the TA produced by false alarms is lower for spectrum analyzer based data for the same Pfa value. Summarizing, I/Q sampling is the recommended option. However, if the 41 ms delay could be eliminated, the option of using directly the spectrum analyzer will be quite interesting due to its fully configurability.

The best probability of false alarm value has been found depending on the amount of RFI present in the measurement, so there is not an optimal probability of false alarm value however values of Pfa = 0.01 for spectrograms computed from I/Q data and Pfa = 0.02 for spectrum analyzer based data give in general good results. A final conclusion of this Chapter is that The FIAT algorithm is the best RFI detection and mitigation algorithm as it can be used with the two configuration of the MERITXELL radiometer, gives the best results for the lower Pfa values, and it is the most computationally simple of the spectrogram-based algorithms. However, if sampled data is available, FIAT algorithm can be combined with normality tests (Kurtosis and Anderson Darling). This way, K and AD tests can be applied to the radiometric data

239

Chapter 8

with high Pfa values (Pfa ≥ 0.1), and those data segments that fail the test could be analyzed by the FIAT algorithm, having a more efficient algorithm.

240

Conclusions and future research lines

Chapter 9
9. Conclusions and future research lines

Conclusions obtained from this Ph. D. and future research lines are sumarized in this Chapter.

241

Chapter 9

9.1 Conclusions and Summary
The present Ph. D. Thesis is a contribution to the RFI detection and mitigation in microwave radiometry Several RFI detection algorithms have been developed, that can be classified in different groups based on the approach: normality tests, time/frequency domains, and wavelet domain. These algorithms have been tested with real radiometric data obtained with a dual-polarized multi-band radiometer (MERITXELL) developed in the framework of this Ph. D Thesis for this purpose. This thesis is divided in three parts.

9.1.1 Background presentation
The first part comprises the Chapters 1 and 2 and it is related to the theoretical background and review of the state of the art. First of all, the microwave radiometry theory is introduced summarizing the most important concepts related to microwave radiometry and emission theory fields, in order to present the basics to the reader, including the main applications and the most common radiometer configurations.

Then, the problems created by RFI present in radiometric measurements are described, and several current radiometry missions with RFI problems are presented.

9.1.2 RFI detection algorithms
The second part first presents a summary of the state-of-the-art algorithms for RFI detection (Chapter 3). Then, the different RFI detection algorithms studied and developed in this thesis are presented. Furthermore, these RFI detection algorithms are classified in three different groups, depending on the approach used to discriminate between RFI and thermal noise.

The first group embraces the normality test based algorithms, which are based in the fact that the sampled thermal noise follows a Gaussian distribution, and usually, man-made RFI does not follow this type of distribution. Thus, a way to detect the

242

Conclusions and future research lines

presence of RFI consists of applying normality tests to this data set. If RFI is detected, the whole set is eliminated. In this Ph. D. thesis 10 normality tests have been reviewed, described and tested, concluding that the best normality tests are the Kurtosis test which usually has the better performance among all the others, and the Anderson-Darling test, which has a better performance than the Kurtosis test in case of some RFI signals with determined duty cycles (e.g. sinusoidal signals with 50% duty cycle).

The elimination of all the samples of a data set can be too restrictive if the RFI is not present in all the samples. On the other hand, normality tests cannot be applied to the power samples of the radiometric data (which follow an exponential distribution, not a Gaussian one). The second group of RFI detection algorithms are based on the detection of high power values of the radiometric signal in the time-frequency domains, thus sampled voltage data is not required.

This second group is based on the calculation of the spectrogram of the radiometric signal, which can be obtained by means of the Short Time Fourier Transform of the sampled data, or by capturing data from a spectrum analyzer. Another advantage of the spectrogram based RFI detection algorithms is that they can discriminate more deeply RFI from thermal noise, as RFI usually appears as a cluster of abnormal high power values in the time-frequency domain which can be more easily detected and discriminated as RFI.

The third group of RFI detection algorithms is based on denoising the interfering signals in order to substract them from the sampled thermal noise leaving the thermal noise signal. With this technique, the RFI signals must be very well estimated, since errors in the estimation will lead into errors in the resulting thermal noise. This group of algorithms is based in the wavelet decomposition to obtain the main components of the RFI signal, and the truncation of the components series at a given threshold.

These three different groups of RFI detection algorithms are extensively described in Chapters 4, 5 and 6, respectively.

243

Chapter 9

9.1.3 Experimental results
In the frame of this Ph. D. Thesis, a dual-polarized multi-band radiometer has been designed and implemented (Chapter 7). This radiometer covers eight protected bands usually used in Earth remote sensing and houses three cameras in order to obtain data in the near and thermal infrared and optical range. In addition, a GPS reflectometer has been included.

This hardware has been used to obtain the radiometric data used to test the RFI detection algorithms described in the Chapters 4 and 5, as this radiometer has a spectrum analyzer as a back-end which can create spectrograms with the received data and can also sample the I/Q components of the signal. Only one back-end is needed with this configuration, hence, the implementation of the multi-band radiometer is simpler.

However, this configuration also has some drawbacks as: • Sampling process of the radiometric data only permits to sample 65536 samples in a row, which is enough to apply RFI detection algorithms, but is a bit poor for obtaining the radiometric resolution, as the ADC stops sampling data every time that performs a new measurement. •

Distance between the different front-ends of every band and the spectrum

analyzer can increase the attenuation in the radiometric signal in the high frequency (> 20 GHz) bands degrading the radiometric resolution of these bands. Moreover, the spectrum analyzer’s internal noise increases with frequency. • A ~41 ms delay exists between successive temporal traces, that decreases the efficiency of the radiometer. This efficiency can be enhanced by increasing the number of temporal traces, although if the time is too long, the system may have drifted.

These drawbacks could be addressed by using one ADC and FPGA module per band and polarization, but this configuration will also require additional filters, mixers

244

Conclusions and future research lines

and local oscillators. Besides, the measurements of the highest frequency bands cannot be performed for the whole bandwidth, as it would require sampling frequencies of the order of GHz. On the other hand, the spectrum analyzer configuration permits to construct the multiband radiometer, and even apply RFI detection algorithms to its data.

Real radiometric data acquired with the MERITXELL instrument has been used in order to test the normality tests and the spectrogram based RFI detection algorithms, (Chapter 8).

Normality tests results with real data are quite similar to the results obtained with simulated data: kurtosis and ECDF-based tests perform similar, and the best tests of both groups are the Kurtosis and the AD tests. Tests perform better with long segments of data, and with higher probabilities of false alarm, however, these requirements increase the number of RFI-free samples being eliminated, either due to RFI-free segments eliminated as false alarms, and the methodology of this algorithm itself, which a segment must be entirely eliminated if a RFI is detected.

On the other hand, among the spectrogram-based algorithms the FIAT algorithms is the most efficient in the RFI detection and elimination followed by the Smoothing algorithm, and the FIAT algorithm is the simplest one from the computational point of view. However, this algorithm eliminates the entire frequency or time segment even if a RFI does not occupy the whole frequency or time segment. Nevertheless, the FIAT algorithm is the recommended one to be used, as it is preferable to eliminate a larger number of time-frequency bins than leaving corrupted ones, to obtain a more accurate brightness temperature. The probability of false alarm used in the spectrogram-based algorithms must be low: lower than 0.05 for the FIAT algorithm, or even lower for the rest of the spectrogram based algorithms, in order to avoid errors in the estimated antenna temperature. These errors are caused by the thresholding process as high power values are eliminated while low power ones remains, leading to a decrease in the mean value of the brightness temperature.

Finally, the best RFI detection and mitigation algorithm studied in this Ph. D. Thesis is the FIAT algorithm as it can be used with both hardware configuration of our 245

Chapter 9

MERITXELL radiometer, and gives the best results for the lower Pfa values. However, if sampled data is available, FIAT algorithm can be combined with the Kurtosis and Anderson Darling tests. This way, K and AD tests can be applied to the radiometric data with high Pfa values (Pfa ≥ 0.1), and those data segments that fail the test could be analyzed by the FIAT algorithm, having a more efficient algorithm.

9.2 Future research lines
In the moment that this Ph. D. is concluded, some problems have been encountered. Therefore future research lines could be defined in order that other Ph. D. students can research in this field. •

VHDL implementation of the FIAT algorithm with I/Q sampled data in order to have a hardware based RFI detection algorithm. This system should form part of the front-end of a narrowband radiometer in order to try to obtain a real-time RFI detection and mitigation algorithm that will reduce the amount of data to be saved.

•

A combination between normality tests (Kurtosis test, AD test or both), and spectrogram-based algorithms (FIAT algorithm) should be developed. A first step detection performed by a combination of Kurtosis and AD tests with a high probability of false alarm (0.1 or higher) in order to detect very low RFI values in data segments. A next step, applying the FIAT algorithm to segments that have not passed the tests, composed by RFI contaminated segments and false alarms.

•

Wavelet-based algorithms described in Chapter 6 should be tested with real data. A RSLab Ph. D. student is working in this line at the present time.

• •

Installation of the MERITXELL radiometer in the truck.

Development of data fusion algorithms to combine radiometric data and data obtained from the three cameras housed in the MERITXELL.

246

Conclusions and future research lines

•

Performing field experiments alone (e.g. RFI surveys) or in conjunction with the PAU-SA radiometer in order to take the full potential of the MERITXELL radiometer (8 bands, 3 cameras and a GPS reflectometer).

9.3 List of publications
Publications developed during this thesis are detailed in this section. Publications are sorted in function of the participation, classified in two groups: journal articles and conference proceedings.

9.3.1 Journal articles
• J. M. Tarongi and A. Camps "Normality Analysis for RFI Detection in

Microwave Radiometry ", Remote Sens., Vol. 2(1), pp. 191-210, Jan. 2010. •

J. M. Tarongi and A. Camps "Radio Frequency Interference Detection

and Mitigation Algorithms Based on Spectrogram Analysis", Algorithms, Vol. 4(4), pp. 239-261, Oct. 2011. •

A. Camps and J. M. Tarongi, "RFI Mitigation in Microwave

Radiometry Using Wavelets", Algorithms, Vol. 2(3) pp. 1248-1262, Sep. 2009. •

A. Camps, X. Bosch-Lluis, I. Ramos-Perez, J. F. Marchán-Hernández,

N. Rodríguez, E. Valencia, J. M. Tarongi, A. Aguasca and R. Acevo, "New Passive Instruments Developed for Ocean Monitoring at the Remote Sensing Lab — Universitat Politècnica de Catalunya", Sensors, Vol. 9(12) pp. 1017110189, Dec. 2009. •

A. Camps and J. M. Tarongí, "Microwave Radiometer Resolution

Optimization Using Variable Observation Times", Remote Sens., Vol. 2(7) pp. 1826-1843, Jul. 2010.

247

Chapter 9

•

A. Camps, J. Gourrion, J. M. Tarongi, M. Vall Llossera, A. Gutierrez,

J. Barbosa and R. Castro, "Radio-Frequency Interference Detection and Mitigation Algorithms for Synthetic Aperture Radiometers", Algorithms., Vol. 4(3) pp. 155-182, Aug. 2011.

9.3.2 Conference proceedings
• J. M. Tarongi, A. Camps and J. A. Pulido, "K-Band Radiometer

Designed for Academic Purposes: Intercomparison of Performances as Total Power, Dicke or Noise Injection Radiometers", International Geoscience and Remote Sensing Symposium IGARSS 2006, pp. 2927-2930, Barcelona, Spain, 23-27 Jul. 2007. •

J. M. Tarongi and A. Camps, "Multifequency Experimental Radiometer and Littoral:

with Interference Tracking for Experiments over Land

MERITXELL", International Geoscience and Remote Sensing Symposium IGARSS 2009, pp. IV-653-IV-656, Cape Town, South Africa, 13-17 Jul. 2009. •

J. M. Tarongi and A. Camps, "Normality Tests Analysis of Radiometric

Signals for Radio Frequency Interference Detection", 3rd International Workshop on Marine Technolog, MARTECH 2009, Vilanova i La Geltru, Spain, 19-20 Nov. 2009. •

J. M. Tarongi and A. Camps, "Radio Frequency Interference detection

algorithm based on spectrogram analysis", International Geoscience and Remote Sensing Symposium IGARSS 2010, pp. 2499-2502, Honolulu, USA, 25-30 Jul. 2010. •

J. M. Tarongi and A. Camps, "Normality analysis as a radio frequency

interference detection", 11th Specialist Meeting on Microwave Radiometry and Remote Sensing of the Environment µRad 2010, pp. 288-293, Washington DC, USA, 1-4 Mar. 2010.

248

Conclusions and future research lines

•

J. M. Tarongi, G. F. Forte and A. Camps, "Experimental study of radio-

frequency interference detection algorithms in microwave radiometry", International Geoscience and Remote Sensing Symposium IGARSS 2011, pp. 4213-4216, Vancouver, Canada, 24-29 Jul. 2011. •

J. F. Marchan Hernandez, N. Rodriguez-Alvarez, A. Camps, I. Ramos-

Perez, E. Valencia, X. Bosch-LLuis, M. Talone, J. M. Tarongi and M. Piles, "Ground-based GNSS-R Measurements with the PAU Instrument and their Application to the Sea Surface Salinity Retrieval: Firsts Results", International Geoscience and Remote Sensing Symposium IGARSS 2008, pp. IV-530-IV533, Boston, USA, 6-11 Jul. 2008. •

M. Talone, A. Camps, J. F. Marchan-Hernandez, J. M. Tarongi, M.

Piles, X. Bosch-Lluis, I. Ramos-Perez, E. Valencia, N. Rodriguez, M. VallLlossera and P. Ferre, "Preliminary Results of the Advanced L-band Transmission And Reflection Observation of the Sea Surface (ALBATROSS) Campaign: Preparing the SMOS Calibration and Validation Activities", International Geoscience and Remote Sensing Symposium IGARSS 2009, pp. IV-725-IV-728, Cape Town, South Africa, 13-17 Jul. 2009. •

E. Valencia, J. F. Marchan-Hernandez, A. Camps, N. Rodríguez, J. M.

Tarongi, M. Piles, I. Ramos-Perez, X. Bosch-Lluis, M. Vall-llossera and P. Ferre, "Experimental relationship between the sea brightness temperature changes and the GNSS-R delay-Doppler maps: Preliminary results of the albatross field experiments", International Geoscience and Remote Sensing Symposium IGARSS 2009, pp. III-741-III-744, Cape Town, South Africa, 13-17 Jul. 2009. •

A. Camps, J. Gourrion, J. M. Tarongi, A. Gutierrez, J. Barbosa and R.

Castro, "RFI Analysis in SMOS Imagery", International Geoscience and Remote Sensing Symposium IGARSS 2010, pp. 2007-2010, Honolulu, USA, 25-30 Jul. 2010.

249

Chapter 9

•

O. Nogues-Correig, S. Ribo, J. C. Arco, E. Cardellach, A. Rius, E.

Valencia, J. M. Tarongi, A. Camps, H. van der Marel and M. Martin-Neira, "The proof of concept for 3-cm Altimetry using the Paris Interferometric Technique", International Geoscience and Remote Sensing Symposium IGARSS 2010, pp. 3620-3623, Honolulu, USA, 25-30 Jul. 2010. •

G. F. Forte, J. M. Tarongi and A. Camps, "Hardware implementation of

a wavelet-based radio frequency interference mitigation algorithm for microwave radiometers", International Geoscience and Remote Sensing Symposium IGARSS 2011, pp. 2241-2244, Vancouver, Canada, 24-29 Jul. 2011. •

V. Depau, J. M. Tarongi, G. F. Forte and A. Camps, "Preliminary

performance study of different radio-frequency interference detection and mitigation algorithms in microwave radiometry", 12th Specialist Meeting on Microwave Radiometry and Remote Sensing of the Environment µRad 2012, pp. 1-4, Rome, Italy, 5-9 Mar. 2012. •

G. F. Forte, A. Camps, J. M. Tarongi and M. Vall-llossera, "Study of

Radio Frequency Interference Effects on Radiometry Bands in Urban Environments", International Geoscience and Remote Sensing Symposium IGARSS 2012, Munich, Germany, 23-28 Jul. 2012.

9.4 Master thesis supervised during this Ph.D.
• V. Depau. "Performance Study Of Different Radio-Frequency

Interference Detection And Mitigation Algorithms In Microwave Radiometry", Advisors: J. M. Tarongi and A. Camps, March 2012.

250

Bibliography

Bibliography
[1] A. Camps, "Application of Interferometric Radiometry to Earth Observation" Ph.D. dissertation, Universitat Politécnica de Catalunya, 1996. [Online]. Available: http://www.tdx.cat/handle/10803/6885, last visited: 10 Oct. 2012.

[2] F. T. Ulaby, R. K. Moore and A. K. Fung, "Microwave Remote Sensing. Active and Passive Vol. 1. Fundemantals and Radiometry", Artech House, 1981.

[3] F. T. Ulaby, R. K. Moore and A. K. Fung, "Microwave Remote Sensing. Active and Passive Vol. 2. Fundemantals and Radiometry", Artech House, 1981.

[4] J. Randa, J, Lahtinen, A. Camps, A. J. Gasiewski, M. Hallikainen, D. M. Le Vine, M. Martin-Neira, J. Piepmeier, P. W. Rosenkranz, C. S. Ruf, J. Shiue and N. Skou, “Recommended Terminology for Microwave Radiometry,” NIST Technical Note TN1551, National Institute of Standards and Technology, Gaithersburg, MD, USA, 2008.

[5] R.H. Dicke, "The measurement of thermal radiation at microwave frequencies", Revision on Science Instrumentation, Vol. 17, pp. 268-275, 1946.

[6] V. Pankonin and R. M. Price, 1981, "Radio Astronomy and Spectrum Management: The impact of WARC-79", IEEE Transactions on Electromagnetic Compatibility, Vo. EMC-23, p.308-317.

[7] G. Swarup and T. Venkatasubramani, "RFI Survey for a Giant Metrewave Radio Telescope for India", Light Pollution, Radio Interference, and Space Debris, ASP Conference Series, Vol. 17, IAU Colloquium 112, 1991, D.L. Crawford, Ed., p. 190.

[8] R. J. Cohen, "Interference Problems and Radio Astronomy in the U.K.", Light Pollution, Radio Interference, and Space Debris, ASP Conference Series, Vol. 17, IAU Colloquium 112, 1991, D.L. Crawford, Ed., p. 267.

251

Bibliography

[9] N. Skou, S. Misra, J. E. Balling, S. S. Kristensen and S. S. Søbjærg, "L-band RFI as Experienced During Airborne Campaigns in Preparation for SMOS" IEEE Transactions on Geoscience and Remote Sensing, Vol. 48 (3), pp. 1398-1407, Mar. 2010.

[10] J. E. Belz, B. L. Huneycutt and M. W. Spencer, "A Study of Radio Frequency Interference in the Space-to-Earth Explotarion Allocation at L-Band", 2011 IEEE Aerospace Conference, Montana, USA, 5-12 March 2011.

[11] A. Camps, J. Gourrion, J. Tarongi, A. Gutierrez, J. Barbosa and R. Castro, "RFI Analysis in SMOS Imagery", International Geoscience and Remote Sensing Symposium IGARSS 2010, pp. 2007-2010, Honolulu, USA, 25-30 Jul. 2010.

[12] M. Aksoy, J. Park and J. T. Johnson, "Joint analysis of radio frequency interference from SMOS measurements and from airborne observations", 17th ICOMOS General Assembly and Scientific Symposium 2011, Paris, France, 13-20 Aug. 2011.

[13] S. W. Ellingson and J. T. Johnson, "A polarimetric survey of radio frequency interference in C- and X-bands in the continental United States using WindSAT radiometry," IEEE Transactions on Geoscience and Remote Sensing, Vol. 44 (3), pp. 540-548, Mar. 2006.

[14] E.G. Njoku, P. Ashcroft, T.K. Chan and Li Li, "Global survey and statistics of radio-frequency interference in AMSR-E land observations", IEEE Transactions on Geoscience and Remote Sensing, Vol. 43 (5), pp. 938-947, May 2005.

[15] J. T. Johnson, A. J. Gasiewski, B. Güner, G. A. Hampson, S. W. Ellingson, R. Krishnamachari, N. Niamsuwan, E. McIntyre, M. Klein and V. Y. Leuski "Airborne Radio-Frequency Interference Studies at C-Band Using a Digital Receiver", IEEE Transactions on Geoscience and Remote Sensing, Vol. 44 (7), pp. 1974-1985, Jul. 2006.

[16] M. Younis, J. Maurer, J. Fortuny-Guasch, R. Schneider, W. Wiesbeck and A. J. Gasiewski, "Interference From 24-GHz Automotive Radars to Passive Microwave

252

Bibliography

Earth Remote Sensing Satellites", IEEE Transactions on Geoscience and Remote Sensing, Vol. 42 (7), pp. 1387-1398, Jul. 2004.

[17] S. Curry, M. Ahlers, H. Elliot, S. Gross, D. McKague, S. Misra, J. Puckett, C. Ruf, "K-Band Radio frequency Interference Survey of Southeastern Michigan" 2010 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp.24862489, 25-30 July 2010.

[18] Centre d’Etudes Spatiales de la BIOsphere, http://www.cesbio.upstlse.fr/SMOS_blog/?p=233, last visited: 10 Oct. 2012.

[19] F. Torres, I. Corbella, N. Duffo, V. González-Gambau, I. Duran, M. Pablos and M. Martin-Neira, "One Point Calibration in Interferometric Radiometers: MIRAS/SMOS Preliminary Results", 11th Specialist Meeting on Microwave Radiometry and Remote Sensing of the Environment µRad 2010, pp. 243-246, Washington DC, USA, 1-4 Mar. 2010.

[20]

WindSat

Payload

Description

http://www.nrl.navy.mil/WindSat/

Description.php, last visited: 10 Oct. 2012.

[21] Aqua Project Science http://aqua.nasa.gov/about/instrument_amsr.php, last visited: 10 Oct. 2012.

[22] Y. Taniguchi and A. Shibata, "RFI in the radiometric calibration data", Joint AMSR Science Team Meeting, Huntsville, AL, USA, 3 Jun 2010.

[23] P. A. Fridman and W. A. Baan, "RFI mitigation methods in radio astronomy", Astronomy and Astrophysics, Vol. 378(1), pp. 327-344, Oct. 2001.

[24] N. Niamsuwan, J. T. Johnson and W. Ellingson, "Examination of a simple pulse-blanking technique for radiofrequency interference mitigation", Radio Science, Vol. 40(3), Jun 2005.

253

Bibliography

[25] B. Winkel, J. Kerp and S. Stanko, "RFI detection by automated feature extraction and statistical analysis", Astronomische Nachrichten, Vol. 328, pp. 68-79, Jan. 2007.

[26] B. Guner, J. T. Johnson and N. Niamsuwan, "Time and Frequency Blanking for Radio-Frequency Interference Mitigation in Microwave Radiometry" IEEE Transactions on Geoscience and Remote Sensing, Vol.45(11), pp.3672-3679, Nov. 2007

[27] D. Zhihong, S. Xin, H. Jie and W. Siliang, "Setting CFAR threshold coefficient for non-coherent accumulation result of a square-law detector's outputs," 2009 IET International Radar Conference, pp.1-4, 20-22 Apr. 2009

[28] S. W. Ellingson and G. A. Hampson, "Mitigation of Radar Interference in L-Band Radio Astronomy", Astrophysics Journal Supplement Series, Vol. 147(1), pp. 167-176, Jul. 2003.

[29] C. S. Ruf, S. M. Gross, S. Misra, "RFI detection and mitigation for microwave radiometry with an agile digital detector", IEEE Transactions on Geoscience and Remote Sensing, Vol. 44(3), pp. 694-706, Mar. 2006.

[30] A. J. Gasiewski, M. Klein, A. Yevgrafov and V. Leuskiy, "Interference Mitigation in Passive Microwave Radiometry", International Geoscience and Remote Sensing Symposium IGARSS 2002, Vol. 3, pp. 1682-1684, Toronto, Canada, 24-28 Jun. 2002.

[31] B. Guner, M. T. Frankford and J. T. Johnson, "A Study of the Shapiro–Wilk Test for the Detection of Pulsed Sinusoidal Radio Frequency Interference", IEEE Transactions on Geoscience and Remote Sensing, Vol. 47(6), pp. 1745-1751, Jun. 2009.

[32] P. A. Fridman, "RFI excision using a higher order statistics analysis of the power spectrum", Astronomy & Astrophysics, Vol. 368, pp. 369-376, Oct. 2001.

254

Bibliography

[33] C. Ruf, S. Misra, S. Gross, and R. De Roo, "Detection of RFI by its Amplitude Probability Distribution", International Geoscience and Remote Sensing Symposium IGARSS 2006, pp. 2289-2291, Denver, USA, 31 Jul - 4 Aug 2006.

[34] J. R. Piepmeier, P. N. Mohammed and J. J. Knuble, "A Double Detector for RFI Mitigation in Microwave Radiometers", IEEE Transactions on Geoscience and Remote Sensing, Vol. 46(2), pp. 458-465, Feb. 2008.

[35] P. Fanise, M. Parde, M. Zribi, M. Dechambre and C. Caudoux, "Analysis of RFI Identification and Mitigation in CAROLS Radiometer Data Using a Hardware Spectrum Analyser", Sensors, Vol. 11(3), pp. 3037-3050, Mar. 2011.

[36] V. D. Vrabie, P. Granjon, and C. Serviere, "Spectral Kurtosis: from definition to application", Workshop on Nonlinear Signal and Image Processing, NSIP03, June 2003.

[37] G. Nita, D. E. Gary, Z. Liu, G. Hurford, and S. M. White, "Radio frequency interference excision using spectral domain statistics" Publications of the Astronomical Society of the Pacific, Vol. 119(857), pp. 805-827, Jul. 2007.

[38] J. T. Johnson, L. C. Potter, "Performance Study of Algorithms for Detecting Pulsed Sinusoidal Interference in Microwave Radiometry". IEEE Transactions of Geoscience and Remote Sensing, Vol. 47(2), pp. 628-636, Feb. 2009.

[39] S.W. Ellingson, J. Bunton, and J.F. Bell, "Removal of the GLONASS C/A Signal from OH Spectral Line Observations Using a Parametric Modelling Technique", Astrophysical Journal Supplement, Vol. 135(1), pp. 87-93, Jul. 2001.

[40] A. Camps and J. M. Tarongi, "RFI Mitigation in Microwave Radiometry Using Wavelets", Algorithms, Vol. 2(3) pp. 1248-1262, Sep. 2009.

[41] B. Widrow, J. R. Glover Jr., J. M. McCool, J. Kaunitz, C. S. Williams, R. H. Hearn, J. R. Zeidler, E. Dong Jr. and R. C. Goodlin "Adaptive-Noise Cancelling:

255

Bibliography

Principles And Applications", Proceedings of the IEEE, Vol. 63(12), pp. 1692-1716, Dec. 1975.

[42] C. Barnbaum and R. F. Bradley, "A New Approach to Interference Excision in Radio Astronomy: Real-Time Adaptive Cancellation", The Astronomical Journal, Vol. 116(5), pp. 2598-2614, Nov. 1998.

[43] A. Leshem, A.J. van der Veen, and A.J. Boonstra, "Multichannel interference mitigation techniques in radio astronomy", Astrophysical Journal Supplements, Vol. 131, No. 1, pp. 355-374, Nov. 2000.

[44] S. Misra and C. S. Ruf, "Analysis of Radio Frequency Interference Detection Algorithms in the Angular Domain for SMOS", IEEE Transactions of Geoscience and Remote Sensing, Vol. 50, No. 5, pp. 1448-1457, May 2012.

[45] R. D. De Roo, S. Misra and C. S. Ruf, "Sensitivity of the Kurtosis statistic as a detector of pulsed sinusoidal RFI", IEEE Transactions on Geoscience and Remote Sensing, Vol. 45(7), pp. 1938-1946, Jul. 2007.

[46] S. Misra and C. S. Ruf, "Detection of radio-frequency interference for the aquarius radiometer". IEEE Transactions on Geoscience and Remote Sensing, Vol. 46(10), pp. 3123-3128, Oct. 2008.

[47] R.D. De Roo and S. Misra, "Effectiveness of the sixth moment to eliminate a kurtosis blind spot in the detection of interference in a radiometer", International Geoscience and Remote Sensing Symposium IGARSS 2008, pp. 331-334, Boston, USA, 6-11 Jul 2008.

[48] B. Guner, M. T. Frankford and J. T. Johnson, "On the Shapiro–Wilk test for the detection of Pulsed Sinusoidal Radio Frequency Interference", International Geoscience and Remote Sensing Symposium IGARSS 2008, pp. II-157-II-160, Boston, USA, 6-11 Jul 2008.

256

Bibliography

[49] C. M. Jarque and A. K. Bera, "A Test for Normality of Observations and Regression Residuals", International Statistical Review, Vol. 55(2), pp. 163-172, Aug. 1987.

[50] C. M. Urzua, "On the correct use of omnibus tests for normality", Economic Letters, Vol. 53, pp 247-251, Nov. 1996.

[51] G. Poitras, "More on the correct use of omnibus tests for normality", Economic Letters, Vol. 90, pp. 304-309, Jan. 2006.

[52] R. B. D’Agostino, A. Belanger and R. B. D’Agostino Jr. "A suggestion for using powerful and informative tests of normality", American Statistical Assotiation, Vol. 44(4) pp. 316-321 Nov. 1990.

[53] H. W. Lilliefors, "On the Kolmogorov-Smirnov test for normality with mean and variance unknown". Journal of the American Statistical Association, Vol. 62(138), pp. 399-402, Jun. 1967. [54] R. B. D'Agostino and M. A. Stephens, "Goodness-of-Fit Techniques", 1st edition, Marcel Dekker: New York, NY, USA, pp. 122–133, 1986.

[55] A. Trujillo-Ortiz, R. Hernandez-Walls, K. Barba-Rojo and A. Castro-Perez, "AnDartest: Anderson-Darling test for assessing normality of a sample data", MATLAB file, 2007: http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId =14807, last visited: 10 Oct. 2012.

[56] S. S. Shapiro and M. B. Wilk, "An analysis of variance test for normality (complete samples) ". Biometrika, Vol.52(3/4), pp. 591–611, Dec. 1965.

[57] J. P. Royston "Approximating the Shapiro-Wilk W-test for non-normality". Statistics and Computing, Vol. 2(3), pp. 117-119, Sep. 1992.

[58] J. P. Royston, "An extension of Shapiro and Wilk’s W test for normality to large samples", Applied Statistics, Vol. 31(2), pp. 115-124, 1982. 257

Bibliography

[59] H. Cramer, "On the composition of elementary errors. Second paper: Statistical applications", Skandinavisk Aktuarietidskrift, Vol. 11, pp. 141-180, 1928. [60] H. Tong, "Non-Linear Time Series: a Dynamical System Approach", 1st edition; Oxford University Press: New York, NY, USA, pp. 324-325, 1990;

[61] H. G. Thode Jr., "Testing for Normality", 1st edition; Marcel Dekker: New York, NY, USA, pp. 88-89, 2002.

[62] C. C. Lin and G. S. Mudholkar "A simple test for normality against asymmetric alternatives". Biometrika, Vol. 67(2), pp. 455-461, Jan. 1980.

[63] K. Fazel and S. Kaiser, "Multi-carrier and spread spectrum systems", 1st Edition, John Wiley & Sons, West Sussex, pp. 24-30, 2003.

[64] A. R. Offringa, A. G. de Bruyn, M. Biehl, S. Zaroubi, G. Bernardi and V. N. Pandey, "Post-correlation radio frequency interference classification methods". Monthly Notices of the Royal Astronomical Society, Vol. 405(1), pp. 155-167, Jun. 2010. [65] K. Williston "Digital Signal Processing: World Class Designs", 1st edition, Elsevier: Burlington, VT, USA, pp. 141-142, 2009.

[66] J. A. Tenreiro, I. J. Rudas and B. Patkai, "Intelligent Engineering Systems and Computational Cybernetics", 1st edition, Springerlink, Berlin, Germany, pp. 56-58, 2009. [67] L. B. Jackson, "Digital Filters and Signal Processing", 5th edition, Kluwer Academic Publishers, Norwell, MA, USA, p. 205, 2002.

[68] F. J. Harris, "On the use for windows for harmonic analysis with the discrete Fourier transform". Proceedings of the IEEE, Vol. 66(1), pp. 51-83, Jan. 1978.

258

Bibliography

[69] J. O. Smith and X. Serra, "PARSHL: An Analysis/Synthesis Program for Non-Harmonic Sounds Based on a Sinusoidal Representation". Proceedings of International Computer Music Conference, pp. 290-297, Urbana, IL, USA, 23-26 August, 1987. [70] W. B. Nelson, " Applied Life Data Analysis", 1st edition, John Wiley & Sons, Hoboken, NJ, USA, p. 41, 2004.

[71] J. M. Tarongi and A. Camps "Radio Frequency Interference Detection and Mitigation Algorithms Based on Spectrogram Analysis", Algorithms, Vol. 4(4), pp. 239-261, Oct. 2011.

[72] J. S. Lee, "Digital image enhancement and noise filtering by use of local statistics". IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-2, pp. 165-168, Mar. 1980.

[73] D. Middleton, "Non-Gaussian noise models in signal processing for telecommunications: New methods and results for Class A and Class B noise models", IEEE Transactions on Information Theory, Vol. 45(4), pp. 1129-1149, May. 1999. [74] B. L. Evans, "Mitigation of Radio Frequency Interference from the Computer Platform to Improve Wireless Data Communication", Intel Labs Seminar University of Texas, 16th April 2007, http://users.ece.utexas.edu/~bevans/projects/rfi/ talks/April2007.html, last visited: 10 Oct. 2012 [75] MATLAB tutorial on denoising: http://www.mathworks.com/access/ helpdesk/help/toolbox/wavelet/index.html?/access/helpdesk/help/toolbox/wavelet/ch06_ a12.html, last visited: 10 Oct. 2012. [76] M. Misiti, Y. Misiti, G. Oppenheim and J. N. Poggi, "Wavelet Toolbox User's Guide", The MathWorks Inc. 1996. [77] E. Aboufadel and S. Schlicker, "Discovering Wavelets", Ed. John Wiley and Sons, New York, NY, USA, 1999, pp. 14. [78] D. L. Donoho and I. M. Johnstone, "Ideal spatial adaption via wavelet shrinkage", Biometrika, Vol. 81(3), 425- 455, Aug. 1994.

259

Bibliography

[79] S. P. Ghael, A. M. Sayeed and R. G. Baraniuk, "Improved Wavelet Denoising via Empirical Wiener Filtering", Proceedings of SPIE, Vol. 3169, pp. 389399, Mathematical Imaging, San Diego, USA, 30th July 1997. [80] M. Lang, H. Guo, E. Odegard, C. S. Burrus and R. O. Wells, "Nonlinear processing of a shift-invariant discrete wavelet transform (DWT) for noise reduction", Proceeding of the SPIE, Vol. 2491, pp. 640-651, Wavelet Applications II, Orlando, USA, 21th April 1995 [81] H. Resnikoff and R. O. Wells, "Wavelet Analysis: The Scalable Structure of Information", Ed. Springer-Verlag, New York, NY, USA, 2002; pp. 365 [82] Wavelet families and properties summary: http://www.mathworks.com/ access/helpdesk/help/toolbox/wavelet/index.html?/access/helpdesk/help/toolbox/wavele t/ch06_a12.html, last visited: 10 Oct. 2012. [83] M. Hallikainen, M. Kemppinen, K. Rautiainen, J. Pihlflyckt, J. Lahtinen, T. Tirri, I. Mononen and T. Auer, "Airborne 14-Channel Microwave Radiometer", International Geoscience and Remote Sensing Symposium IGARSS 1996, Vol. 4, pp. 2285-2287, Nebraska, USA, 27-31 May. 1996.

[84] J. R. Piepmeier and A. J. Gasiewski, "Polarimetric scanning radiometer for airborne microwave imaging studies", Proceedings of IEEE Sensing Symposium, Vol. 2, pp. 1120-1122, 27-31 May. 1996. Geoscience Remote

[85] W. Campbell, S. Swadley, W. Bell, C. Blankenship, B. Ruston and N Baker, "Pre-operational Assimilation Testing of the Defense Meteorological Satellite Program (DMSP) Special Sensor Microwave Imager/Sounder (SSMI/S)", ITSC XV Program, Maratea, Italy, 4-10 Oct. 2006.

[86] E. G. Njoku, "AMSR Land Surface Parameters: Surface Soil Moisture, Land Surface Temperature, Vegetation Water Content" Algorithm Theoretical Basis Document, Version 3.0, pp. 1-47, 10 Dec. 1999.

[87] A. Camps, X. Bosch-Lluis, I. Ramos-Perez, J. Marchan-Hernandez, B. Izquierdo, and N. Rodriguez-Alvarez, "New Instrument Concepts for Ocean Sensing: 260

Bibliography

Analysis of the PAU-Radiometer" IEEE Transactions on Geoscience and Remote Sensing, Vol. 45, No. (10), pp. 3180-3192, Oct. 2007.

[88] N. Skou and D. LeVine, "Microwave radiometer systems", Artech House Publishers, Norwood, MA, USA, 1989.

[89] I. Ramos-Perez, "PAU-Synthetic Aperture: a New Instrument to Test Potential Improvements for Future Interferometric Radiometers", Chapter 5. Ph. D. Thesis, Universidad Politecnica de Catalunya, Feb. 2012. [Online]. Available: http://www.tdx.cat/handle/10803/80600, last visited: 10 Oct. 2012.

[90] R. M. Villarino, "Empirical Determination of the Sea Surface Emissivity at L-band: A contribution to ESA's SMOS Earth Explorer Mission", Chapter 2. Ph. D. Thesis, Universidad Politecnica de Catalunya, Jun. 2004. [Online]. Available: http://www.tdx.cat/handle/10803/6884, last visited: 10 Oct. 2012.

[91] SiGe GN3S Sampler v.2. http://www.sparkfun.com/products/8238, last visited: 10 Oct. 2012.

[92] Rohde&Schwarz http://www2.rohde-schwarz.com/file_3418/FSP_OpMa_ en.pdf, last visited: 10 Oct. 2012.

261

List of Acronyms

List of Acronyms
AD ADC AMR AMSR-E ASK AWGN CHI2 CM CW DC DOA DWT ECDF EOS ERS-2 FT FFT FOV FPGA FWLD GLONASS GMI GNSS GPM HUTRAD IF INR IR JASON JB K Anderson-Darling Analog to Digital Converter Advanced Microwave Radiometer Advanced Microwave Scanning Radiometer EOS Amplitude Shift Keying Additive Gaussian White Noise Chi-square Cramer-von Mises Continuous Wave Direct Current Direction Of Arrival Discrete Wavelet Transform Empirical Cummulative Distribution Function Earth Observing System European Remote Sensing satellite 2 Fourier Transform Fast Fourier Transform Field Of View Field Programmable Gate Array Frequency Wavelet Line Detection GLObalnaya NAvigatsionnaya Sputnikovaya Sistema GPM Microwave Imager Global Navigation Satellite System Global Precipitation Measurement HUT multifrequency RADiometer Intermediate Frequency Interference to Noise Ratio InfraRed Joint Altimetry Satellite Oceanography Network Jarque-Bera Kurtosis 263

List of Acronyms

K2 KS L LAURA LEO LICEF LLMMSE LM LO MERITXELL

Agostino-Pearson K squared Kolmogorov-Smirnov Lilliefors L-band AUtomatic RAdiometer Low Earth Orbit Lightweight Cost Effective Front End Locally adaptive Linear Minimum Mean Square Error Lin-Muldhokar Local Oscilator Multifequency Experimental Radiometer with Interference

Tracking for Experiments over Land and Littoral MIRAS MLSG MWR NIR OFDM OMT OOSK PAU PDF PID PLC PRN PSR RBW RFI RMS RMSE ROC S SMOS SNR SRR SSMIS 264 Microwave Imaging Radiometer using Aperture Synthesis Maximum Length Sequence Generator Micro-Wave Radiometer Near InfraRed Orthogonal Frequency Division Multiplexing OrthoMode Transducer On-Off Shift Keying Passive Advanced Unit Probability Distribution Function Proportional, Integral, and Derivative Programmable Logic Controller Pseudo-Random Noise Polarimetric Scanning Radiometer Resolution BandWidth Radio Frequency Interference Root Mean Square Root Mean Square Error Receiver Operating Characteristic Skewness Soil Moisture and Ocean Salinity Signal-to-Noise Ratio Short-Range Radars Special Sensor Microwave Imager/Sounder

List of Acronyms

STFT SURE SW TWLD UWB WLD WSRT

Short Time Fourier Transform Stein's Unbiased Risk Estimate Shapiro-Wilk Time Wavelet Line Detection UltraWideBand Wavelet Line Detection Westerbork Synthesis Radio Telescope

265