Removing Diurnal Signals and Longer Term Trends From Electron Flux and ULF Correlations: A Comparison of Spectral Subtraction, Simple Differencing, and ARIMAX Models

Abstract

Simultaneously cycling space weather parameters may show high correlations even if there is no immediate relationship between them. We successfully remove diurnal cycles using spectral subtraction, and remove both diurnal and longer cycles (e.g., the 27 days solar cycle) with a difference transformation. Other methods of diurnal cycle removal (daily averaging, moving averages [MAs], and simpler spectral subtraction using regression) are less successful at removing cycles. We apply spectral subtraction (a finite impulse response equiripple bandstop filter) to hourly electron flux (Los Alamos National Laboratory satellite data) and a ground‐based ULF index to remove a 24 hr noise signal. This results in smoother time series appropriate for short‐term (approximately < 1 week) correlation and observational studies. However, spectral subtraction may not remove longer cycles such as the 27 days and 11 yr solar cycles. A differencing transformation (yt – yt−24) removes not only the 24 hr noise signal but also the 27 days solar cycle, autocorrelation, and longer trends. This results in a low correlation between electron flux and the ULF index over long periods of time (maximum of 0.1). Correlations of electron flux and the ULF index with solar wind velocity (differenced at yt – yt−1) are also lower than previously reported (≤0.1). An autoregressive, MA transfer function model (ARIMAX) shows that there are significant cumulative effects of solar wind velocity on ULF activity over long periods, but correlations of velocity and ULF waves with flux are only seen over shorter time spans of more homogeneous geomagnetic activity levels.Plain Language SummaryRelationships between space physics processes are often based on correlations. However, variables following the same cycles or trends may show a spurious correlation that has nothing to do with their physical relationship. In space weather data, these common cycles may result from satellites orbiting the Earth daily, or the 27 days or 11 yr activity cycles of the Sun. The daily cycle can be removed using noise reduction techniques similar to that used to clean audio data. Differencing (subtracting the previous observation) can also remove both short‐term cycles and longer trends. However, we find that an autoregressive‐moving average time series model (ARIMAX) most successfully removes cycles and trends and allows the actual correlations between variables to be measured. Using ARIMAX models, we confirm that there are cumulative effects of solar wind velocity on the ULF index, but little correlation between either velocity or ULF waves with electron flux over long periods of time. This argues for limiting correlational studies to periods of constant geomagnetic activity.Key PointsCorrelations between space weather data can be artificially inflated by common cycles and trends unrelated to physical relationshipsCycles and trends can be removed by spectral subtraction, differencing, or autoregressive moving average transfer function modelsTransfer function models with cycles removed show correlations between wave activity and solar wind velocity, but not with electron flux