Removing Diurnal Signals and Longer Term Trends From Electron Flux and ULF Correlations: A Comparison of Spectral Subtraction, Simple Differencing, and ARIMAX Models
dc.contributor.author | Simms, Laura E. | |
dc.contributor.author | Engebretson, Mark J. | |
dc.contributor.author | Reeves, Geoffrey D. | |
dc.date.accessioned | 2022-03-07T03:13:53Z | |
dc.date.available | 2023-03-06 22:13:50 | en |
dc.date.available | 2022-03-07T03:13:53Z | |
dc.date.issued | 2022-02 | |
dc.identifier.citation | Simms, Laura E.; Engebretson, Mark J.; Reeves, Geoffrey D. (2022). "Removing Diurnal Signals and Longer Term Trends From Electron Flux and ULF Correlations: A Comparison of Spectral Subtraction, Simple Differencing, and ARIMAX Models." Journal of Geophysical Research: Space Physics 127(2): n/a-n/a. | |
dc.identifier.issn | 2169-9380 | |
dc.identifier.issn | 2169-9402 | |
dc.identifier.uri | https://hdl.handle.net/2027.42/171887 | |
dc.description.abstract | Simultaneously cycling space weather parameters may show high correlations even if there is no immediate relationship between them. We successfully remove diurnal cycles using spectral subtraction, and remove both diurnal and longer cycles (e.g., the 27 days solar cycle) with a difference transformation. Other methods of diurnal cycle removal (daily averaging, moving averages [MAs], and simpler spectral subtraction using regression) are less successful at removing cycles. We apply spectral subtraction (a finite impulse response equiripple bandstop filter) to hourly electron flux (Los Alamos National Laboratory satellite data) and a ground‐based ULF index to remove a 24 hr noise signal. This results in smoother time series appropriate for short‐term (approximately < 1 week) correlation and observational studies. However, spectral subtraction may not remove longer cycles such as the 27 days and 11 yr solar cycles. A differencing transformation (yt – yt−24) removes not only the 24 hr noise signal but also the 27 days solar cycle, autocorrelation, and longer trends. This results in a low correlation between electron flux and the ULF index over long periods of time (maximum of 0.1). Correlations of electron flux and the ULF index with solar wind velocity (differenced at yt – yt−1) are also lower than previously reported (≤0.1). An autoregressive, MA transfer function model (ARIMAX) shows that there are significant cumulative effects of solar wind velocity on ULF activity over long periods, but correlations of velocity and ULF waves with flux are only seen over shorter time spans of more homogeneous geomagnetic activity levels.Plain Language SummaryRelationships between space physics processes are often based on correlations. However, variables following the same cycles or trends may show a spurious correlation that has nothing to do with their physical relationship. In space weather data, these common cycles may result from satellites orbiting the Earth daily, or the 27 days or 11 yr activity cycles of the Sun. The daily cycle can be removed using noise reduction techniques similar to that used to clean audio data. Differencing (subtracting the previous observation) can also remove both short‐term cycles and longer trends. However, we find that an autoregressive‐moving average time series model (ARIMAX) most successfully removes cycles and trends and allows the actual correlations between variables to be measured. Using ARIMAX models, we confirm that there are cumulative effects of solar wind velocity on the ULF index, but little correlation between either velocity or ULF waves with electron flux over long periods of time. This argues for limiting correlational studies to periods of constant geomagnetic activity.Key PointsCorrelations between space weather data can be artificially inflated by common cycles and trends unrelated to physical relationshipsCycles and trends can be removed by spectral subtraction, differencing, or autoregressive moving average transfer function modelsTransfer function models with cycles removed show correlations between wave activity and solar wind velocity, but not with electron flux | |
dc.publisher | John Wiley & Sons, Inc. | |
dc.subject.other | ULF waves | |
dc.subject.other | signal processing | |
dc.subject.other | ARIMAX models | |
dc.subject.other | solar wind velocity | |
dc.subject.other | Relativistic electron flux | |
dc.title | Removing Diurnal Signals and Longer Term Trends From Electron Flux and ULF Correlations: A Comparison of Spectral Subtraction, Simple Differencing, and ARIMAX Models | |
dc.type | Article | |
dc.rights.robots | IndexNoFollow | |
dc.subject.hlbsecondlevel | Astronomy and Astrophysics | |
dc.subject.hlbtoplevel | Science | |
dc.description.peerreviewed | Peer Reviewed | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/171887/1/jgra57021.pdf | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/171887/2/jgra57021_am.pdf | |
dc.identifier.doi | 10.1029/2021JA030021 | |
dc.identifier.source | Journal of Geophysical Research: Space Physics | |
dc.identifier.citedreference | Reeves, G. D., Baker, D. N., Belian, R. D., Blake, J. B., Cayton, T. E., Fennell, J. F., et al. ( 1998 ). The global response of relativistic radiation belt electrons to the January 1997 magnetic cloud. Geophysical Research Letters, 25 ( 17 ), 3265 – 3268. https://doi.org/10.1029/98gl02509 | |
dc.identifier.citedreference | Loto’aniu, T. M. ( 2006 ). Radial diffusion of relativistic electrons into the radiation belt slot region during the 2003 Halloween geomagnetic storms. Journal of Geophysical Research: Space Physics, 111, A04218. https://doi.org/10.1029/2005JA011355 | |
dc.identifier.citedreference | Loto’aniu, T. M., Singer, H. J., Waters, C. L., Angelopoulos, V., Mann, I. R., Elkington, S. R., & Bonnell, J. W. ( 2010 ). Relativistic electron loss due to ultralow frequency waves and enhanced outward radial diffusion. Journal of Geophysical Research: Space Physics, 115, A12245. https://doi.org/10.1029/2010JA015755 | |
dc.identifier.citedreference | Makridakis, S. G., Wheelwright, S. C., & Hyndman, R. J. ( 1998 ). Forecasting: Methods and applications ( 3rd ed., p. 652). New York, NY: John Wiley and Sons. | |
dc.identifier.citedreference | Mathie, R. A., & Mann, I. R. ( 2000 ). A correlation between extended intervals of ULF wave power and storm‐time geosynchronous relativistic electron flux enhancements. Geophysical Research Letters, 27 ( 20 ), 3261 – 3264. https://doi.org/10.1029/2000GL003822 | |
dc.identifier.citedreference | Mathie, R. A., & Mann, I. R. ( 2001 ). On the solar wind control of Pc5 ULF pulsation power at mid‐latitudes—Implications for MeV electron acceleration in the outer radiation belt. Journal of Geophysical Research: Space Physics, 106, 29783 – 29796. https://doi.org/10.1029/2001JA000002 | |
dc.identifier.citedreference | Neter, J., Wasserman, W., & Kutner, M. ( 1985 ). Applied linear statistical models ( 2 nd ed., p. 112). Homewood, IL, USA: Richard D. Irwin, Inc. | |
dc.identifier.citedreference | O’Brien, T. P., & McPherron, R. L. ( 2003 ). An empirical dynamic equation for energetic electrons at geosynchronous orbit. Journal of Geophysical Research: Space Physics, 108 ( A3 ), 1137. https://doi.org/10.1029/2002JA009324 | |
dc.identifier.citedreference | O’Brien, T. P., Sornette, D., & McPherron, R. L. ( 2001 ). Statistical asynchronous regression: Determining the relationship between two quantities that are not measured simultaneously. Journal of Geophysical Research: Space Physics, 106, 13247 – 13259. https://doi.org/10.1029/2000JA900193 | |
dc.identifier.citedreference | Oppenheim, A. V., & Schafer, R. W. ( 2010 ). Discrete‐time signal processing ( 3rd ed., p. 1132). Hoboken, NJ: Pearson Education, Inc. | |
dc.identifier.citedreference | Pankratz, A. ( 1991 ). Forecasting with dynamic regression models (p. 386). New York, NY: John Wiley & Sons Inc. | |
dc.identifier.citedreference | Parks, T., & McClellan, J. ( 1972 ). Chebyshev approximation for nonrecursive digital filters with linear phase. IEEE Transactions on Circuit Theory, 19 ( 2 ), 189 – 194. https://doi.org/10.1109/TCT.1972.1083419 | |
dc.identifier.citedreference | Pilipenko, V. A., Kozyreva, O. V., Engebretson, M. J., & Soloviev, A. A. ( 2017 ). ULF wave power index for space weather and geophysical applications: A review. Russian Journal of Earth Sciences, 17, ES1004. https://doi.org/10.2205/2017ES000597 | |
dc.identifier.citedreference | Potapov, A. S., Tsegmed, B., & Ryzhakova, L. V. ( 2012 ). Relationship between the fluxes of relativistic electrons at geosynchronous orbit and the level of ULF activity on the Earth’s surface and in the solar wind during the 23rd solar activity cycle. Cosmic Research, 50 ( 2 ), 124 – 140. https://doi.org/10.1134/S0010952512020086 | |
dc.identifier.citedreference | Reeves, G. D., Morley, S. K., Friedel, R. H. W., Henderson, M. G., Cayton, T. E., Cunningham, G., et al. ( 2011 ). On the relationship between relativistic electron flux and solar wind velocity: Paulikas and Blake revisited. Journal of Geophysical Research: Space Physics, 116, A02213. https://doi.org/10.1029/2010JA015735 | |
dc.identifier.citedreference | Reeves, G. D., Spence, H. E., Henderson, M. G., Morley, S. K., Friedel, R. H. W., Funsten, H. O., et al. ( 2013 ). Electron acceleration in the heart of the Van Allen radiation belts. Science, 341 ( 6149 ), 991 – 994. https://doi.org/10.1126/science.1237743 | |
dc.identifier.citedreference | Rigler, E. J., Baker, D. N., Weigel, R. S., Vassiliadis, D., & Klimas, A. J. ( 2004 ). Adaptive linear prediction of radiation belt electrons using the Kalman filter. Space Weather, 2, S03003. https://doi.org/10.1029/2003SW000036 | |
dc.identifier.citedreference | Simms, L. E., & Engebretson, M. J. ( 2020 ). Classifier neural network models predict relativistic electron events at geosynchronous orbit better than multiple regression or ARMAX models. Journal of Geophysical Research: Space Physics, 125, e2019JA027357. https://doi.org/10.1029/2019JA027357 | |
dc.identifier.citedreference | Simms, L. E., Engebretson, M., Clilverd, M., Rodger, C., Lessard, M., Gjerloev, J., & Reeves, G. ( 2018 ). A distributed lag autoregressive model of geostationary relativistic electron fluxes: Comparing the influences of waves, seed and source electrons, and solar wind inputs. Journal of Geophysical Research: Space Physics, 123. https://doi.org/10.1029/2017JA025002 | |
dc.identifier.citedreference | Simms, L. E., Engebretson, M. J., Pilipenko, V., Reeves, G. D., & Clilverd, M. ( 2016 ). Empirical predictive models of daily relativistic electron flux at geostationary orbit: Multiple regression analysis. Journal of Geophysical Research: Space Physics, 121, 3181 – 3197. https://doi.org/10.1002/2016JA022414 | |
dc.identifier.citedreference | Simms, L. E., Engebretson, M. J., Rodger, C. J., Dimitrakoudis, S., Mann, I. R., & Chi, P. J. ( 2021 ). The Combined influence of lower band chorus and ULF waves on radiation belt electron fluxes at individual L‐shells. Journal of Geophysical Research: Space Physics, 126 ( 5 ), e2020JA028755. https://doi.org/10.1029/2020JA028755 | |
dc.identifier.citedreference | Simms, L. E., Pilipenko, V. A., Engebretson, M. J., Reeves, G. D., Smith, A. J., & Clilverd, M. ( 2014 ). Prediction of relativistic electron flux following storms at geostationary orbit: Multiple regression analysis. Journal of Geophysical Research: Space Physics, 119, 7297 – 7318. https://doi.org/10.1002/2014JA019955 | |
dc.identifier.citedreference | Su, Y.‐J., Quinn, J. M., Johnston, W. R., McCollough, J. P., & Starks, M. J. ( 2014 ). Specification of >2 MeV electron flux as a function of local time and geomagnetic activity at geosynchronous orbit. Space Weather, 12, 470 – 486. https://doi.org/10.1002/2014SW001069 | |
dc.identifier.citedreference | Baker, D. N., McPherron, R. L., Cayton, T. E., & Klebesadel, R. W. ( 1990 ). Linear prediction filter analysis of relativistic electron properties at 6.6 RE. Journal of Geophysical Research: Space Physics, 95 ( A9 ), 15133 – 15140. https://doi.org/10.1029/ja095ia09p15133 | |
dc.identifier.citedreference | Borovsky, J. E., & Denton, M. H. ( 2014 ). Exploring the cross‐correlations and autocorrelations of the ULF indices and incorporating the ULF indices into the systems science of the solar wind‐driven magnetosphere. Journal of Geophysical Research: Space Physics, 119, 4307 – 4334. https://doi.org/10.1002/2014JA019876 | |
dc.identifier.citedreference | Boynton, R. J., Amariutei, O. A., Shprits, Y. Y., & Balikhin, M. A. ( 2019 ). The system science development of local time‐dependent 40 keV electron flux models for geostationary orbit. Space Weather, 17, 894 – 906. https://doi.org/10.1029/2018SW002128 | |
dc.identifier.citedreference | Boynton, R. J., Aryan, H., Dimmock, A. P., & Balikhin, M. A. ( 2020 ). System identification of local time electron fluencies at geostationary orbit. Journal of Geophysical Research: Space Physics, 125, e2020JA028262. https://doi.org/10.1029/2020JA028262 | |
dc.identifier.citedreference | Brown, R. G., & Hwang, Y. C. ( 2012 ). Introduction to random signals and applied Kalman filtering ( 4th ed. ). USA: John Wiley & Sons, Inc. | |
dc.identifier.citedreference | Engebretson, M., Glassmeier, K. H., Stellmacher, M., Hughes, W. J., & Luhr, H. ( 1998 ). The dependence of high‐latitude Pc5 wave power on solar wind velocity and on the phase of high‐speed solar wind streams. Journal of Geophysical Research, 103, 26271 – 26283. https://doi.org/10.1029/97JA03143 | |
dc.identifier.citedreference | Granger, C. W. J., & Newbold, P. ( 1974 ). Spurious regression in econometrics. Journal of Econometrics, 2, 111 – 120. https://doi.org/10.1016/0304-4076(74)90034-7 | |
dc.identifier.citedreference | Hyndman, R. J., & Athanasopoulos, G. ( 2018 ). Forecasting: Principles and practice ( 2nd ed, p. 291 ). Victoria, Australia: OTexts, Heathmont. | |
dc.identifier.citedreference | Kozyreva, O., Pilipenko, V., Engebretson, M. J., Yumoto, K., Watermann, J., & Romanova, N. ( 2007 ). In search of a new ULF wave index: Comparison of Pc5 power with dynamics of geostationary relativistic electrons. Planetary Space Sciences, 55, 755 – 769. https://doi.org/10.1016/j.pss.2006.03.013 | |
dc.working.doi | NO | en |
dc.owningcollname | Interdisciplinary and Peer-Reviewed |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.