034796-2-T Parallel Electromagnetic Solvers for High Frequency Antenna and Circuit Design: Efficient Numerical Solutions to Large Scale Electromagnetic Problems by Code Parallelization and Near-Matrix Diagonalization Techniques ANNUAL PROGRESS REPORT ARL Contract: DAAH04-96-1-0390 Linda P.B. Katehi February 1998 34796-2-T = RL-2477

Reporting Period: January 1 997 - December 1 997 Students and Research Associates: Jui-Ching Cheng Dong Hoon Chen Eray Yasan Jong Gwan Yook Faculty: Linda P.B. Katehi Manuscripts Published or Submitted During the Reporting Period. [1] J.-G. Yook, J. Cheng, D. Chen and Linda katehi, "Parallel Electromagnetic Solvers for High Frequency Antenna and Circuit Design," DoD HPCMP User Group Meeting, San Diego, CA. 1997. [2] J.-G. Yook, J. Cheng and Linda katehi, "Parallel Electromagnetic Solvers for High Frequency Antenna and Circuit Design," DoD Mission Success Storyfrom High Performance Computing. [3] G. Ponchak, D.-H. Chen, J.-G. Yook and Linda katehi, "Characterization of Plated Via Hole Fensces for Isolation between Stripline Circuits in LTCC Packages," IEEE MTT-S 1998 Symposium (accepted). [4] E. Yasan, J.-G. Yook and Linda katehi, "Generalized Method for Including Two Port Networks in Microwave circuits Using the Finite Element Method," ACES 97 (presented). [5] E. Yasan and Linda katehi, "An FEM based Method on Full Wave Characterization of Distributed Circuits Including Linear and Nonlinear Elements," IEEE AP-S/URSI 1998 Symposium (submitted).

Annual Progress Report on Parallel Electromagnetic Solvers for High Frequency Antennna and Circuit Design: Efficient Numerical Solutions to Large Scale Electromagnetic Problems by Code Parallelization and Near-Matrix Diagonallization Techniques by Linda P.B. Katehi (with students J. Cheng, DH Chen, E. Yasan and Postdoctoral Fellow JG Yook) ARL Contract: DAAH04-96-1-0390 January 1997 - December 1997

1 Introduction This report summarizes the effort on the development of accurate and computationally efficient codes for large scale electromagnetic problems with emphasis on three-dimensional monolithic circuits for high-frequency applications. This development will be successfully accomplished by two approaches: * increasingly sparse matrix techniques also known as near matrix diagonalization (NMD) techniques * code parallelization using Massage Passing Interface (MPI). In the following, description of CHSSI scalable software project goal and progress summaries for variuos efforts are provided. 2 Project Goal High-frequency radar systems for communication, detection and surveillance incorporate MMIC modules which are characterized by high density and substantial geometrical complexity. In most cases these modules are packaged for electrical and/or environmental protection and the resulting 3D structures exhibit a considerable complexity that impedes design and influences electrical performance due to unwanted parasitics. The design of these complex three-dimensional monolithic circuit (MMIC modules) with hundreds of vias and lines interconnecting a large number of active and passive circuit components to a rather large number of radiating elements, has been a problem of critical importance to DoD. The design and characterization of these circuits requires numerical approaches which can fully characterize the excited electromagnetic fields. Among the various techniques, the method of moments has demonstrated superiority in solving radiation problems but it has been hindered by long computational times and is limited, for this reason, to small circuits not exceeding more than a few components and interconnecting lines. In this study, we have concentrated on the development of codes with capability of accelerating numerical computations in electromagnetic problems with large computational domains and/or computationally intensive tasks. Among the existing full-wave methods which have been developed for solving EM problems are the Integral Equation Technique (IE) and Finite Element Method (FEM). These methods are numerical techniques which solve Maxwell's Equations in the frequency domain using large computational volumes and intensive numerical calculations which have to be performed repeatedly at all frequency points of interest. In addition to these problems, the numerical solution of Maxwell's Equations results in full (IE case) and huge sparse (FEM case) matrices which further limit applicability of the technique. With regards to substantially increasing the computational efficiency of these techniques, we concentrate to improve IE 1

method through near-matrix diagonalization technique (multiresolution analysis) as well as code parallelization strategy and FEM through task parallelization. The project leverages existing synergism between DOD and the University of Michigan scientists and engineers conducting large-scale simulations of advanced software systems. The team will concentrate on the development, validation, documentation, visualization, and demonstration of scaleable algorithms for modeling critical DOD problems in high frequency electromagnetic circuits. Technology transfer to the DOD user community is an integral component of the project. In addition, cross-platform portability and software reusability will be emphasized using the Message-Passing Interface (MPI) standard. In our effort, developement of scalable softwares which are based on the IE and FEM techniques so that it sustains the high accuracy capability while, at the same time, solves complex planar circuits including their packages in very short times, allowing for real time simulations. Such softwares can eventually lead to real time design and optimization. 3 Integral Equation with Multiresolution Analysis [1,2] From our previous study, it has been indicated that the use of Multiresolution expansions in integral based formulations leads to the generation of highly sparse linear systems whose numerical solution is fast and efficient with minimal memory usage. This finding allows us to believe that application of wavelet techniques to the method of moments has made the modeling of large scale electromagnetic problems possible. The capability of the multiresolution analysis (MRA) to compete with the fastest up-to-date techniques, such as the Fast Multipole Method (FMM) and Multilevel Fast Multipole Method (MFMM) has been demonstrated through very thorough computer experiments. We expect this trend to sustain as the size of matrix increases to hundreds of thousands or millions. The roof top multiresolution basis is used for the expansion of planar currents, these basis functions satisfy the relevant boundary and edge conditions found in printed circuits in an exact manner. Through this project it is shown that wavelet-dominated expansion bases generate highly sparse moment matrices and the resulting sparse linear systems can be solved numerically using very efficient computational tools. The fast wavelet transform is exploited for the numerical integration of the Green's function to speed up the matrix fill process. This work is emphasizing results for microstrip patch antenna and finite size arrays of such antennas. In addition, special emphasis is being placed on the solution of large-scale problems, which could not otherwise be handled by conventional moment method implementations. These efforts should yield a unique design capability for maintaining technology leadership. The MRA allows the analysis software to perform a thresholding process on the entire matrix. An additional benefit of development of this software will be a theoretical foundation of the 2

thresholding phenomenon and it will be expanded upon in detail using the mathematical properties of multiresolution analysis. The result of this detailed analysis and code development will be to accelerate the exploitation of scaleable parallel high performance computing systems to solve electrically large electromagnetics problems of critical importance to DOD. While in these problems we have been able to provide matrix sparsities of the order of 99.9% (a perfectly diagonal matrix has a sparsity of ( - 1/N)% which is equal to 99.99% for N = 1000), numerical computations have become more complex and led ngthy. A large portion of these computation s are in the form of multiple nested loops where similar types of intensive numerical computations are repeated. Furthermore, a large number of branched or cascaded loops are performed in a serial fashion. Parallelization of these code segments improves the speed and efficiency to a large extend. Another approach to improve the computation time is through the fast wavelet algorithm. This algorithm performs all the computations at the highest resolution level only once and then to reconstruct the rest of the computations in the remain resolution levels very efficiently. In addition to code parallelization, domain decomposition techniques and task parallelization have a great potential with this technique and are applied in addition to near-matrix diagonlization. Demonstration of scaleable simulations will address the critical DOD applications area of electrically large antennas integrated to multi-layer RF front end electronics which will itself be electrically large. Such systems need to be analyzed at the phenomenological level at one time in order to understand the details of device performance in the large subsystems environment. 4 Parallelized FEM for MMIC Simulation [1, 2, 3] The numerical analysis of MMIC using parallel computer becomes powerful especially for FEM. The frequency parallelization of FEM acheives linearly scaleable performance with the number of processors being used. The frequency parallelization is completed using MPI standard for distributed memory machine, IBM SP2. The application of frequency parallelizated FEM code to microstrip feeding network for patch array antenna is accomplished and its field plot is available by recently developed post field plotting Matlab file. Parallelization scheme will be further applied to the patch antenna structure including microstrip feeding network using hybrid method(MoM/FEM). This work will involve the parallelization of matrix generation for hybrid method, the parallelization of far field calculation for each array antenna and the development of post-processor files for radiation patterns. Another application of the parallelized FEM scheme is accomplished for the characterization of microwave/millimeterwave package. As the efficient packaging technology is emphasized for high frequency and high performance microwave/millimeter-wave circuits, it is very important to provide design rules to avoid unwanted electromagnetic effects of package on the packaged circuit performance. In this study, the electromagnetic characteristics of via on microstrip/stripline and the suppression of higher 3

order mode in the cavity with striplines are investigated and give the design rules for successful package design. In the study of electromagnetic characteristics of vias on microstrip and stripline, three different types of via fence is considered: continuous metal filled via fence on both sides of the circuits, short section of via fence on both sides of the circuits, and via fence cross to the circuits. In each case, via fence is moved between 1 h to 4 h from the circuit where 1 h is the height between the ground plane and the center conductor of the circuit. As the scattering parameters and radiation loss of the circuits do not vary by more than a few percent over the simulation frequency (10 GHz - 40 GHz), the average values are calculated for each case. Moreover, the field distributions over the circuits are generated. The result shows that the via fence closer to the circuits have more interactions with the EM field increasing the reflection coefficient, the insertion and radiation loss. Furthermore, the result shows that the closely spaced via fence confines more electric field than the wider spaced via fence giving less radiation loss. It is noted that stripline shows better EM performance than microstrip over the via fence in the package. The suppression of higher order mode in the package is studied for the cavity with two different types of striplines: the cavity with two isolated striplines and the cavity with one bended stripline. In the former case, via fence is placed between two isolated striplines and via gaps are changed from 57.228 h to 13.932 h. In order to visualize the suppression of higher order mode in quantative way, only one stripline is excited and the ratio, maximum magnitude of total electric field on the unexcited stripline over the one on the excited stripline, is calculated. The result shows that the via gap should be less than 0.394 Ac for the size of 1 h x 1 h via to suppress unwanted cavity mode under -50 dB, where Ac is the cutoff wavelength of dominant mode in the cavity. The field distribution over the cavity is shown in the result. The cavity with bended stripline is simulated for different via gaps, from 56.76 h to 2.6 h. The scattering paramenters of the stripline are calculated for each case and field distributions over the cavity are presented. It is seen that via gap should not larger than 0.248 Ac for the size of 1 h x 1 h via to suppress cavity mode in the package, where Ac is the cutoff wavelength of dominant mode in the cavity. All simulations are accomplished by a parallelized 3D-FEM. The frequency parallelization scheme is implemented in this study, because of the independent nature of the problem in the frequency domain. The parallel computer used for the simulation is the IBM SP2 with distributed memory in the University of Michigan. The performance of the parallelized 3-D FEM program is near linearly improved with respect to the number of processes that are used. This study shows the electromagnetic characteristics of via fence on microstrip/stripline and the suppression of higher order mode with via fence in the package. The result can be used as guide lines for designing high performance and high frequency packages. 4

5 Active Circuits in FEM [4, 5] Based on our previous research, we have launched an effort to analyze active as well as passive circuits all together, such as a mixer circuit (with 4 diodes), by the help of FEM. This is an extension of what we have been conducting for last 2 years in the sense that we used nonlinear elements. Using the FEM, the Z parameters (or Y parameters) of the passive part of the circuit are computed and then these values are fed into a software such as Libra in the form of a linear data element with inner and outer ports. The linear or nonlinear elements are connected to inner ports and the whole circuit is analyzed by the software in order to find in/output characteristics (most of the time S parameters). In mixer case (or multiplier) analysis of the whole circuit after analyzing the linear part is done using harmonic balance method (which is also a bench in libra). The goal is to verify the experimental results of mixer circuits under consideration and make some further improvement. The next step is finding a method to integrate harmonic balance method into the FEM method and then parallelization of this method. The direct inclusion of linear passive elements (resistors, capacitiors etc.) into FEM code has been accomplished. References [1] J.-G. Yook, J. Cheng, D. Chen and Linda katehi, "Parallel Electromagnetic Solvers for High Frequency Antenna and Circuit Design," DoD HPCMP User Group Meeting, San Diego, CA. 1997. [2] J.-G. Yook, J. Cheng and Linda katehi, "Parallel Electromagnetic Solvers for High Frequency Antenna and Circuit Design," DoD Mission Success Storyfrom High Performance Computing. [3] G. Ponchak, D.-H. Chen, J.-G. Yook and Linda katehi, "Characterization of Plated Via Hole Fensces for Isolation between Stripline Circuits in LTCC Packages," IEEE MTT-S 1998 Symposium (accepted). [4] E. Yasan, J.-G. Yook and Linda katehi, "Generalized Method for Including Two Port Networks in Microwave circuits Using the Finite Element Method," ACES 97 (presented). [5] E. Yasan and Linda katehi, "An FEM based Method on Full Wave Characterization of Distributed Circuits Including Linear and Nonlinear Elements," IEEE AP-S/URSI 1998 Symposium (submitted). 5

PARALLEL ELECTROMAGNETIC SOLVERS FOR HIGH FREQUENCY ANTENNA/CIRCUIT DESIGN Barry S. Perlman Jong-Gwan Yook*, Juiching Cheng Army Research Laboratory Donghoon Chen, and Linda P. B. Katehi Sensors and Electronics Directorate Radiation Laboratory Fort Monmouth, NJ 07703 The University of Michigan, Ann Arbor, MI 48109 yookjong @engin.umich.edu Presented/Published in DoD HPCMP User Group Meeting, San Diego, CA, 1997 Abstract In this paper, several different parallelization strategies for the method of moment (MoM) and the finite element method (FEM) are implemented on a distributed memory parallel computer (IBM SP2) having 48 CPUs. For the portability of the parallel codes on various platforms, standard message passing paradigm (MPI) has been adapted. Having in mind that a major bottleneck of the parallelized codes is the interprocessor communication overhead, data independence is fully exploited for efficient parallelization and as a result near linear scalability has been achieved. I. Introduction High-frequency radar systems for communication, detection and surveillance incorporate MMIC modules which are characterized by high density and substantial geometrical complexity. In most cases these modules are packaged for electrical and/or environmental protection and the resulting 3D structures exhibit a considerable complexity that impedes design and influences electrical performance due to unwanted parasitics. The design of these complex three-dimensional monolithic circuit (MMIC modules) with hundreds of vias and lines interconnecting a large number of active and passive circuit components to a rather large number of radiating elements, has been a problem of critical importance to DoD. The design and characterization of these circuits requires numerical approaches which can fully characterize the excited electromagnetic fields. Among the various techniques, the method of moments has demonstrated superiority in solving radiation problems but it has been hindered by long computational times and is limited, for this reason, to small circuits not exceeding more than a few components and interconnecting lines. In this paper, a novel technique based on the multiresolution expansion and the hybrid combination of the integral equation technique and the finite element method is parallelized to solve very large problems. II. Parallelization Strategies In this section, several different parallelization strategies for MoM and FEM have been presented. In the former case, parallel MoM impedance matrix generation and parallel fast wavelet transform (FWT) are 1

Parallelization Model of Impeduance Matrix Fill I'arallelizing Model of Fast \Vavelet Transform - Command path Worker - Data path - Data pa!h _ Worker Transformed Impedance - Impedance Impedance Matrix Matrix Matrix Dispatcher * Send commands to Workers. < * Evenly divided by N. ~ Gather results frot * Receive data from Workers. Send each share to each CPU. * Store data in impedance matrix. \each CPU. m JoWorker * Receive commands from Dispatcher. * Perform FWT. * Perform computation. * Send result to Dispatcher. (a) (b) Figure 1: Parallelization schemes for MoM: (a) Parallel impedance matrix filling. (b) Parallel fast wavelet transform. implemented, while parallelization of iterative linear equation solver and frequency parallelization schemes are tested for the later case. A. Parallel MoM Matrix Element Generation In method of moments, each matrix element is produced by the integration of a source function and a test function with an integration kernel. When the closed form representation is not available, numerical integration, which is time-consuming, must be used. However, the procedure of the integration of each matrix element fits quite well to the Single Instruction Multiple Data (SIMD) parallelization model, enabling the utilization of parallel computers to speed up the process. Because the computation time of each matrix element can vary in a great range, an algorithm that dynamically assigns jobs to each CPU is devised as illustrated in figure 1(a). In this algorithm, one CPU is reserved as a dispatcher to send jobs to and gather results from other CPUs. Jobs are sent whenever there are free CPUs. To achieve performance gain, the communication time between the CPUs must be much smaller than the computation time which is true in most impedance matrix calculation. B. Fast Wavelet Transform In order to take the advantage of multiresolution analysis [1], the MoM matrix is further transformed by FWT to its equivalent representation in wavelet bases. The FWT also fits to SIMD model, but comparing to the generation of MoM matrix elements, the computation time for each element is much smaller and the total number of iterations is much higher. If the same algorithm as the MoM matrix element generation is used, communication overhead will be too high. Instead, another algorithm is devised to reduce the communication overhead as shown in figure l(b). In this algorithm, jobs are evenly divided to each CPU. Only one communication is needed when each CPU completes its entire job list and sends the results back to the designated CPU for further processing. C. Task Parallelization 2

Geometry Definition Pre-processing: Mesh Generation Frequency Sweep fl... fn Element Matrix Computation -.Matrix Assembly: Ax=b r ---------- --------- Linear Equation Solver: BiCG Parallelization to- -- ----- S-Parameters Q.. k Geometry Definition Pre-processing: Mesh Generation '' Frequency Sweep', Parallelization -- f1... fn -' I ----. --- —----------- z~t, ------------—, --- —-- — t L --- —-------- -—. —J - --- ------------ -— I f l f2, Element Matrix Computation i fn1 fn ------------- - j Matrix Assembly: Ax=b r --------- r — -------------, --- --—,,,,_,,,,,, j Linear Equation Solver: BiCG j ------------- T --- —-------- j,J j, I JS-Parameters j t-t. —. ---:: -- --—. — Output (b) (a) Figure 2: Parallelization schemes for FEM: (a) Parallelization of linear equation solver- BiCG. (b) Task parallelization. The generation of the matrix entries in the finite element method is in general a much easier and faster process compare to that of the MoM [2, 3]. Under most practical situations, FEM matrix generation takes less than 10% of total FEM solution time. As a result, the parallelization of linear equation solver may result in an efficient method (refer fig. 2(a)). On the other hand, based on the property of the frequency domain finite element method, each frequency point can be parallelized with minimum inter-process communication overhead which might lead to scalable FEM code as shown in figure 2(b). In the following section, the performance of two different parallelization schemes for FEM is investigated and the speedup curves are presented. III. Numerical Results A. Parallelized MoM Code for Antenna Applications In order to demonstrate the performance gain of the parallelization models discussed in the previous section, the impedance matrix of a 6 x 6 microstrip patch array antenna is calculated and solved by taking the FWT. The individual patch size, the dielectric constant of the substrate, and the thickness of the substrate is 59.5 mm x 50 mm, 2.62, and 1.59 mm, respectively. The rooftop function is used as the basis function of the field expansion. The number of cells are 16 x 16 on each patch and the resulting total number of unknowns is 17280 for the whole array. The radiation patterns for Ek and Es are calculated at 1.536 GHz. Figure 3(a) and (b) show the measured computation time v.s. number of CPUs and theoretical result for both the impedance matrix filling process and FWT using the models presented in Section II. As can be seen from the figures, the measured result matches very well to the theoretical prediction. The radiation patterns of the antenna are computed as shown in figure 4 for Ed and Es plane patterns. The thresholding is taken after 3

the FWT and the resulting sparse matrix is solved with iterative linear equation solver, such as biconjugate gradient method. Impedance Matrix Fill, 240 matrix elements Fast Wavelet Transform, 3568 matrix elements 1200,,.,,, win I I I I I I 1000. c 0 E.o0 V c 600 0 c ' 400 I -- Measured --- Theory Time(i) = P + Comm x N + Comp x N /(i - 1), for i >1 P: preprocessing time. l, Comm: commnunication time. [\ Comp: computation time. ~\ i: number of CPUs. N: number of matrix elements. -- ius ' - E 0 4 E3 s - Measured --- Theory Time(i) = 0 + Comm x N + Comp x N / i 0: overhead. Comm: communication time. Comp: computation time. i: number of CPUs. N: number of matrix elements. 1[ 200[ 1 UJ.... I.I.... 1 5 10 15 20 25 30 Number of CPUs Note: the turn arwound time of 2 CPUs is higher than 1 CPU. This Is due to the higher preprocessing time of parallelized program and the communication cost. Also note that, due to the programming model, one CPU is always reserved for coordinating other CPUs, which leaves only one CPU to perform the computation when only two CPUs are used. (a) Figure 3: Parallelization per ansfor MoM: wavelet transform. 2 4 6 8 10 12 14 16 18 Number of CPUs Note: the overhead includes the extra function calls for measuring time and parallel processing. (b) (a) Parallel impedance matrix generation, (b) Parallel fast A. Parallelized FEM Code for Packaging Applications To measure the performance of the parallelized Bi-CG routine on distributed memory machine, two different size of problems are solved. As shown in figure 5(a), parallelization efficiencies are minimal, even though the larger problem size shows slightly better result. With this observation, the task parallelization scheme is tested as shown in figure 5(b) and reveals excellent result. The poor performance of the parallel Bi-CG routine is mainly due to heavy communication overhead. On the other hand, the linearly scalable performance of the task parallelization scheme is coming from the minimum data dependency between the CPUs. As an application of the parallelizaed FEM code, we have characterized two typical microwave/millimeter-wave packages and the internal electric field distributions are calculated as presented in figure 6. IV. Conclusions In this paper, single ingle instruction multiple data (SIMD) model have been fully implemented for the finite element method and the method of moment using the MPI standard on distributed memory machine. Due to the data independence between the processes, linearly scalable parallel codes are achieved. In particular, matrix element generation in MoM has been parallelized in view of its heavy computational burden. In the FEM code, frequency parallelization scheme has been implemented after the observation that the performance of the parallelized linear equation solver has been poor. 4

E, (I - 0) Piaem to( 6x6 Patch Arramy E (* = 90) Pattern tot 6x6 Patch Array [.. -I. -I I.. Threho = 1 0 - -10 I.eal Array -20 " / - 2-30 - " - -40 - _ 0o - -40 I -60 - -so. 0 60 -80 - _ -70 - I Thraeshold15 i. Ideal Array I -----— ~ ---81 I, -80- I-100 * *. -90 -120 -100 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 (deg] e (deg] (a) (b) Figure 4: Radiation patterns for 6 x 6 rectangular patch array: (a) Eq pattern, (b) E~ pattern Acknowledgment This work was partially supported by ARL under contract QK8820. Also, the authors would like to thank the Maui High Performance Computing Center (MHPCC) and the University of Michigan Center for Parallel Computing (CPC), which is partially funded by NSF grant CDA-92-14296 and the Ford Motor Company, for the use of their computational facilities. References [1] Kazem F. Sabet, Jui-Ching Cheng and Linda P. B. Katehi, "Efficient Wavelet-based Modeling of Printed Circuit Antennas and Arrays," submitted to IEEE Transactions on Antennas and Propagation. [2] Jong-Gwan Yook and Linda katehi, "Characterization of MIMICs Packages Using A Parallelized 3D FEM Code," 1996 ACES Symposium, Monterey, CA., pp. 954-961. [3] Jong-Gwan Yook and Linda katehi, "Parallelization of the Finite Difference Time Domain Code on Shared Memory Machine," PIERS 1996, Innsbruck, Austria. 5

1600 1400 1200 1000. 77 <u.-1 v p p 800 " I I T! I I I o -. -- M=38109 ------ M= 76709...... --- - 3 3 O E C-. 600 400 200 |,............... I. I 0 2 4 6 8 10 12 14 16 18 20 0 5 10 15 20 25 30 35 Number of Processors Number of Processors (a) (b) Figure 5: Parallelization performances for FEM ("M" is the number of unknowns to be solved): (a) Parallel Bi-CG routine, (b) Frequency parallelization scheme. Microstrip / L.J.-.P~ --- —-— ~ ---- 4 ---,, --- —-------— r!? i:: ma "m i I Lie -.. - -- X Zi,f f f: s: 00: f V / yf f f. / /;fX f / / / f / /:/,/ H Substrate ir L I.. V (a) (b) Figure 6: Two different packaging structures and electric field distributions inside of the structures. 6

DoD Mission Success Story from High Performance Computing: 1998 ed." PARALLEL ELECTROMAGNETIC SOLVERS FOR HIGH FREQUENCY ANTENNA AND CIRCUIT DESIGN J.-G. Yook, J. Cheng and L. Katehi Radiation Laboratory Department of Electrical Engineering and Computer Science The University of Michigan, Ann Arbor, MI 48109-2122, U. S. A. Tel: 313-764-0608. Fax: 313-647-2106 DoD partner: Barry S. Perlman, Army Communications - Electronics Commands. Computer Resource: IBM SP2 [MHPCC] Research Objective: The purpose of this effort is development of linearly scalable parallel electromagnetic solvers for large array antenna and complicated microwave/millimeter-wave circuits which have not been successfully characterized by other means. Fast and accurate characterization of these electromagnetic structures are of critical importance for a successful DoD mission in digitizing the battlefield and developing secure communication systems. Methodology: For parallel electromagnetic tools, we have developed wavelet-based method of moment (MoM) and tetrahedral finite element method (FEM) codes for arbitrary three-dimensional structures, and a message passing interface (MPI) has been utilized for parallelization on distributed memory parallel computers. Furthermore, fast wavelet transform and thresholding techniques are applied to achieve accelerated computation of the MoM solution. The linear scalable performance of the parallelized MoM and FEM is also investigated for real time design and optimization of the large array antenna and high frequency circuits. Results: Preliminary results of the parallel impedance matrix filling and fast wavelet transform for MoM and task parallelization strategy for FEM show linearly scalable performance improvement. This truly scalable parallel MoM and FEM code performs successfully due the minimal communication overhead between the computing nodes and is not subject to the bandwidth of the network or switches. The number of unknowns of the problem ranges from 15,000 to 20,000 in the MoM case and from 150,000 to 200,000 in the FEM case. Figures below shows three-dimensional electric field distributions at far region from the 6x6 rectangular patch array antenna which are computed on parallel computers very efficiently. Significance: Design and optimization of large electromagnetic structures, such as array antennas and high density circuits and packages, are of critical importance in the development of the advanced sensors and tactical communication systems. Through our research, we have demonstrated linearly scalable parallel strategies for various numerical electromagnetic codes and solved various class of problems which were not possible or extremely time consuming with conventional method and computers. This project has been funded by NASA Lewis Research Center, Army Research Office and the DoD High Performance Computing Program. : -:m Figure: Far field radiation patterns for 6x6 rectangular patch array antenna. Authors: Jong-Gwan Yook, Juiching Cheng and Linda P.B. Katehi in the University of Michigan. Tel: 313-764-0608.

Characterization of Plated Via Hole Fences for Isolation Between Stripline Circuits in LTCC Packages George E. Ponchak', Donghoon Chen', Jong-Gwan Yook2, and Linda P. B. Katehi',_ Z:._ I. 2. NASA Lewis Research Center, 21000 Brookpark Rd., MS 54/5, Cleveland, OH 44135 University of Michigan, 3240 EECS Building, Ann Arbor, MI 48109-2122 Abstract Reduced coupling between adjacent striplines in LTCC packages is commonly accomplished by walls made of plated via holes. In this paper, a 3DFEM electromagnetic simulation of stripline with filled via fences on both sides is presented. It is shown that the radiation loss of the stripline and the coupling between striplines increases if the fence is placed too close to the stripline. Introduction Smaller packages with more circuitry are required for the advanced RF systems being built today and planned for tomorrow. These new packages house a variety of high density circuits for data processing, biasing, and memory in addition to the RF circuits. While the size is being reduced and the complexity increased, the cost of the package must also decrease. To accomplish these contradictory goals, new packaging technologies are required. Low Temperature Cofired Ceramics (LTCC) is an ideal packaging technology. The material has a moderate dielectric constant, 4<Er<8, which permits wider strips and thus lower conductor loss than circuits on Si, GaAs, or Alumina. In addition, the loss tangent is on the order of 0.002 at 10 GHz which yields an acceptably low dielectric loss. LTCC packages are comprised of many 0.1-0.15 mm thick ceramic layers with transmission lines on each layer. This increases the level of integration by allowing bias, digital routing, and RF transmission lines and interconnects to be built up in three dimensions. The multilayer character of these circuits leads to RF transmission lines which are not of microstrip type but of stripline type. Even with the high levels of integration LTCC offers, designers are required to decrease the spacing between striplines to meet the new size and cost requirements. In doing so, coupling between adjacent striplines severely limits the overall packaged circuit performance. As a solution to this problem, package designers have included filled via fences adjacent to the stripline to confine the electromagnetic fields around the center strip, Ah Metal-filled Ground Plan 4 -- h Substrate (a) Metal-filled Via (b) Fig. 1 (a) Stripline with a continuous filled via fense on both sides (b) Cross section. partition the package and reduce electrical couplings [1-2]. In this paper, we utilize a 3D-Finite Element Method (FEM) [3] to evaluate stripline structures in the vicinity of filled via fences. It is shown for the first time that filled via fences increase the radiation loss of the stripline and decrease the coupling between adjacent striplines. Design guidelines are given to minimize these parasitic effects which degrade overall performance. Results The cross section of the stripline structure is shown in Figure 1. Throughout this paper: the relative permittivity of the LTCC material is 5.3; the via diameter, Dv, is 0.25 mm; the width, W, of the stripline is 0.19 mm to yield a 50 Ohm characteristic impedance; and the thickness, h, is 0.25 mm. The distance between the stripline and the vias is kept greater than 0.25 mm and the via-to-via spacing ranges between 1.3 and 5 times the via diameter. These values are all standard for typical LTCC

'E b:7 --— iS! (A h = 5.2 h) -------- Rad. Lossl! A h = 52 h 0 -o- S h = 1.3h h - -— o --- Rad. Lossl A h = 1.3 h -1::-:::: ---—.-.............. - - - - - - - --- -) - - - - - -20i - ------------—.. -25 --30 _ -35 11 '44S~.. C '4 — ~.-X "95,;S ~ G~.- V'f.-50 -53 I lip-. -4U,.I ---.. 1 2 3 4 AS /h (a) I0.,.. Fig. 3 Electric field distribution of stripline with a continuous via fence at 25GHz with AS/h= I and Ah/h=5.2. -0.2 --0.4 --0.6 --0.8 -1.0 --— IS,21 (Ah=5.2h) -0 —IS,,I (Ah=1.3h) 0 -5 -10 -20 -25 -30 -35 -40 -50 -55, I I 2 3 *I1u1l1"Mompm~~sllrm n ~~ Milr i nuIlIIImY"n3 ~I I -'Mmr'I AS/h (b) Fig. 2 (a) Return loss and radiation loss, I-IS112_-IS2I2, of stripline with a continuous via fence as a function of Ah/h and AS/h, (b) Insertion loss. processes and thus make these results directly applicable to the packages being designed today. The first structure investigated is a single stripline with a via fence on both sides of the strip as shown in Figure 1. This is a typical structure to ensure suppression of the parasitic parallel plate waveguide modes. The transmission lines are first analyzed over the frequency range of 10 to 40 GHz. It is found that the scattering parameters do rnot vary by more than a few percent over this frequency range as long as 2AS+W<Xd/2 where Xd is the wavelength in the dielectric. This condition is necessary to avoid the excitation of dielectric filled rectangular waveguide modes. Based on this finding, the results are presented as a function of the line geometry with the average of the scattering parameters plotted. Figures 2 (a) and (b) show the magnitude of SI, S21, and the radiation loss, l-ISi112-IS2112, as a function of the transmission line geometry. It is seen that the reflection coefficient, the insertion loss, and the radiation loss all decrease as the separation between Fig. 4 Electric field distribution of stripline with a continuous via fence at 25GHz with AS/h= I and Ah/h= 1.3. the strip and the via holes, AS/h, increases. Reduction of the parasitic effects levels off as AS/h approaches 2. Furthermore, it is seen that the insertion loss decreases as the distance between the via holes decreases. Field distribution for a stripline with large narrow via hole spacing are shown in Figures 3 and 4 respectively. These plots indicate that the closely spaced via fence completely confines the electromagnetic field while the wider spaced via holes permit a significant leakage of power. Note that AS/h= I in both of these field plots. In case where we need to locally improve isolation, the continuous via fence may be replaced by a short via fence as shown in Figure 5 [2]. Figures 6 (a) and (b) show the simulated performance for this line. It is seen that the scattering parameters and radiation loss vary with AS/h in the same way as the continuous via fence. However, upon comparison with Figures 2 and 3, there is 5 dB degradation in the line characteristics. Figure 7 shows that this is due to

Ah PEC / ^^ / 1 - tIII Metal-filled Via /A I. I U.dl -- — m I 11 if If 11 11 I M I A-i L-2 L-A &-A A-i IL-i co 00000 nhgtrate Ground Pla E -t AL. - h I p I. ADWILI i It I11 I I -s 1-iC -15 -25 -3S -35 -40 -45 -so -ss ane....,,... I ] k....., b::S L-r.......;-...~~. r1~ Fig. 5 Stripline with a short section of a filled via fence on both sides. -| - IS,SI (Ah= 1.3h) -— o- IRad. Lossl (A h = 1.3 h) 0I... -5 - -10 -15 Fig. 7 Electric field distribution of stripline with a short via fence at 25GHz with AS/h=1 and Ah/h=1.3. AS -20 --25 --30 --35 --40 - m..m_._.__, 2 Z AS/h (a) 0.(, I, I u,-) = to ct e~ -0.2 --0.4 --0.6 --0.8 - - -*-IS,,I Ah=1.3h -,1.. 2 Fig.8 Adjacent striplines seperated by via fences. A S / h (b) Fig. 6 (a) Return loss and radiation loss, l-ISl12-lS2, of stripline with a short via fence as a function of AS/h, Ah/h= 1.3, (b) Insertion loss of stripline with a continuous via fence as a function of AS/h, Ah/h= 1.3.

the perturbation of the fields upon reaching the via fence on either side of the strip. The electromagnetic fields that escape the confinement of the via fences and result in radiation loss cause coupling between adjacent striplines as shown in Figure 8. Figure 9 shows the magnitude of the total electric fields for two striplines separated by a via fence with AS/h=0.615 and Ahlh=0.764 (h=0.7366mm). The stripline on the left has an imposed voltage on it while the stripline on the right is coupled. Separation of the field into its components shows that electric field in the plane of the strips is approximately 10 dB greater than the electric field normal to the strips near the via fence. The large magnitude of this parallel field component is due to the small via size and large value of Ah/h, resulting -10 dB coupling between the strips. Although perfect conductors were assumed in this study, the current distribution on the strip with a close via fence is expected to cause an increase in conductor loss compared to the conventional stripline case. Conclusions It has been shown that when filled via fences are placed too close to the stripline, radiation loss increases resulting in coupling between adjacent striplines. To minimize the radiation loss, the via fence should be kept at least four times the ground plane separation away from the strip, or AS/h>2. Furthermore, the use of via fences locally only where isolation is required, can actually degrade the stripline characteristics by causing a large perturbation in the electric fields that normally extend on either side of the strip. Acknowledgement The authors would like to thank the University of Michigan Center for Parallel Computing (CPC), which is partially funded by NSF grant CDA-92 -14296 and Ford Motor Company, for the use of computational facilities. References 1. J. W. Gipprich, "EM modeling of via wall structures for high isolation stripline," Digest for Workshop WWFC, Interconnect and Packaging Technologies and Issues, IEEE Int. Microwave Symp., San Diego, CA, pp. 78-114, June 1994. 2. J. Gipprich, C. Wilson, and D. Stevens, "Multilayered 3-D packaging issues," Digest for Workshop WFC, Interconnects and Packaging for RF Wireless Communications Systems, IEEE Int. Microwave Symp.. Denver. CO. June 8-13. 1997. 3. J.-G. Yook, "Electromagnetic modeling of highspeed high-frequency Interconnects," Ph. D. dissertation, University of Michgan, Ann Arbor, MI, 1996. -20 -30 -40 -so -70 Fig. 9 Electric field distribution of two striplines separated by a continuous fence f=10GHz, AS/h=0.45, and Ah/h=0.5).

"presented to Applied Computational Electromagnetics Society(A CES) Conference. March 17-'1 1997" A GENERALIZED METHOD FOR INCLUDING TWO PORT NETWORKS IN MICROWAVE CIRCUITS USING THE FINITE ELEMENT METHOD Eray Yasan, Jong-Gwan Yook and Linda P. Katehi Radiation Laboratory Dept. of Electrical Engineering and Computer Sci. The University of Michigan, Ann Arbor, MI 48109-2122, U.S.A. Tel: 313-764-0502, Fax: 313-747-2106, Email: eray@engin.umich.edu Abstract This paper discusses the methodologies required to introduce basic passive elements into an FEM modelled microstrip circuit. Three different approaches have een investigated and are presented herein. The first two methods use the basic principles of circuit theory while the third employs the so called zeroth order approximation or volume current method. One of the EM-circuit method uses the voltage-current relations applying at the passive element(s) in the circuit. This is done through the impedance of that, element and requires a modification of the functional of the FEM equations. The second method uses the S-parameters of the passive element(s) to provide the required circuit relation. By using these methods the effect of the presence of the passive element(s) on the microstrip circuit can be observed. 1 Introduction The Finite Element Method (FEM) has been established during the past ten years as an accurate and versatile frequency-domain technique for passive circuit problems. Despite the capability of the technique to treat a broad variety of circuit geometries, it has been limited to only distributed elements that are mostly passive and linear. To be able to make the technique applicable to more complete microwave and millimeter wave circuits, its capability needs to be extended to handle passive and active elements. Some studies about these issues have been made by using different techniques such as FDTD, TLM and FEM [1]-[3]. The techniques presented herein are divided into circuit element methods and volume current methods. In the following, a short description of each technique is given.

I, I2 + I; ---- + V, [Z] V2 - 2 — o -- 0 — Figure 1: Two port network with Z parameters 2 FEM Formulation Starting with Maxwell's equations, the following wave equation is weighted according to Galerkin's method and discretized V x V x E - C2tcEE =-jWiJi (8) where Ji is the impressed current source. Discretization is done in a conventional way using tetrahedral elements. The following weak form of Maxwell's equations is obtained //j [(V x E) (V x P)- 2/eE P]dv =,jwi(H x P) nds- // juP Jidv. (9) Expanding the electric field as a summation of linear basis -functions and choosing the weighting function as the same edge basis function give 6 E = ZaiWi (10) i=1 P =Wj, j = 1,..., 6 (11) where ai are the unknowns to be determined, and W's are the first order edge based Whitney functions. For the circuit method #1 the first term on the right hand side of the FEM equation (9) is expressed in such a way that the effect of the lumped element is included. The resulting term in the FEM equations becomes -~ [e12 Eb(r - r,)][j E d\]ds (12) ZL where e12 is the unit vector pointing from node 1 to node 2 along which the element is placed and ZL is the load impedance. According to the circuit method #2, having additional set of equations from I - V relations enables us to write another matrix equation having N unknowns not necessarily being equal to the number of additional equations. By careful examination, elimination of some of the

Millirreter-Wave Computer-Aided Engineering. Vol.3. No. 3 pp. 23S-250. 199!3. [2] V.A. Thomas. M.E. Jones.M. Piket-Mla. A. Taflove and E. Harrioan.l"Thle use of SPICE lumped circuits as sub-grid models for FDTD analysis." IEEE Microwave Guided N7Wave Ltters.vol. 4. pp. 141-143. May 1994. [3] A. Jahnsen and Xr. Hansen.' Spectral analysis of multiport microstrip circuit with active and passive lumped elements."EMIC, p. 1053,1990. [4] R.H. Voelker and R. Lomax, "A Finite Difference Transmission Line Matrix method incorporating a nonlinear device model," IEEE Trans. Microwave Theory Tech.. vol.38. March 1990,pp.302-312. [5] K. Guillouard, M.-F. Wong, V. Fouad Hanna, J. Citerne, "A new global Finite Element analysis of microwave circuits including lumped elements," Digest of the 1996 IEEE Microwave Theory Tech. Symposium,vol. 1. [6] E. Yasan, J.-G. Yook and L. Katehi, "Modeling the interface between fields and devices using the Finite Element Method."' Digest of 1996 Progress In Electromagnetic Research Symposium (PIERS). [7] Jong-Gwan Yook "Electromagnetic modeling of high-speed high frequency interconnects. PhD ThesisThe University of Michigan 1996.

PEC PEC PEC Figure 4: Shielded microstrip circuit with a PEC wire between the conductors '. I --,- r _ _ -' _ I ' - __ -- __ _ -.-.. --- — -- -- -- - -'- -- -5 -10 -15 S -20 cu n-25 oa n -30 -35 -40 -45 nr S 11(-.) Circuit Method #2 S11(-) Regular FEM S12(o) Circuit Method #2 S12( —) Regular FEM -! h,..I 15 frequency (GHz) 20 25 Figure 5: S-parameters of the shielded microstrip circuit with a PEC wire

'submitted to U'RSI-98'" An FEM Based Method on Full Wave Characterization of Distributed Circuits Including Linear and Nonlinear Elements Eray Yasan- and Linda P. Katehi Radiation Laboratory Dept. of Electrical Engineering and Computer Sci. The University of Michigan, Ann Arbor, MI 48109-2122, U.S.A. Tel: 313-764-0502, Fax: 313-747-2106, Email: eray@engin.umich.edu Theoritical characterization of microwave circuits including linear and nonlinear elements has been an interest in many ways using numerical techniques including Finite Element Method (FEM). Since the lenghts of elements connected to the microwave circuit are very short so that Kirchoff:s las can be applied, the conventional scheme to find S parameters by adding extra line lengths is no more applicable. In this case the circuit is treated as a combination of inner (internal) ports where the linear or nonlinear elements are connected and the outer (external) ports where the input and output of the circuit are taken. Work has been done in time domain using "Compression Approach"'(J. Kunish,M. Rittweger, S. Heinen and I. Wolff,Proc. 21st European Microwave Conf.,1296-1301,1991). The linear passive part of the circuit is analyzed using the FEM. The ports are excited one at a time while the others are left open to derive the appropriate voltages and currents at all ports. The linear part of the circuit is thus characterized by the impedance matrix (Z-matrix). To reduce the computational cost, circuit symmetries are used and assuming circuit lossless, S parameters can be found using the simple relation ([] + [) ([z- [I]) = [S] Following the computation of scattering matrix, a simulation software such as HP EEsof Libra and connecting the linear or nonlinear elements to the inner ports yields the input and output S-parameters of the microwave circuit. As an example a microstrip line with a gap where a lumped element is connected has been analyzed. Simple lumped elements like resistors,capacitors and inductors have been connected and the resulting circuits have been modeled successfully. A diode mixer on a coplanar waveguide (cpw) designed for higher power handling capability has been analyzed using the above method in connection with Harmonic Balance Method (HBM). Results from these studies will be presented and discussed extensively.