Initializing AMReX (v2024-34-g9e8f8d60eca3)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-34-g9e8f8d60eca3) initialized Starting run at 07:51:08 UTC on 2024-03-26. Successfully read inputs file ... Castro git describe: 24.03-26-g5707c0c2f AMReX git describe: v2024-34-g9e8f8d60e Microphysics git describe: 24.03-21-g81de2adb reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.030292927 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.029430677 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.370914137 Gravity::multilevel_solve_for_new_phi() time = 1.778931503 Gravity::fill_multipole_BCs() time = 0.378247012 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.54052218 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.886204296 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793356e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.315215348 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 20.68213273 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.629658486 Gravity::multilevel_solve_for_new_phi() time = 1.670430437 Castro::construct_new_gravity() time = 1.785050827 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.296156101 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02493198123 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 37.65664036 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.03084149 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.414485994 Gravity::multilevel_solve_for_new_phi() time = 1.953568111 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.802808804 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.020067868 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.45146986 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.033159772 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.574830857e+22 ymom added: 1.446224735e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.197162806 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.909616742 on level 1 Castro::reflux() at level 0 : time = 2.300127036 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.474491997 Gravity::multilevel_solve_for_new_phi() time = 1.435229126 Castro::construct_new_gravity() time = 1.547264397 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 2.020627672 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029975588 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764827561e+18 ymom added: -1.361438167e+19 zmom added: -8.219114249e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.143269747 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.0227584817 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.92898959 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031257813 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.413439983 Gravity::multilevel_solve_for_new_phi() time = 0.54312096 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.716574159 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.032820307 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.70864749 on level 0 Castro::construct_old_gravity() time = 0.032499275 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.44217652e+22 ymom added: 4.427218578e+21 zmom added: 7.103293505e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.197516034 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.931536329 on level 1 Castro::reflux() at level 0 : time = 2.312443924 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.473503375 Gravity::multilevel_solve_for_new_phi() time = 1.033767545 Castro::construct_new_gravity() time = 1.144351403 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.028451955 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.030411191 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.813673386e+17 ymom added: 2.352230085e+19 zmom added: 9.539392267e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.149037465 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02285267595 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.76211171 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.030962971 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.409026905 Gravity::multilevel_solve_for_new_phi() time = 0.537130256 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.712438446 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 1.973371217 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.25493288 on level 0 Castro::construct_old_gravity() time = 0.037519187 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.20092497e+22 ymom added: 1.239621202e+22 zmom added: 8.023757211e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.196066645 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.917204137 on level 1 Castro::reflux() at level 0 : time = 2.264835184 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.467321928 Gravity::multilevel_solve_for_new_phi() time = 1.022137269 Castro::construct_new_gravity() time = 1.132958845 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.967248218 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029254205 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598476879e+18 ymom added: -2.567952508e+19 zmom added: -1.336957926e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.140694945 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02328599142 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 46.96226728 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.031417462 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402817764 Gravity::multilevel_solve_for_new_phi() time = 0.532151391 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.702159599 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 1.985794536 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.62015415 on level 0 Castro::construct_old_gravity() time = 0.038838465 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013392062e+23 ymom added: 1.770887431e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.195672143 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.924488636 on level 1 Castro::reflux() at level 0 : time = 2.363906019 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.480601433 Gravity::multilevel_solve_for_new_phi() time = 1.041759769 Castro::construct_new_gravity() time = 1.152699421 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171514e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.012805775 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029264486 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727187202e+18 ymom added: -2.54093091e+19 zmom added: -1.252339167e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.142030525 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02297613632 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.49044855 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.535093359 seconds Ending run at 07:55:10 UTC on 2024-03-26. Run time = 241.8162751 Run time without initialization = 230.3373614 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 241.8 ... 241.8 ... 241.8 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 93.4406 103.6457 107.6791 44.53% Castro::computeTemp() 88 14.99 16.86 17.52 7.25% check_for_negative_density() 9 0.7285 4.767 15.06 6.23% amrex::Copy() 679 12.25 13.72 14.39 5.95% MultiFab::AddProduct() 4205 10.64 11.86 12.35 5.11% Castro::normalize_species() 87 10.13 11.3 11.89 4.92% FabArray::ParallelCopy_finish() 830 1.638 4.093 10.72 4.43% Castro::reset_internal_energy(Fab) 1476 8.364 9.327 9.69 4.01% Castro::estTimeStep() 26 4.023 5.344 8.904 3.68% FabArray::setVal() 5070 7.119 7.978 8.332 3.45% MultiFab::Subtract() 175 6.821 7.641 7.998 3.31% Castro::reset_internal_energy(MultiFab) 88 3.341 4.099 4.911 2.03% Gravity::fill_multipole_BCs() 11 4.319 4.513 4.842 2.00% FabArray::sum() 4205 3.542 3.954 4.114 1.70% FluxRegister::Reflux() 24 2.94 3.359 3.67 1.52% FabArray::ParallelCopy_nowait() 830 2.121 2.262 2.609 1.08% MLPoisson::Fsmooth() 1376 2.14 2.383 2.505 1.04% Castro::enforce_min_density() 87 1.539 1.916 2.475 1.02% Castro::reflux() 4 1.49 1.687 1.789 0.74% Castro::FluxRegCrseInit 9 1.453 1.616 1.714 0.71% Castro::wd_update() 6 0.4879 0.8695 1.689 0.70% FillPatchInterp(MF) 30 1.289 1.375 1.484 0.61% Amr::InitAmr() 1 1.191 1.191 1.191 0.49% Castro::initMFs() 5 1.102 1.136 1.155 0.48% amrex::Add() 136 0.9842 1.101 1.154 0.48% Castro::construct_old_gravity_source() 18 0.8872 1.002 1.073 0.44% MLPoisson::Fapply() 318 0.9095 1.012 1.052 0.43% Castro::volInBoundary() 42 0.7271 0.8117 0.9434 0.39% Castro::construct_new_gravity_source() 9 0.7815 0.8747 0.9319 0.39% Castro::problem_post_timestep() 10 0.8165 0.845 0.9155 0.38% FillBoundary_finish() 1553 0.2277 0.4987 0.892 0.37% FillPatchIterator::Initialize 62 0.7536 0.8277 0.8613 0.36% FabArray::Saxpy() 182 0.7234 0.8082 0.8409 0.35% VisMF::Write(FabArray) 3 0.5907 0.6631 0.7172 0.30% StateData::FillBoundary(geom) 50 0.5782 0.594 0.6052 0.25% Castro::construct_new_source() 63 0.4863 0.5352 0.5611 0.23% MLCellLinOp::applyBC() 1742 0.3572 0.4348 0.5049 0.21% MultiFab::contains_nan() 18 0.4009 0.4473 0.466 0.19% FillBoundary_nowait() 1553 0.3251 0.4048 0.4624 0.19% FabArray::Xpay() 283 0.3486 0.3894 0.4056 0.17% MultiFab::max() 18 0.01889 0.1181 0.3947 0.16% Castro::initialize_do_advance() 9 0.2346 0.2595 0.2731 0.11% Amr::writePlotFile() 2 0.08159 0.1451 0.2449 0.10% FabArray::setDomainBndry() 92 0.1896 0.2136 0.2266 0.09% amrex::average_down 233 0.1963 0.2066 0.2125 0.09% Castro::construct_old_source() 63 0.1813 0.1976 0.2062 0.09% Castro::fill_hybrid_hydro_source() 27 0.1661 0.1855 0.1939 0.08% MLCellLinOp::defineAuxData() 11 0.1547 0.1698 0.1773 0.07% Castro::initData() 1 0.125 0.1452 0.1639 0.07% MLPoisson::FFlux() 150 0.004728 0.04165 0.1522 0.06% MLCellLinOp::compGrad() 18 0.1255 0.1399 0.1461 0.06% FabArray::mult() 56 0.1152 0.1288 0.1342 0.06% MLMG::addInterpCorrection() 155 0.1095 0.1219 0.1267 0.05% StateData::define() 25 0.1074 0.1172 0.1215 0.05% Castro::evaluate_source_change() 193 0.09978 0.1102 0.1153 0.05% Castro::initialize_advance() 9 0.06605 0.0844 0.1105 0.05% Castro::swap_state_time_levels() 9 0.0825 0.09508 0.1101 0.05% Gravity::get_new_grav_vector() 10 0.08696 0.09526 0.09962 0.04% Gravity::get_old_grav_vector() 9 0.07866 0.08715 0.09222 0.04% MLMG::prepareForSolve() 10 0.0628 0.06707 0.06963 0.03% MLCellLinOp::updateSolBC() 25 0.06699 0.06773 0.06851 0.03% MLMG::solve() 10 0.0009893 0.04091 0.06574 0.03% FabArray::norminf(mask) 21 0.05386 0.06022 0.06293 0.03% AmrLevel::FillPatch() 62 0.05195 0.05568 0.05795 0.02% Castro::apply_tagging_restrictions() 6 0.0467 0.05257 0.05479 0.02% AmrMesh::MakeNewGrids() 6 0.04548 0.04729 0.05134 0.02% MLMG::ResNormInf() 54 0.001648 0.01573 0.04916 0.02% MLCellLinOp::updateCorBC() 17 0.04543 0.04597 0.0468 0.02% BndryData::define() 27 0.04235 0.04369 0.04538 0.02% FabArray::norminf() 147 0.03809 0.04034 0.04264 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.0295 0.03395 0.03742 0.02% MLMG::MLResNormInf() 15 9.809e-05 0.02322 0.03398 0.01% MLCellLinOp::reflux() 30 0.004378 0.01187 0.03393 0.01% Castro::do_new_sources() 9 0.0005533 0.008644 0.03276 0.01% Castro::FluxRegFineAdd() 9 0.02946 0.03055 0.03231 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.02073 0.02341 0.02888 0.01% MultiFab::Multiply() 8 0.02183 0.02467 0.02674 0.01% MLCellLinOp::defineBC() 11 0.02581 0.02621 0.02667 0.01% MLLinOp::makeSubCommunicator() 11 0.0004904 0.009324 0.02297 0.01% Castro::build_fine_mask() 44 0.01749 0.01885 0.01969 0.01% MLMG::interpCorrection_1 17 0.0185 0.01906 0.01947 0.01% FabArrayBase::CPC::define() 416 0.01346 0.01466 0.01768 0.01% Castro::Castro() 5 0.01468 0.01646 0.01722 0.01% Castro::buildMetrics() 5 0.0151 0.01636 0.01703 0.01% StateDataPhysBCFunct::() 95 0.00207 0.008409 0.01671 0.01% Castro::linear_to_hybrid_momentum() 1 0.01378 0.01553 0.01614 0.01% Castro::construct_old_gravity() 9 0.0004179 0.005237 0.01523 0.01% FillPatchTwoLevels 31 0.01315 0.01382 0.01434 0.01% MLCellLinOp::prepareForSolve() 11 0.01173 0.01189 0.01215 0.01% Castro::enforce_consistent_e() 1 0.009347 0.0105 0.01094 0.00% Gravity::actual_solve_with_mlmg() 11 0.009483 0.009696 0.01023 0.00% FabArrayBase::getCPC() 1440 0.009682 0.01002 0.01017 0.00% AMRErrorTag::operator() 6 0.008607 0.00966 0.01004 0.00% Castro::finalize_do_advance() 9 0.007723 0.008595 0.009683 0.00% Gravity::actual_multilevel_solve() 10 0.007724 0.008941 0.009675 0.00% Gravity::update_max_rhs() 10 0.007179 0.008358 0.00898 0.00% Castro::apply_problem_tags() 6 0.006331 0.007101 0.007384 0.00% average_down_faces 48 0.005675 0.005834 0.006087 0.00% MLCellLinOp::setLevelBC() 19 0.00401 0.00487 0.006046 0.00% DistributionMapping::LeastUsedCPUs() 5 0.001561 0.00359 0.00557 0.00% FabArray::FillBoundary() 1553 0.004354 0.004496 0.004798 0.00% AmrLevel::derive() 6 0.004194 0.004261 0.0043 0.00% MLCellLinOp::smooth() 688 0.003862 0.003984 0.004151 0.00% MLCGSolver::bicgstab 7 0 0.0009417 0.003767 0.00% Gravity::test_composite_phi() 1 0.002135 0.002723 0.003675 0.00% amrex::Dot() 115 0 0.0008934 0.003574 0.00% FabArrayBase::getFB() 1553 0.003065 0.003233 0.00353 0.00% Castro::do_old_sources() 9 0.0006203 0.001286 0.003211 0.00% FabArrayBase::FB::FB() 19 0.002585 0.002845 0.003093 0.00% FabArray::ParallelCopy() 830 0.002908 0.002961 0.003005 0.00% Castro::construct_new_gravity() 9 0.0004557 0.0007927 0.00173 0.00% main() 1 0.001531 0.001591 0.001626 0.00% MLLinOp::defineGrids() 11 0.001296 0.00147 0.001578 0.00% Castro::clean_state() 87 0.001104 0.001238 0.001458 0.00% MLCellLinOp::apply() 318 0.0009157 0.001073 0.001454 0.00% Amr::regrid() 5 0.0009737 0.001111 0.001352 0.00% TagBoxArray::collate() 6 0.0003697 0.0008278 0.001231 0.00% MLCGSolver::ParallelAllReduce 183 0 0.000278 0.001112 0.00% Amr::coarseTimeStep() 5 0.0005101 0.000931 0.001076 0.00% AmrLevel::FillCoarsePatch() 5 0.000914 0.0009424 0.0009777 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008941 0.0009052 0.0009214 0.00% OwnerMask() 6 0.0007041 0.0008258 0.0009141 0.00% MLCellLinOp::correctionResidual() 189 0.0006684 0.0007033 0.0007313 0.00% Amr::grid_places() 6 0.0003978 0.0004934 0.0006286 0.00% MLMG::mgVcycle() 65 0.0005928 0.0006074 0.0006282 0.00% FillPatchSingleLevel 92 0.000533 0.0005482 0.0005643 0.00% Castro::do_advance_ctu() 5 0.0004759 0.0005063 0.0005511 0.00% Castro::finalize_advance() 9 0.0002619 0.0003738 0.0004815 0.00% TagBoxArray::mapPRD 6 0.000427 0.0004457 0.0004677 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003666 0.0003931 0.0004366 0.00% MLMG:computeResOfCorrection() 155 0.0003878 0.0004016 0.0004275 0.00% Castro::subcycle_advance_ctu() 5 5.932e-05 0.0001849 0.0004153 0.00% MLCellLinOp::solutionResidual() 72 0.0003382 0.0003543 0.0003708 0.00% MLMG::oneIter() 31 0.0003414 0.0003538 0.0003659 0.00% MLMG::actualBottomSolve() 31 3.982e-05 0.0001171 0.0003385 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002756 0.000294 0.0003136 0.00% MLMG::mgVcycle_down::0 31 0.000268 0.0002869 0.000299 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002295 0.0002441 0.0002715 0.00% Amr::defBaseLevel() 1 0.0002273 0.0002462 0.0002684 0.00% Castro::post_regrid() 10 0.0001794 0.0002098 0.0002634 0.00% Castro::enforce_speed_limit() 87 0.00025 0.0002549 0.0002627 0.00% Amr::timeStep() 9 0.0001903 0.0002139 0.000257 0.00% Castro::computeNewDt() 8 0.0001046 0.0001444 0.0002532 0.00% MLMG::mgVcycle_down::1 31 0.0002307 0.0002447 0.0002525 0.00% FabArrayBase::TheFPinfo() 25 0.0001599 0.0001852 0.0002246 0.00% MLMG::mgVcycle_bottom 65 0.0002023 0.0002135 0.0002177 0.00% MLMG::mgVcycle_down::2 31 0.0001839 0.0001955 0.0002137 0.00% MLMG::mgVcycle_down::3 31 0.0001853 0.0001959 0.000211 0.00% AmrLevel::AmrLevel(dm) 5 0.0001855 0.0001963 0.0002053 0.00% MLMG::mgVcycle_up::0 31 0.0001766 0.0001864 0.0001945 0.00% MLMG::mgVcycle_down::4 31 0.0001667 0.0001768 0.0001907 0.00% makeSFC 33 0.0001542 0.0001713 0.0001897 0.00% Castro::check_for_nan() 18 0.0001252 0.0001381 0.0001632 0.00% Castro::react_state() 18 0.0001193 0.000139 0.0001605 0.00% MLMG::mgVcycle_up::1 31 0.0001297 0.0001387 0.0001518 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001289 0.0001359 0.0001468 0.00% MLMG::mgVcycle_up::2 31 0.0001243 0.0001311 0.0001412 0.00% FPinfo::FPinfo() 3 9.561e-05 0.0001156 0.0001339 0.00% MLMG::mgVcycle_up::3 31 0.0001211 0.0001259 0.0001317 0.00% MLMG::mgVcycle_up::4 31 0.0001147 0.0001186 0.0001245 0.00% MLLinOp::makeAgglomeratedDMap 11 0.000104 0.0001126 0.0001202 0.00% Castro::errorEst() 6 0.0001094 0.0001116 0.0001165 0.00% Castro::expand_state() 13 0.000108 0.0001118 0.0001158 0.00% FabArray::RecvLayoutMask() 8 9.006e-05 0.0001081 0.0001156 0.00% Castro::advance() 5 9.597e-05 0.0001037 0.0001139 0.00% MLPoisson::define() 11 0.0001002 0.0001064 0.000113 0.00% DistributionMapping::Distribute() 35 0.0001049 0.000109 0.0001115 0.00% Castro::apply_source_to_state() 18 9.305e-05 0.0001033 0.0001105 0.00% Amr::FinalizeInit() 1 3.068e-05 5.281e-05 9.885e-05 0.00% Amr::writeSmallPlotFile() 1 8.845e-05 9.373e-05 9.62e-05 0.00% MLMG::computeResidual() 31 8.54e-05 9.014e-05 9.52e-05 0.00% Gravity::swapTimeLevels() 9 8.14e-05 8.753e-05 9.297e-05 0.00% MLMG::computeMLResidual() 15 7.833e-05 8.664e-05 8.972e-05 0.00% Castro::post_timestep() 9 7.38e-05 7.811e-05 8.873e-05 0.00% Castro::create_source_corrector() 9 4.754e-05 6.045e-05 7.95e-05 0.00% knapsack() 3 5.883e-05 6.885e-05 7.939e-05 0.00% ClusterList::intersect() 1 0 1.965e-05 7.86e-05 0.00% MLMG::miniCycle() 34 6.747e-05 7.157e-05 7.448e-05 0.00% Castro::avgDown(state_indx) 20 6.72e-05 6.899e-05 7.381e-05 0.00% Castro::avgDown() 4 3.078e-05 4.157e-05 7.34e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.41e-05 6.732e-05 7.095e-05 0.00% MLLinOp::define() 11 5.854e-05 6.388e-05 7.021e-05 0.00% Castro::retry_advance_ctu() 5 2.247e-05 3.534e-05 6.952e-05 0.00% MLMG::getGradSolution() 10 6.02e-05 6.164e-05 6.359e-05 0.00% amrex::average_down_w_geom 20 4.851e-05 5.408e-05 5.843e-05 0.00% Castro::construct_new_hybrid_source() 9 4.738e-05 5.115e-05 5.569e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.466e-05 4.943e-05 5.531e-05 0.00% Castro::init(old) 3 4.146e-05 4.606e-05 5.238e-05 0.00% Amr::initSubcycle() 1 4.906e-05 5.035e-05 5.209e-05 0.00% MLPoisson::prepareForSolve() 11 4.432e-05 4.782e-05 4.976e-05 0.00% Castro::post_init() 1 2.388e-05 3.246e-05 4.233e-05 0.00% BoxList::parallelComplementIn() 12 2.471e-05 3.105e-05 4.11e-05 0.00% Gravity::set_mass_offset() 6 2.911e-05 3.502e-05 4.06e-05 0.00% Castro::construct_old_hybrid_source() 9 3.49e-05 3.577e-05 3.776e-05 0.00% AmrMesh-cluster 1 0 9.393e-06 3.757e-05 0.00% MLMG::MLRhsNormInf() 10 3.161e-05 3.319e-05 3.566e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.714e-05 3.098e-05 3.443e-05 0.00% MLMG::compResidual() 1 2.695e-05 2.84e-05 3.092e-05 0.00% Castro::construct_new_rotation_source() 9 2.192e-05 2.382e-05 2.567e-05 0.00% Amr::bldFineLevels() 1 1.729e-05 2.137e-05 2.525e-05 0.00% Castro::init() 1 1.579e-05 1.828e-05 2.515e-05 0.00% Amr::InitializeInit() 1 7.519e-06 1.234e-05 2.49e-05 0.00% BoxList::complementIn 12 2.143e-05 2.286e-05 2.473e-05 0.00% Castro::construct_old_rotation_source() 9 2.162e-05 2.281e-05 2.449e-05 0.00% Castro::computeInitialDt() 2 1.771e-05 2.027e-05 2.299e-05 0.00% Castro::derive() 6 1.948e-05 2.038e-05 2.13e-05 0.00% Amr::init() 1 6.824e-06 8.567e-06 1.31e-05 0.00% ClusterList::chop() 1 0 2.477e-06 9.908e-06 0.00% Amr::initialInit() 1 4.322e-06 5.447e-06 7.119e-06 0.00% knapsack()swap 1 3.398e-06 3.917e-06 4.336e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 241.8 241.8 241.8 100.00% Amr::coarseTimeStep() 5 229.8 229.8 229.8 95.03% Amr::timeStep() 9 228.5 228.5 228.5 94.47% Castro::advance() 5 201 201 201 83.12% Castro::subcycle_advance_ctu() 5 192.3 192.7 193.7 80.09% Castro::do_advance_ctu() 5 192.3 192.7 193.7 80.09% Castro::construct_ctu_hydro_source() 9 118.1 118.2 118.3 48.93% Castro::clean_state() 87 71.37 80.37 83.81 34.66% Castro::computeTemp() 88 45.36 51.17 53.45 22.10% Castro::reset_internal_energy(MultiFab) 88 30.37 34.31 35.93 14.86% Castro::evaluate_source_change() 193 17.91 19.98 20.8 8.60% Castro::enforce_min_density() 87 16.67 18.8 19.76 8.17% Castro::post_regrid() 10 15.85 17.39 17.94 7.42% Amr::regrid() 5 15.73 17.14 17.65 7.30% Castro::initialize_do_advance() 9 14.86 15.09 15.21 6.29% check_for_negative_density() 9 0.7285 4.767 15.06 6.23% amrex::Copy() 679 12.25 13.72 14.39 5.95% AmrLevel::FillPatch() 62 7.731 9.439 13.89 5.74% FabArray::ParallelCopy() 830 3.82 6.417 12.91 5.34% FillPatchIterator::Initialize 62 6.083 7.865 12.47 5.16% MultiFab::AddProduct() 4205 10.64 11.86 12.35 5.11% Castro::normalize_species() 87 10.13 11.3 11.89 4.92% Gravity::solve_phi_with_mlmg() 11 11.33 11.53 11.86 4.90% Gravity::multilevel_solve_for_new_phi() 10 11.03 11.23 11.54 4.77% Gravity::actual_multilevel_solve() 10 11 11.19 11.51 4.76% Castro::do_old_sources() 9 11 11.04 11.11 4.60% Castro::do_new_sources() 9 9.296 10.45 10.9 4.51% FabArray::ParallelCopy_finish() 830 1.638 4.093 10.72 4.43% FillPatchSingleLevel 92 3.825 5.677 10.42 4.31% Amr::init() 1 10.29 10.29 10.29 4.25% Castro::post_timestep() 9 10.15 10.15 10.15 4.20% FillPatchIterator::FillFromLevel0() 31 3.513 5.314 10.02 4.14% Amr::initialInit() 1 9.746 9.746 9.746 4.03% Castro::reset_internal_energy(Fab) 1476 8.364 9.327 9.69 4.01% Castro::reflux() 4 9.017 9.122 9.241 3.82% Castro::estTimeStep() 26 4.023 5.344 8.904 3.68% Castro::initialize_advance() 9 7.331 8.266 8.642 3.57% FabArray::setVal() 5070 7.119 7.978 8.332 3.45% MultiFab::Subtract() 175 6.821 7.641 7.998 3.31% Gravity::actual_solve_with_mlmg() 11 6.916 6.924 6.932 2.87% Castro::construct_new_gravity() 9 6.378 6.571 6.88 2.85% Castro::expand_state() 13 5.081 5.543 6.573 2.72% MLMG::solve() 10 6.405 6.414 6.434 2.66% Amr::FinalizeInit() 1 5.663 5.792 6.146 2.54% MLMG::oneIter() 31 5.201 5.224 5.234 2.16% Gravity::fill_multipole_BCs() 11 4.399 4.59 4.912 2.03% FluxRegister::Reflux() 24 4.464 4.539 4.652 1.92% MLMG::mgVcycle() 65 4.365 4.404 4.425 1.83% FabArray::sum() 4205 3.542 3.954 4.114 1.70% Amr::InitializeInit() 1 3.6 3.953 4.083 1.69% Amr::defBaseLevel() 1 3.6 3.953 4.083 1.69% Castro::finalize_do_advance() 9 1.346 2.086 4.029 1.67% Castro::problem_post_timestep() 10 2.853 3.21 4.019 1.66% Castro::construct_old_gravity() 9 0.6839 1.574 3.953 1.63% Gravity::get_old_grav_vector() 9 0.6687 1.569 3.952 1.63% Castro::computeNewDt() 8 2.176 2.627 3.891 1.61% MLCellLinOp::smooth() 688 3.331 3.351 3.366 1.39% Castro::post_init() 1 2.587 2.682 2.955 1.22% Castro::initData() 1 2.204 2.521 2.674 1.11% FabArray::ParallelCopy_nowait() 830 2.18 2.32 2.67 1.10% Castro::FluxRegCrseInit 9 1.749 1.999 2.644 1.09% MLPoisson::Fsmooth() 1376 2.14 2.383 2.505 1.04% Castro::wd_update() 6 1.323 1.681 2.49 1.03% Castro::construct_new_source() 63 1.822 2.026 2.14 0.88% MLMG::mgVcycle_down::0 31 1.925 2.001 2.032 0.84% MLCellLinOp::applyBC() 1742 1.171 1.349 1.722 0.71% FillPatchIterator::FillFromTwoLevels() 31 1.489 1.51 1.534 0.63% FillPatchTwoLevels 31 1.489 1.509 1.534 0.63% FillPatchInterp(MF) 30 1.289 1.375 1.484 0.61% Amr::bldFineLevels() 1 1.256 1.402 1.456 0.60% MLMG::mgVcycle_up::0 31 1.396 1.432 1.447 0.60% Castro::Castro() 5 1.36 1.391 1.423 0.59% FabArray::FillBoundary() 1553 0.6662 0.914 1.365 0.56% MLCellLinOp::apply() 318 1.302 1.327 1.344 0.56% Amr::InitAmr() 1 1.191 1.191 1.192 0.49% Castro::initMFs() 5 1.107 1.141 1.16 0.48% amrex::Add() 136 0.9842 1.101 1.154 0.48% Castro::construct_old_source() 63 0.9081 1.018 1.092 0.45% Amr::writePlotFile() 2 1.076 1.076 1.076 0.45% Castro::construct_old_gravity_source() 18 0.8872 1.002 1.073 0.44% MLPoisson::Fapply() 318 0.9095 1.012 1.052 0.43% MLCellLinOp::solutionResidual() 72 0.9733 1.026 1.05 0.43% Castro::computeInitialDt() 2 0.5098 0.6392 0.993 0.41% Castro::volInBoundary() 42 0.7272 0.8117 0.9435 0.39% Castro::construct_new_gravity_source() 9 0.7815 0.8747 0.9319 0.39% FillBoundary_finish() 1553 0.2277 0.4987 0.892 0.37% FabArray::Saxpy() 182 0.7234 0.8082 0.8409 0.35% MLCellLinOp::correctionResidual() 189 0.8144 0.8241 0.8293 0.34% Gravity::get_new_grav_vector() 10 0.7693 0.779 0.7858 0.32% VisMF::Write(FabArray) 3 0.5907 0.6631 0.7172 0.30% MLMG:computeResOfCorrection() 155 0.6625 0.6948 0.7099 0.29% StateDataPhysBCFunct::() 95 0.5803 0.6024 0.6132 0.25% StateData::FillBoundary(geom) 50 0.5782 0.594 0.6052 0.25% MLMG::computeResWithCrseSolFineCor() 17 0.5756 0.5772 0.5798 0.24% Castro::init() 1 0.4934 0.4942 0.4954 0.20% AmrLevel::FillCoarsePatch() 5 0.4933 0.4941 0.4954 0.20% MLMG::computeMLResidual() 15 0.414 0.4368 0.4944 0.20% FillBoundary_nowait() 1553 0.3309 0.4109 0.4686 0.19% Castro::check_for_nan() 18 0.401 0.4475 0.4661 0.19% MultiFab::contains_nan() 18 0.4009 0.4473 0.466 0.19% Gravity::update_max_rhs() 10 0.07762 0.1745 0.4455 0.18% Gravity::test_composite_phi() 1 0.4196 0.421 0.4237 0.18% MLMG::mgVcycle_down::1 31 0.3137 0.3419 0.411 0.17% FabArray::Xpay() 283 0.3486 0.3894 0.4056 0.17% amrex::average_down 233 0.284 0.3551 0.3969 0.16% MultiFab::max() 18 0.01889 0.1181 0.3947 0.16% MLCellLinOp::reflux() 30 0.1792 0.2304 0.3657 0.15% MLMG::computeResidual() 31 0.3504 0.3583 0.3647 0.15% Castro::init(old) 3 0.359 0.3605 0.3616 0.15% Castro::apply_source_to_state() 18 0.289 0.3234 0.3366 0.14% MLPoisson::define() 11 0.2927 0.2986 0.306 0.13% Castro::react_state() 18 0.2124 0.2416 0.2575 0.11% MLMG::mgVcycle_up::1 31 0.2203 0.2245 0.2287 0.09% FabArray::setDomainBndry() 92 0.1896 0.2136 0.2266 0.09% MLMG::prepareForSolve() 10 0.2136 0.2174 0.2222 0.09% MLCellLinOp::defineAuxData() 11 0.1815 0.1993 0.2075 0.09% Castro::fill_hybrid_hydro_source() 27 0.1661 0.1855 0.1939 0.08% MLMG::mgVcycle_bottom 65 0.1577 0.1651 0.1855 0.08% MLMG::addInterpCorrection() 155 0.1303 0.167 0.1852 0.08% Amr::grid_places() 6 0.1842 0.1843 0.1843 0.08% AmrMesh::MakeNewGrids() 6 0.1836 0.1838 0.1839 0.08% MLMG::getGradSolution() 10 0.1728 0.1766 0.1785 0.07% MLCellLinOp::compGrad() 18 0.1728 0.1766 0.1784 0.07% MLMG::miniCycle() 34 0.1575 0.1582 0.159 0.07% MLPoisson::FFlux() 150 0.004728 0.04165 0.1522 0.06% FabArray::mult() 56 0.1152 0.1288 0.1342 0.06% Castro::construct_new_hybrid_source() 9 0.111 0.124 0.1299 0.05% Castro::avgDown() 4 0.1207 0.1249 0.1279 0.05% Castro::avgDown(state_indx) 20 0.1207 0.1248 0.1279 0.05% amrex::average_down_w_geom 20 0.1206 0.1248 0.1278 0.05% AmrLevel::AmrLevel(dm) 5 0.1076 0.1174 0.1217 0.05% StateData::define() 25 0.1074 0.1172 0.1215 0.05% Castro::errorEst() 6 0.1019 0.1132 0.1175 0.05% MLMG::ResNormInf() 54 0.07411 0.08527 0.1114 0.05% Castro::swap_state_time_levels() 9 0.0825 0.09508 0.1101 0.05% MLCellLinOp::updateSolBC() 25 0.07646 0.08265 0.09305 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08667 0.08941 0.09119 0.04% MLCellLinOp::defineBC() 11 0.08609 0.08794 0.08922 0.04% MLMG::mgVcycle_down::2 31 0.06931 0.07394 0.08221 0.03% MLMG::mgVcycle_up::3 31 0.02526 0.06402 0.07794 0.03% MLMG::MLResNormInf() 15 0.04304 0.06535 0.07354 0.03% Castro::construct_old_hybrid_source() 9 0.05524 0.06162 0.06414 0.03% FabArray::norminf(mask) 21 0.05386 0.06022 0.06293 0.03% MLMG::interpCorrection_1 17 0.02435 0.03592 0.06237 0.03% BndryData::define() 27 0.05865 0.0605 0.06129 0.03% Castro::apply_tagging_restrictions() 6 0.0467 0.05257 0.05479 0.02% MLCellLinOp::updateCorBC() 17 0.04999 0.05194 0.05398 0.02% MLMG::mgVcycle_up::2 31 0.05149 0.05273 0.05341 0.02% Castro::derive() 6 0.0401 0.04371 0.04518 0.02% AmrLevel::derive() 6 0.04008 0.04369 0.04516 0.02% FabArray::norminf() 147 0.03809 0.04034 0.04264 0.02% Castro::buildMetrics() 5 0.03419 0.03829 0.04204 0.02% Gravity::swapTimeLevels() 9 0.03236 0.03627 0.03811 0.02% MLMG::mgVcycle_down::3 31 0.03026 0.03169 0.03478 0.01% Castro::FluxRegFineAdd() 9 0.02946 0.03055 0.03231 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.02073 0.02341 0.02888 0.01% MLMG::MLRhsNormInf() 10 0.02486 0.02766 0.02882 0.01% MLMG::actualBottomSolve() 31 3.982e-05 0.007052 0.02808 0.01% FabArrayBase::getCPC() 1440 0.02357 0.02468 0.02781 0.01% MLCGSolver::bicgstab 7 0 0.006845 0.02738 0.01% MultiFab::Multiply() 8 0.02183 0.02467 0.02674 0.01% MLLinOp::define() 11 0.002248 0.01124 0.02501 0.01% MLLinOp::defineGrids() 11 0.002189 0.01118 0.02495 0.01% TagBoxArray::mapPRD 6 0.003847 0.01115 0.02471 0.01% MLLinOp::makeSubCommunicator() 11 0.0004904 0.009324 0.02297 0.01% MLMG::compResidual() 1 0.01914 0.02068 0.02148 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01911 0.02049 0.02127 0.01% Castro::build_fine_mask() 44 0.01749 0.01885 0.01969 0.01% MLMG::mgVcycle_down::4 31 0.005485 0.009111 0.01849 0.01% FabArrayBase::CPC::define() 416 0.01346 0.01466 0.01768 0.01% Castro::linear_to_hybrid_momentum() 1 0.01378 0.01553 0.01614 0.01% MLMG::mgVcycle_up::4 31 0.004527 0.007093 0.01389 0.01% MLPoisson::prepareForSolve() 11 0.01178 0.01194 0.0122 0.01% MLCellLinOp::prepareForSolve() 11 0.01173 0.01189 0.01215 0.01% average_down_faces 48 0.008856 0.01003 0.01103 0.00% Castro::enforce_consistent_e() 1 0.009347 0.0105 0.01094 0.00% AMRErrorTag::operator() 6 0.008607 0.00966 0.01004 0.00% Castro::apply_problem_tags() 6 0.006331 0.007101 0.007384 0.00% FabArrayBase::getFB() 1553 0.00578 0.006078 0.006533 0.00% MLCellLinOp::setLevelBC() 19 0.00401 0.00487 0.006046 0.00% FabArrayBase::TheFPinfo() 25 0.001835 0.003917 0.005877 0.00% FPinfo::FPinfo() 3 0.001675 0.003732 0.005698 0.00% DistributionMapping::KnapSackDoIt() 3 0.001579 0.003616 0.005576 0.00% DistributionMapping::LeastUsedCPUs() 5 0.001561 0.00359 0.00557 0.00% amrex::Dot() 115 0 0.0008934 0.003574 0.00% FabArrayBase::FB::FB() 19 0.002585 0.002845 0.003093 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007494 0.002997 0.00% TagBoxArray::collate() 6 0.0003697 0.0008278 0.001231 0.00% OwnerMask() 6 0.0007041 0.0008258 0.0009141 0.00% Castro::finalize_advance() 9 0.0002619 0.0003738 0.0004815 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003533 0.0003827 0.000409 0.00% makeSFC 33 0.0002494 0.0002701 0.0002907 0.00% Castro::enforce_speed_limit() 87 0.00025 0.0002549 0.0002627 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001254 0.0001372 0.0001585 0.00% AmrMesh-cluster 1 0 3.152e-05 0.0001261 0.00% FabArray::RecvLayoutMask() 8 9.006e-05 0.0001081 0.0001156 0.00% DistributionMapping::Distribute() 35 0.0001049 0.000109 0.0001115 0.00% Amr::writeSmallPlotFile() 1 8.845e-05 9.373e-05 9.62e-05 0.00% knapsack() 3 6.223e-05 7.277e-05 8.357e-05 0.00% Castro::create_source_corrector() 9 4.754e-05 6.045e-05 7.95e-05 0.00% ClusterList::intersect() 1 0 1.965e-05 7.86e-05 0.00% Castro::retry_advance_ctu() 5 2.247e-05 3.534e-05 6.952e-05 0.00% BoxList::parallelComplementIn() 12 4.614e-05 5.391e-05 6.355e-05 0.00% Amr::initSubcycle() 1 4.906e-05 5.035e-05 5.209e-05 0.00% Gravity::set_mass_offset() 6 2.911e-05 3.502e-05 4.06e-05 0.00% Castro::construct_new_rotation_source() 9 2.192e-05 2.382e-05 2.567e-05 0.00% BoxList::complementIn 12 2.143e-05 2.286e-05 2.473e-05 0.00% Castro::construct_old_rotation_source() 9 2.162e-05 2.281e-05 2.449e-05 0.00% ClusterList::chop() 1 0 2.477e-06 9.908e-06 0.00% knapsack()swap 1 3.398e-06 3.917e-06 4.336e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-34-g9e8f8d60eca3) finalized