Initializing AMReX (v2024-33-g9e71c2cb9714)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-33-g9e71c2cb9714) initialized Starting run at 14:52:33 UTC on 2024-03-25. Successfully read inputs file ... Castro git describe: 24.03-30-g437ebcb19 AMReX git describe: v2024-33-g9e71c2cb9 Microphysics git describe: 24.03-21-g81de2adb reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.032887299 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031291942 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.410089089 Gravity::multilevel_solve_for_new_phi() time = 1.803042394 Gravity::fill_multipole_BCs() time = 0.371908373 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.546253872 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.933167161 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.449362872e+25 ymom added: -1.046793356e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.286751744 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 20.72893671 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.584538023 Gravity::multilevel_solve_for_new_phi() time = 1.808907253 Castro::construct_new_gravity() time = 1.922601202 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.276451383 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02482492436 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 37.69772142 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.031663833 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.416181849 Gravity::multilevel_solve_for_new_phi() time = 1.908616172 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.811715629 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 1.98950636 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.23944346 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.031315119 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.574557263e+22 ymom added: 1.475739526e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.189085583 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.760860687 on level 1 Castro::reflux() at level 0 : time = 2.193361757 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.424896998 Gravity::multilevel_solve_for_new_phi() time = 1.349372088 Castro::construct_new_gravity() time = 1.460464386 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.966445588 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029078674 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764982644e+18 ymom added: -1.361438167e+19 zmom added: -8.219114469e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.141638471 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02313197632 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.05578118 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.032116901 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.406324964 Gravity::multilevel_solve_for_new_phi() time = 0.533985304 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.744705993 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 1.995233922 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.04438306 on level 0 Castro::construct_old_gravity() time = 0.031084358 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.442735529e+22 ymom added: 4.427218578e+21 zmom added: 7.10314939e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.197493375 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.7945505 on level 1 Castro::reflux() at level 0 : time = 2.218738166 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.436134966 Gravity::multilevel_solve_for_new_phi() time = 0.978575417 Castro::construct_new_gravity() time = 1.09080742 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.957501031 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029298056 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.81977516e+17 ymom added: 2.351329365e+19 zmom added: 9.539391841e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.142163905 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02353202155 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 46.45356243 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.031559505 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.416466174 Gravity::multilevel_solve_for_new_phi() time = 0.544873691 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.713633277 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 1.972825107 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.89346967 on level 0 Castro::construct_old_gravity() time = 0.040888301 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.20120881e+22 ymom added: 1.180591621e+22 zmom added: 8.023901326e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.192150826 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.794524813 on level 1 Castro::reflux() at level 0 : time = 2.210218455 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.432012672 Gravity::multilevel_solve_for_new_phi() time = 0.97152221 Castro::construct_new_gravity() time = 1.082482686 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.963702963 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.02921657 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598260361e+18 ymom added: -2.567952508e+19 zmom added: -1.336957101e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.141221292 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02300401152 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.5162189 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.031501563 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.418229961 Gravity::multilevel_solve_for_new_phi() time = 0.545249076 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.764772952 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 1.983181993 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.14305956 on level 0 Castro::construct_old_gravity() time = 0.032313457 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013345236e+23 ymom added: 1.829917012e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.190791441 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.793616597 on level 1 Castro::reflux() at level 0 : time = 2.209305543 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.431562211 Gravity::multilevel_solve_for_new_phi() time = 0.973599257 Castro::construct_new_gravity() time = 1.085159176 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171514e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.953721198 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.03226326 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727218968e+18 ymom added: -2.54093091e+19 zmom added: -1.252338301e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.153075331 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02348828518 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 46.52858902 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.529343575 seconds Ending run at 14:56:33 UTC on 2024-03-25. Run time = 239.5137644 Run time without initialization = 227.7830705 Average number of zones advanced per microsecond: 0.022 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 239.5 ... 239.5 ... 239.5 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 92.7705 102.5460 106.5952 44.50% Castro::computeTemp() 88 15.03 16.83 17.54 7.32% check_for_negative_density() 9 0.7353 4.797 14.65 6.12% amrex::Copy() 679 12.36 13.85 14.52 6.06% MultiFab::AddProduct() 4205 10.2 11.36 11.86 4.95% Castro::normalize_species() 87 9.975 11.21 11.69 4.88% FabArray::ParallelCopy_finish() 830 1.528 4.126 10.72 4.48% Castro::reset_internal_energy(Fab) 1476 8.381 9.291 9.646 4.03% Castro::estTimeStep() 26 4.043 5.397 8.782 3.67% FabArray::setVal() 5070 7.031 7.833 8.222 3.43% MultiFab::Subtract() 175 6.748 7.531 7.882 3.29% Gravity::fill_multipole_BCs() 11 4.353 4.475 4.672 1.95% Castro::reset_internal_energy(MultiFab) 88 3.222 3.941 4.572 1.91% FabArray::sum() 4205 3.504 3.898 4.068 1.70% FluxRegister::Reflux() 24 2.87 3.22 3.37 1.41% FabArray::ParallelCopy_nowait() 830 2.036 2.232 2.614 1.09% MLPoisson::Fsmooth() 1376 2.126 2.366 2.474 1.03% Castro::enforce_min_density() 87 1.53 1.895 2.354 0.98% Castro::wd_update() 6 0.4594 0.943 1.825 0.76% Castro::FluxRegCrseInit 9 1.423 1.615 1.74 0.73% Castro::reflux() 4 1.457 1.652 1.725 0.72% FillPatchInterp(MF) 30 1.195 1.303 1.418 0.59% Amr::InitAmr() 1 1.206 1.206 1.206 0.50% Castro::initMFs() 5 1.115 1.153 1.187 0.50% amrex::Add() 136 1.003 1.117 1.168 0.49% MLPoisson::Fapply() 318 1.036 1.048 1.06 0.44% Castro::construct_old_gravity_source() 18 0.9026 1.009 1.055 0.44% Castro::construct_new_gravity_source() 9 0.7788 0.8421 0.8883 0.37% FillPatchIterator::Initialize 62 0.7346 0.8211 0.8652 0.36% FabArray::Saxpy() 182 0.7307 0.8183 0.8548 0.36% FillBoundary_finish() 1553 0.4221 0.5423 0.8408 0.35% Castro::volInBoundary() 42 0.6319 0.7181 0.8309 0.35% Castro::problem_post_timestep() 10 0.7232 0.7539 0.817 0.34% VisMF::Write(FabArray) 3 0.5798 0.6541 0.69 0.29% StateData::FillBoundary(geom) 50 0.5786 0.5941 0.6176 0.26% Castro::construct_new_source() 63 0.484 0.5337 0.5613 0.23% MultiFab::max() 18 0.0251 0.2173 0.504 0.21% MLCellLinOp::applyBC() 1742 0.3705 0.4214 0.4478 0.19% MultiFab::contains_nan() 18 0.3738 0.4159 0.4332 0.18% FillBoundary_nowait() 1553 0.3277 0.3706 0.4131 0.17% FabArray::Xpay() 283 0.3531 0.3854 0.4002 0.17% Castro::initialize_do_advance() 9 0.233 0.2581 0.2733 0.11% Amr::writePlotFile() 2 0.09638 0.146 0.2453 0.10% FabArray::setDomainBndry() 92 0.1907 0.2124 0.2247 0.09% amrex::average_down 233 0.2084 0.2103 0.2133 0.09% Castro::construct_old_source() 63 0.1807 0.1992 0.208 0.09% Castro::fill_hybrid_hydro_source() 27 0.1678 0.1864 0.1958 0.08% Castro::initData() 1 0.1271 0.1494 0.1769 0.07% MLCellLinOp::defineAuxData() 11 0.1477 0.1617 0.168 0.07% MLPoisson::FFlux() 150 0.004524 0.04121 0.1511 0.06% MLCellLinOp::compGrad() 18 0.1273 0.1427 0.1502 0.06% FabArray::mult() 56 0.1105 0.1248 0.131 0.05% MLMG::addInterpCorrection() 155 0.1089 0.1192 0.124 0.05% StateData::define() 25 0.1081 0.1174 0.1211 0.05% Castro::evaluate_source_change() 193 0.1029 0.1113 0.1159 0.05% Gravity::get_new_grav_vector() 10 0.08407 0.09458 0.1005 0.04% Castro::swap_state_time_levels() 9 0.07997 0.09147 0.09658 0.04% Gravity::get_old_grav_vector() 9 0.07864 0.08678 0.09056 0.04% Castro::initialize_advance() 9 0.06628 0.0773 0.08207 0.03% MLMG::solve() 10 0.02575 0.05907 0.07493 0.03% MLMG::prepareForSolve() 10 0.061 0.06831 0.07419 0.03% MLCellLinOp::updateSolBC() 25 0.06721 0.06785 0.06873 0.03% FabArray::norminf(mask) 21 0.05485 0.06147 0.06424 0.03% StateDataPhysBCFunct::() 95 0.002024 0.02 0.06279 0.03% AmrLevel::FillPatch() 62 0.05023 0.05469 0.05896 0.02% Castro::apply_tagging_restrictions() 6 0.04715 0.05278 0.05535 0.02% AmrMesh::MakeNewGrids() 6 0.04775 0.04969 0.05399 0.02% MLCellLinOp::updateCorBC() 17 0.04567 0.04618 0.04694 0.02% BndryData::define() 27 0.04097 0.04237 0.04341 0.02% FabArray::norminf() 147 0.03698 0.03983 0.04303 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.03029 0.03516 0.04094 0.02% MLMG::MLResNormInf() 15 0.0001175 0.02367 0.03457 0.01% MLCellLinOp::reflux() 30 0.00421 0.01148 0.03322 0.01% MLMG::ResNormInf() 54 0.01149 0.02124 0.03104 0.01% Castro::construct_old_gravity() 9 0.0004019 0.01433 0.03085 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01819 0.02187 0.02772 0.01% MLCellLinOp::defineBC() 11 0.02582 0.0262 0.02638 0.01% Castro::FluxRegFineAdd() 9 0.0252 0.02551 0.02583 0.01% MultiFab::Multiply() 8 0.02133 0.0238 0.02497 0.01% Castro::build_fine_mask() 44 0.01696 0.01844 0.02043 0.01% Castro::do_new_sources() 9 0.0006033 0.005693 0.01978 0.01% Castro::Castro() 5 0.01484 0.01679 0.01888 0.01% MLMG::interpCorrection_1 17 0.01835 0.01851 0.01865 0.01% FabArrayBase::CPC::define() 416 0.01299 0.0149 0.01819 0.01% Castro::buildMetrics() 5 0.01468 0.01651 0.01761 0.01% Castro::linear_to_hybrid_momentum() 1 0.01403 0.01597 0.01727 0.01% FillPatchTwoLevels 31 0.01192 0.01398 0.01609 0.01% MLLinOp::makeSubCommunicator() 11 0.0004546 0.007698 0.01522 0.01% MLCellLinOp::prepareForSolve() 11 0.0114 0.01159 0.01181 0.00% Castro::enforce_consistent_e() 1 0.009356 0.01066 0.01146 0.00% Gravity::actual_multilevel_solve() 10 0.008673 0.009435 0.0113 0.00% AMRErrorTag::operator() 6 0.008948 0.01011 0.01076 0.00% FabArrayBase::getCPC() 1440 0.009933 0.01007 0.01028 0.00% Gravity::actual_solve_with_mlmg() 11 0.008994 0.009393 0.00991 0.00% Castro::finalize_do_advance() 9 0.007753 0.008582 0.009595 0.00% Gravity::update_max_rhs() 10 0.007128 0.008156 0.00883 0.00% Castro::apply_problem_tags() 6 0.006343 0.007248 0.008064 0.00% average_down_faces 48 0.005794 0.005824 0.005887 0.00% DistributionMapping::LeastUsedCPUs() 5 0.001635 0.003475 0.005705 0.00% MLCellLinOp::setLevelBC() 19 0.003543 0.004518 0.005216 0.00% AmrLevel::derive() 6 0.003729 0.004364 0.005037 0.00% FabArray::FillBoundary() 1553 0.004272 0.004446 0.004747 0.00% Gravity::test_composite_phi() 1 0.00223 0.003114 0.004185 0.00% MLCellLinOp::smooth() 688 0.003913 0.003965 0.003996 0.00% MLCGSolver::bicgstab 7 0 0.0009526 0.003811 0.00% amrex::Dot() 115 0 0.0008922 0.003569 0.00% FabArrayBase::getFB() 1553 0.00313 0.003244 0.003544 0.00% FabArrayBase::FB::FB() 19 0.002554 0.002916 0.003303 0.00% FabArray::ParallelCopy() 830 0.00294 0.003039 0.003168 0.00% Castro::do_old_sources() 9 0.0005774 0.001508 0.002788 0.00% main() 1 0.001607 0.001642 0.00172 0.00% MLLinOp::defineGrids() 11 0.00136 0.001424 0.001537 0.00% MLCellLinOp::apply() 318 0.0008966 0.001057 0.001482 0.00% Amr::regrid() 5 0.0009641 0.0011 0.001418 0.00% Castro::construct_new_gravity() 9 0.0004018 0.0007215 0.001378 0.00% Castro::clean_state() 87 0.00129 0.001326 0.001365 0.00% TagBoxArray::collate() 6 0.0004393 0.0009778 0.001283 0.00% Amr::coarseTimeStep() 5 0.0005555 0.001045 0.001225 0.00% AmrLevel::FillCoarsePatch() 5 0.0009275 0.001004 0.001156 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002773 0.001109 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.000917 0.0009297 0.0009559 0.00% OwnerMask() 6 0.0007395 0.0008242 0.0008778 0.00% MLCellLinOp::correctionResidual() 189 0.0006885 0.0006999 0.0007145 0.00% Castro::do_advance_ctu() 5 0.0005494 0.0005929 0.0006409 0.00% MLMG::mgVcycle() 65 0.0005983 0.0006191 0.0006407 0.00% FillPatchSingleLevel 92 0.0005428 0.0005538 0.0005711 0.00% Amr::grid_places() 6 0.00046 0.0004985 0.0005533 0.00% Castro::finalize_advance() 9 0.0003152 0.0004057 0.000473 0.00% FillPatchIterator::FillFromLevel0() 31 0.0004122 0.0004278 0.0004708 0.00% TagBoxArray::mapPRD 6 0.0004274 0.0004417 0.0004478 0.00% MLMG:computeResOfCorrection() 155 0.0003961 0.0004067 0.0004217 0.00% Castro::subcycle_advance_ctu() 5 6.65e-05 0.0001511 0.000371 0.00% MLCellLinOp::solutionResidual() 72 0.000362 0.0003644 0.0003699 0.00% MLMG::oneIter() 31 0.0003481 0.0003541 0.0003577 0.00% MLMG::actualBottomSolve() 31 4.058e-05 0.0001188 0.0003457 0.00% Castro::computeNewDt() 8 0.0001176 0.0001754 0.0003362 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002757 0.000302 0.0003149 0.00% MLMG::mgVcycle_down::0 31 0.0002921 0.0003005 0.000309 0.00% Amr::timeStep() 9 0.0002155 0.0002493 0.0003037 0.00% Castro::enforce_speed_limit() 87 0.000256 0.0002697 0.0002813 0.00% Amr::defBaseLevel() 1 0.0001769 0.0002188 0.0002731 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002492 0.0002576 0.0002633 0.00% MLMG::mgVcycle_down::1 31 0.0002402 0.0002485 0.0002564 0.00% AmrLevel::AmrLevel(dm) 5 0.0001931 0.0002064 0.0002254 0.00% Castro::post_regrid() 10 0.000197 0.0002135 0.0002232 0.00% MLMG::mgVcycle_bottom 65 0.0002096 0.0002147 0.0002189 0.00% MLMG::mgVcycle_up::0 31 0.0001879 0.0001977 0.0002157 0.00% MLMG::mgVcycle_down::2 31 0.0001929 0.0001999 0.0002063 0.00% MLMG::mgVcycle_down::3 31 0.0001911 0.0001948 0.0002004 0.00% makeSFC 33 0.000168 0.0001779 0.0001918 0.00% FabArrayBase::TheFPinfo() 25 0.000176 0.0001851 0.0001915 0.00% Castro::react_state() 18 0.0001306 0.0001514 0.0001817 0.00% MLMG::mgVcycle_down::4 31 0.0001708 0.0001746 0.0001808 0.00% Castro::check_for_nan() 18 0.0001327 0.0001446 0.0001762 0.00% Amr::writeSmallPlotFile() 1 0.0001035 0.0001232 0.0001696 0.00% Amr::initSubcycle() 1 4.983e-05 7.908e-05 0.0001656 0.00% MLMG::mgVcycle_up::1 31 0.0001349 0.00014 0.0001451 0.00% Castro::expand_state() 13 0.0001095 0.0001213 0.000139 0.00% FPinfo::FPinfo() 3 0.0001133 0.0001237 0.0001369 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001301 0.0001336 0.0001354 0.00% MLMG::mgVcycle_up::3 31 0.000123 0.0001263 0.0001331 0.00% MLMG::mgVcycle_up::2 31 0.0001285 0.0001295 0.0001318 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001105 0.0001146 0.0001264 0.00% MLMG::mgVcycle_up::4 31 0.0001148 0.0001185 0.000125 0.00% Castro::errorEst() 6 0.0001018 0.0001094 0.0001238 0.00% DistributionMapping::Distribute() 35 0.0001063 0.0001118 0.0001228 0.00% MLPoisson::define() 11 0.0001027 0.000111 0.0001186 0.00% FabArray::RecvLayoutMask() 8 8.77e-05 9.81e-05 0.000117 0.00% Castro::advance() 5 8.042e-05 9.558e-05 0.0001146 0.00% Castro::apply_source_to_state() 18 0.0001064 0.0001091 0.0001123 0.00% Gravity::swapTimeLevels() 9 8e-05 9.297e-05 0.0001068 0.00% Amr::FinalizeInit() 1 3.315e-05 5.768e-05 0.0001048 0.00% Castro::post_timestep() 9 7.989e-05 8.454e-05 9.051e-05 0.00% ClusterList::intersect() 1 0 2.254e-05 9.017e-05 0.00% MLMG::computeResidual() 31 8.507e-05 8.764e-05 8.948e-05 0.00% MLMG::computeMLResidual() 15 8.351e-05 8.654e-05 8.898e-05 0.00% Castro::avgDown(state_indx) 20 6.418e-05 6.86e-05 7.512e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 5.009e-05 6.222e-05 7.352e-05 0.00% knapsack() 3 6.42e-05 6.864e-05 7.335e-05 0.00% Gravity::set_mass_offset() 6 3.849e-05 5.754e-05 7.319e-05 0.00% MLLinOp::define() 11 6.954e-05 7.103e-05 7.234e-05 0.00% MLMG::miniCycle() 34 6.395e-05 6.698e-05 6.847e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.298e-05 6.543e-05 6.834e-05 0.00% MLMG::getGradSolution() 10 6.072e-05 6.508e-05 6.721e-05 0.00% amrex::average_down_w_geom 20 5.21e-05 5.515e-05 5.971e-05 0.00% Castro::create_source_corrector() 9 5.599e-05 5.77e-05 5.879e-05 0.00% Castro::construct_new_hybrid_source() 9 5.034e-05 5.128e-05 5.354e-05 0.00% MLPoisson::prepareForSolve() 11 4.504e-05 4.802e-05 5.141e-05 0.00% Castro::init(old) 3 4.482e-05 4.686e-05 4.92e-05 0.00% Castro::post_init() 1 2.266e-05 3.022e-05 4.163e-05 0.00% Castro::retry_advance_ctu() 5 2.618e-05 3.321e-05 3.971e-05 0.00% AmrMesh-cluster 1 0 9.499e-06 3.799e-05 0.00% Castro::construct_old_hybrid_source() 9 3.356e-05 3.542e-05 3.65e-05 0.00% Castro::computeInitialDt() 2 1.702e-05 2.649e-05 3.625e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.989e-05 3.248e-05 3.553e-05 0.00% Amr::bldFineLevels() 1 2.061e-05 2.686e-05 3.479e-05 0.00% Castro::avgDown() 4 2.952e-05 3.203e-05 3.4e-05 0.00% MLMG::MLRhsNormInf() 10 3.245e-05 3.325e-05 3.399e-05 0.00% Castro::construct_new_rotation_source() 9 2.134e-05 2.496e-05 3.303e-05 0.00% MLMG::compResidual() 1 2.612e-05 2.691e-05 2.857e-05 0.00% BoxList::parallelComplementIn() 12 2.496e-05 2.616e-05 2.821e-05 0.00% Castro::construct_old_rotation_source() 9 2.14e-05 2.377e-05 2.713e-05 0.00% Castro::derive() 6 1.942e-05 2.053e-05 2.324e-05 0.00% BoxList::complementIn 12 2.123e-05 2.205e-05 2.254e-05 0.00% Castro::init() 1 1.573e-05 1.691e-05 1.803e-05 0.00% Amr::init() 1 6.214e-06 8.443e-06 1.444e-05 0.00% Amr::InitializeInit() 1 7.521e-06 9.155e-06 1.095e-05 0.00% ClusterList::chop() 1 0 2.378e-06 9.512e-06 0.00% Amr::initialInit() 1 5.887e-06 6.427e-06 6.983e-06 0.00% knapsack()swap 1 3.503e-06 3.726e-06 3.857e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 239.5 239.5 239.5 100.00% Amr::coarseTimeStep() 5 227.3 227.3 227.3 94.88% Amr::timeStep() 9 225.9 225.9 225.9 94.33% Castro::advance() 5 198.9 198.9 198.9 83.03% Castro::subcycle_advance_ctu() 5 190.4 190.7 191.6 80.00% Castro::do_advance_ctu() 5 190.4 190.7 191.6 80.00% Castro::construct_ctu_hydro_source() 9 117 117.1 117.1 48.88% Castro::clean_state() 87 70.58 79.39 82.96 34.64% Castro::computeTemp() 88 45.03 50.59 52.89 22.08% Castro::reset_internal_energy(MultiFab) 88 30 33.76 35.35 14.76% Castro::evaluate_source_change() 193 17.35 19.31 20.16 8.42% Castro::enforce_min_density() 87 16.36 18.48 19.5 8.14% Castro::post_regrid() 10 15.72 17.21 17.86 7.46% Amr::regrid() 5 15.53 16.92 17.49 7.30% Castro::initialize_do_advance() 9 14.64 14.94 15.11 6.31% check_for_negative_density() 9 0.7353 4.797 14.65 6.12% amrex::Copy() 679 12.36 13.85 14.52 6.06% AmrLevel::FillPatch() 62 7.755 9.438 13.79 5.76% FabArray::ParallelCopy() 830 3.761 6.42 12.89 5.38% FillPatchIterator::Initialize 62 6.079 7.844 12.36 5.16% MultiFab::AddProduct() 4205 10.2 11.36 11.86 4.95% Gravity::solve_phi_with_mlmg() 11 11.41 11.53 11.72 4.89% Castro::normalize_species() 87 9.975 11.21 11.69 4.88% Gravity::multilevel_solve_for_new_phi() 10 11.12 11.23 11.41 4.76% Gravity::actual_multilevel_solve() 10 11.08 11.2 11.38 4.75% Castro::do_old_sources() 9 10.87 10.93 10.99 4.59% FabArray::ParallelCopy_finish() 830 1.528 4.126 10.72 4.48% Castro::do_new_sources() 9 9.135 10.24 10.69 4.46% Amr::init() 1 10.52 10.52 10.52 4.39% FillPatchSingleLevel 92 3.868 5.699 10.34 4.32% Amr::initialInit() 1 9.977 9.978 9.978 4.17% Castro::post_timestep() 9 9.969 9.97 9.971 4.16% FillPatchIterator::FillFromLevel0() 31 3.535 5.33 9.944 4.15% Castro::reset_internal_energy(Fab) 1476 8.381 9.291 9.646 4.03% Castro::reflux() 4 8.756 8.809 8.832 3.69% Castro::estTimeStep() 26 4.043 5.397 8.782 3.67% Castro::initialize_advance() 9 7.258 8.138 8.495 3.55% FabArray::setVal() 5070 7.031 7.833 8.222 3.43% MultiFab::Subtract() 175 6.748 7.531 7.882 3.29% Gravity::actual_solve_with_mlmg() 11 6.953 6.96 6.971 2.91% Castro::construct_new_gravity() 9 6.484 6.59 6.759 2.82% MLMG::solve() 10 6.451 6.464 6.483 2.71% Castro::expand_state() 13 5.069 5.466 6.389 2.67% Amr::FinalizeInit() 1 5.805 5.962 6.211 2.59% MLMG::oneIter() 31 5.2 5.215 5.231 2.18% Gravity::fill_multipole_BCs() 11 4.431 4.552 4.741 1.98% MLMG::mgVcycle() 65 4.372 4.401 4.415 1.84% FluxRegister::Reflux() 24 4.279 4.293 4.302 1.80% Amr::InitializeInit() 1 3.766 4.015 4.173 1.74% Amr::defBaseLevel() 1 3.766 4.015 4.173 1.74% Castro::construct_old_gravity() 9 0.6922 1.671 4.102 1.71% Gravity::get_old_grav_vector() 9 0.6667 1.656 4.101 1.71% FabArray::sum() 4205 3.504 3.898 4.068 1.70% Castro::finalize_do_advance() 9 1.364 2.103 4.033 1.68% Castro::problem_post_timestep() 10 2.731 3.101 3.896 1.63% Castro::computeNewDt() 8 2.174 2.631 3.838 1.60% MLCellLinOp::smooth() 688 3.285 3.33 3.361 1.40% Castro::post_init() 1 2.578 2.771 3.056 1.28% Castro::initData() 1 2.256 2.534 2.732 1.14% Castro::FluxRegCrseInit 9 1.789 2.054 2.722 1.14% FabArray::ParallelCopy_nowait() 830 2.094 2.291 2.677 1.12% MLPoisson::Fsmooth() 1376 2.126 2.366 2.474 1.03% Castro::wd_update() 6 1.29 1.661 2.457 1.03% Castro::construct_new_source() 63 1.815 1.991 2.093 0.87% MLMG::mgVcycle_down::0 31 2.01 2.017 2.024 0.85% MLCellLinOp::applyBC() 1742 1.208 1.345 1.635 0.68% Amr::bldFineLevels() 1 1.241 1.428 1.544 0.64% FillPatchIterator::FillFromTwoLevels() 31 1.465 1.481 1.493 0.62% FillPatchTwoLevels 31 1.465 1.481 1.493 0.62% Castro::Castro() 5 1.393 1.438 1.476 0.62% MLCellLinOp::apply() 318 1.323 1.369 1.446 0.60% FillPatchInterp(MF) 30 1.195 1.303 1.418 0.59% MLMG::mgVcycle_up::0 31 1.362 1.388 1.401 0.58% FabArray::FillBoundary() 1553 0.7598 0.9235 1.265 0.53% Amr::InitAmr() 1 1.206 1.206 1.206 0.50% Castro::initMFs() 5 1.121 1.159 1.195 0.50% amrex::Add() 136 1.003 1.117 1.168 0.49% Amr::writePlotFile() 2 1.076 1.076 1.076 0.45% MLCellLinOp::solutionResidual() 72 1.03 1.055 1.068 0.45% Castro::construct_old_source() 63 0.9126 1.024 1.065 0.44% MLPoisson::Fapply() 318 1.036 1.048 1.06 0.44% Castro::construct_old_gravity_source() 18 0.9026 1.009 1.055 0.44% Castro::computeInitialDt() 2 0.5138 0.6711 0.9199 0.38% Castro::construct_new_gravity_source() 9 0.7788 0.8421 0.8883 0.37% MLCellLinOp::correctionResidual() 189 0.8085 0.833 0.8839 0.37% FabArray::Saxpy() 182 0.7307 0.8183 0.8548 0.36% FillBoundary_finish() 1553 0.4221 0.5423 0.8408 0.35% Castro::volInBoundary() 42 0.632 0.7181 0.831 0.35% Gravity::get_new_grav_vector() 10 0.7783 0.7877 0.7964 0.33% MLMG:computeResOfCorrection() 155 0.6975 0.71 0.7413 0.31% VisMF::Write(FabArray) 3 0.5798 0.6541 0.69 0.29% StateDataPhysBCFunct::() 95 0.5912 0.6141 0.6435 0.27% StateData::FillBoundary(geom) 50 0.5786 0.5941 0.6176 0.26% MLMG::computeResWithCrseSolFineCor() 17 0.5664 0.5672 0.569 0.24% Gravity::update_max_rhs() 10 0.0821 0.2732 0.5545 0.23% MultiFab::max() 18 0.0251 0.2173 0.504 0.21% MLMG::computeMLResidual() 15 0.4106 0.4462 0.5002 0.21% Castro::init() 1 0.4583 0.465 0.4696 0.20% AmrLevel::FillCoarsePatch() 5 0.4583 0.465 0.4696 0.20% Castro::check_for_nan() 18 0.3739 0.416 0.4334 0.18% MultiFab::contains_nan() 18 0.3738 0.4159 0.4332 0.18% FillBoundary_nowait() 1553 0.3334 0.3768 0.4196 0.18% amrex::average_down 233 0.2813 0.3636 0.4162 0.17% Gravity::test_composite_phi() 1 0.4107 0.4125 0.4156 0.17% FabArray::Xpay() 283 0.3531 0.3854 0.4002 0.17% MLMG::computeResidual() 31 0.3638 0.3784 0.3872 0.16% MLMG::mgVcycle_down::1 31 0.3629 0.3694 0.3783 0.16% MLCellLinOp::reflux() 30 0.1751 0.2242 0.3591 0.15% Castro::init(old) 3 0.3553 0.3564 0.3569 0.15% Castro::apply_source_to_state() 18 0.2923 0.3279 0.3432 0.14% MLPoisson::define() 11 0.2745 0.2855 0.2985 0.12% Castro::react_state() 18 0.2104 0.2423 0.2586 0.11% MLMG::prepareForSolve() 10 0.2106 0.2221 0.233 0.10% FabArray::setDomainBndry() 92 0.1907 0.2124 0.2247 0.09% MLMG::mgVcycle_up::1 31 0.2137 0.2175 0.2199 0.09% MLCellLinOp::defineAuxData() 11 0.1733 0.1901 0.2011 0.08% Castro::fill_hybrid_hydro_source() 27 0.1678 0.1864 0.1958 0.08% Amr::grid_places() 6 0.191 0.1911 0.1911 0.08% AmrMesh::MakeNewGrids() 6 0.1905 0.1906 0.1906 0.08% MLMG::addInterpCorrection() 155 0.1281 0.1659 0.1843 0.08% MLMG::getGradSolution() 10 0.1731 0.1773 0.1799 0.08% MLCellLinOp::compGrad() 18 0.1731 0.1772 0.1798 0.08% MLMG::mgVcycle_bottom 65 0.1491 0.1573 0.1787 0.07% MLPoisson::FFlux() 150 0.004524 0.04121 0.1511 0.06% MLMG::miniCycle() 34 0.1492 0.1503 0.151 0.06% Castro::avgDown() 4 0.1215 0.1289 0.1337 0.06% Castro::avgDown(state_indx) 20 0.1215 0.1289 0.1336 0.06% amrex::average_down_w_geom 20 0.1215 0.1288 0.1336 0.06% FabArray::mult() 56 0.1105 0.1248 0.131 0.05% Castro::construct_new_hybrid_source() 9 0.1118 0.1241 0.1303 0.05% AmrLevel::AmrLevel(dm) 5 0.1083 0.1176 0.1213 0.05% StateData::define() 25 0.1081 0.1174 0.1211 0.05% Castro::errorEst() 6 0.1022 0.1147 0.1202 0.05% Castro::swap_state_time_levels() 9 0.07997 0.09147 0.09658 0.04% MLCellLinOp::updateSolBC() 25 0.07678 0.08407 0.09545 0.04% MLMG::ResNormInf() 54 0.08785 0.09134 0.09338 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08604 0.08718 0.08889 0.04% MLCellLinOp::defineBC() 11 0.08386 0.08569 0.08776 0.04% MLMG::mgVcycle_down::2 31 0.07631 0.08043 0.08325 0.03% MLMG::mgVcycle_up::3 31 0.02657 0.06658 0.08009 0.03% MLMG::MLResNormInf() 15 0.04413 0.06652 0.07447 0.03% Castro::construct_old_hybrid_source() 9 0.05606 0.0624 0.06556 0.03% FabArray::norminf(mask) 21 0.05485 0.06147 0.06424 0.03% BndryData::define() 27 0.05638 0.05832 0.06076 0.03% MLMG::mgVcycle_up::2 31 0.05427 0.05528 0.05639 0.02% Castro::apply_tagging_restrictions() 6 0.04715 0.05278 0.05535 0.02% MLMG::interpCorrection_1 17 0.02516 0.03406 0.05509 0.02% MLCellLinOp::updateCorBC() 17 0.05005 0.05063 0.05127 0.02% Castro::derive() 6 0.03969 0.04449 0.04718 0.02% AmrLevel::derive() 6 0.03966 0.04447 0.04716 0.02% Castro::buildMetrics() 5 0.04146 0.0431 0.04527 0.02% FabArray::norminf() 147 0.03698 0.03983 0.04303 0.02% Gravity::swapTimeLevels() 9 0.03014 0.03412 0.03742 0.02% MLMG::mgVcycle_down::3 31 0.03222 0.03337 0.03593 0.02% MLMG::MLRhsNormInf() 10 0.02554 0.02803 0.02912 0.01% TagBoxArray::mapPRD 6 0.004611 0.01387 0.02867 0.01% FabArrayBase::getCPC() 1440 0.02311 0.02497 0.02847 0.01% MLMG::actualBottomSolve() 31 4.058e-05 0.007074 0.02817 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01819 0.02187 0.02772 0.01% MLCGSolver::bicgstab 7 0 0.006869 0.02747 0.01% Castro::FluxRegFineAdd() 9 0.0252 0.02551 0.02583 0.01% MultiFab::Multiply() 8 0.02133 0.0238 0.02497 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.0197 0.02026 0.02175 0.01% MLMG::compResidual() 1 0.01926 0.0201 0.02081 0.01% Castro::build_fine_mask() 44 0.01696 0.01844 0.02043 0.01% MLMG::mgVcycle_down::4 31 0.005366 0.008742 0.01857 0.01% FabArrayBase::CPC::define() 416 0.01299 0.0149 0.01819 0.01% Castro::linear_to_hybrid_momentum() 1 0.01403 0.01597 0.01727 0.01% MLLinOp::define() 11 0.002268 0.009585 0.01724 0.01% MLLinOp::defineGrids() 11 0.002197 0.009514 0.01717 0.01% MLLinOp::makeSubCommunicator() 11 0.0004546 0.007698 0.01522 0.01% MLMG::mgVcycle_up::4 31 0.004408 0.006826 0.01389 0.01% MLPoisson::prepareForSolve() 11 0.01145 0.01164 0.01186 0.00% MLCellLinOp::prepareForSolve() 11 0.0114 0.01159 0.01181 0.00% Castro::enforce_consistent_e() 1 0.009356 0.01066 0.01146 0.00% AMRErrorTag::operator() 6 0.008948 0.01011 0.01076 0.00% average_down_faces 48 0.009055 0.009438 0.009804 0.00% Castro::apply_problem_tags() 6 0.006343 0.007248 0.008064 0.00% FabArrayBase::getFB() 1553 0.005683 0.00616 0.00659 0.00% DistributionMapping::LeastUsedCPUs() 5 0.001635 0.003475 0.005705 0.00% FabArrayBase::TheFPinfo() 25 0.00146 0.003406 0.005495 0.00% FPinfo::FPinfo() 3 0.001278 0.003221 0.005304 0.00% MLCellLinOp::setLevelBC() 19 0.003543 0.004518 0.005216 0.00% DistributionMapping::KnapSackDoIt() 3 0.001161 0.003098 0.005177 0.00% amrex::Dot() 115 0 0.0008922 0.003569 0.00% FabArrayBase::FB::FB() 19 0.002554 0.002916 0.003303 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007444 0.002978 0.00% TagBoxArray::collate() 6 0.0004393 0.0009778 0.001283 0.00% OwnerMask() 6 0.0007395 0.0008242 0.0008778 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001773 0.000557 0.0007122 0.00% Castro::finalize_advance() 9 0.0003152 0.0004057 0.000473 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003769 0.0003914 0.0004203 0.00% makeSFC 33 0.0002664 0.0002767 0.0002939 0.00% Castro::enforce_speed_limit() 87 0.000256 0.0002697 0.0002813 0.00% Amr::writeSmallPlotFile() 1 0.0001035 0.0001232 0.0001696 0.00% Amr::initSubcycle() 1 4.983e-05 7.908e-05 0.0001656 0.00% AmrMesh-cluster 1 0 3.442e-05 0.0001377 0.00% DistributionMapping::Distribute() 35 0.0001063 0.0001118 0.0001228 0.00% FabArray::RecvLayoutMask() 8 8.77e-05 9.81e-05 0.000117 0.00% ClusterList::intersect() 1 0 2.254e-05 9.017e-05 0.00% knapsack() 3 6.799e-05 7.236e-05 7.721e-05 0.00% Gravity::set_mass_offset() 6 3.849e-05 5.754e-05 7.319e-05 0.00% Castro::create_source_corrector() 9 5.599e-05 5.77e-05 5.879e-05 0.00% BoxList::parallelComplementIn() 12 4.75e-05 4.821e-05 4.944e-05 0.00% Castro::retry_advance_ctu() 5 2.618e-05 3.321e-05 3.971e-05 0.00% Castro::construct_new_rotation_source() 9 2.134e-05 2.496e-05 3.303e-05 0.00% Castro::construct_old_rotation_source() 9 2.14e-05 2.377e-05 2.713e-05 0.00% BoxList::complementIn 12 2.123e-05 2.205e-05 2.254e-05 0.00% ClusterList::chop() 1 0 2.378e-06 9.512e-06 0.00% knapsack()swap 1 3.503e-06 3.726e-06 3.857e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-33-g9e71c2cb9714) finalized