Initializing AMReX (v2024-32-g33701ffd5611)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-32-g33701ffd5611) initialized Starting run at 07:22:40 UTC on 2024-03-25. Successfully read inputs file ... Castro git describe: 24.03-26-g5707c0c2f AMReX git describe: v2024-32-g33701ffd5 Microphysics git describe: 24.03-21-g81de2adb reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.032339531 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.03061596 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.362817487 Gravity::multilevel_solve_for_new_phi() time = 1.752915286 Gravity::fill_multipole_BCs() time = 0.367580256 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.555296983 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.860781638 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.363399802 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.05197419 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.702151347 Gravity::multilevel_solve_for_new_phi() time = 1.784556795 Castro::construct_new_gravity() time = 1.908591702 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207933e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.353396622 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02441694261 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.43496903 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.031784174 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.42680616 Gravity::multilevel_solve_for_new_phi() time = 1.954272612 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.82284545 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.049761596 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.80839115 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.031617828 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.571938195e+22 ymom added: 1.505254316e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.194159395 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.800110046 on level 1 Castro::reflux() at level 0 : time = 2.226039763 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.422297833 Gravity::multilevel_solve_for_new_phi() time = 1.361297454 Castro::construct_new_gravity() time = 1.478494694 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.980698832 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029578911 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764905094e+18 ymom added: -1.361438167e+19 zmom added: -8.219114489e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.142567786 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02257967581 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 50.19994083 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.032082734 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.404187006 Gravity::multilevel_solve_for_new_phi() time = 0.532579638 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.787724756 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.042740395 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.81677886 on level 0 Castro::construct_old_gravity() time = 0.031884959 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.444541472e+22 ymom added: 4.427218578e+21 zmom added: 7.10314939e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.193957446 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.806773099 on level 1 Castro::reflux() at level 0 : time = 2.241350761 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.424184816 Gravity::multilevel_solve_for_new_phi() time = 0.970640267 Castro::construct_new_gravity() time = 1.087669361 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.974141369 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.031294896 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.818594215e+17 ymom added: 2.350879005e+19 zmom added: 9.5393936e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.172771535 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02291940986 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.6915833 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032768706 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.415129265 Gravity::multilevel_solve_for_new_phi() time = 0.550050062 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.895272007 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.047977546 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.81784077 on level 0 Castro::construct_old_gravity() time = 0.032350419 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.200546555e+22 ymom added: 1.180591621e+22 zmom added: 8.024045442e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.194132616 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.822437307 on level 1 Castro::reflux() at level 0 : time = 2.230604718 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.422410058 Gravity::multilevel_solve_for_new_phi() time = 0.968935394 Castro::construct_new_gravity() time = 1.086725549 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.976788021 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.031203738 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598332534e+18 ymom added: -2.569753947e+19 zmom added: -1.336957981e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.16183753 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02282346323 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.93708054 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.031607047 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.404205036 Gravity::multilevel_solve_for_new_phi() time = 0.531623114 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.611987443 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.010152599 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.76582599 on level 0 Castro::construct_old_gravity() time = 0.044130121 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013324418e+23 ymom added: 1.829917012e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.196903267 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.819602613 on level 1 Castro::reflux() at level 0 : time = 2.282902795 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.425817437 Gravity::multilevel_solve_for_new_phi() time = 0.9947015 Castro::construct_new_gravity() time = 1.1143059 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.012136832 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029736181 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.726107516e+18 ymom added: -2.54093091e+19 zmom added: -1.25233881e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.147160558 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02292963769 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.54736215 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.534284787 seconds Ending run at 07:26:44 UTC on 2024-03-25. Run time = 243.8568709 Run time without initialization = 232.3472036 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 243.9 ... 243.9 ... 243.9 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 93.4287 103.8662 108.9926 44.70% Castro::computeTemp() 88 15.22 17 17.79 7.30% check_for_negative_density() 9 0.4789 5.697 16.21 6.65% amrex::Copy() 679 13.05 14.51 15.07 6.18% MultiFab::AddProduct() 4205 10.88 12.07 12.5 5.12% Castro::normalize_species() 87 10.2 11.4 11.86 4.86% FabArray::ParallelCopy_finish() 830 1.586 4.286 10.45 4.28% Castro::reset_internal_energy(Fab) 1476 8.466 9.396 9.784 4.01% Castro::estTimeStep() 26 4.036 5.381 8.66 3.55% FabArray::setVal() 5070 7.533 8.292 8.62 3.53% MultiFab::Subtract() 175 6.593 7.344 7.635 3.13% Gravity::fill_multipole_BCs() 11 4.275 4.412 4.705 1.93% Castro::reset_internal_energy(MultiFab) 88 3.348 3.925 4.297 1.76% FabArray::sum() 4205 3.426 3.796 3.927 1.61% FluxRegister::Reflux() 24 2.93 3.269 3.42 1.40% FabArray::ParallelCopy_nowait() 830 2.03 2.292 2.849 1.17% MLPoisson::Fsmooth() 1376 2.145 2.404 2.51 1.03% Castro::enforce_min_density() 87 1.539 1.827 2.069 0.85% Castro::wd_update() 6 0.552 0.9583 1.915 0.79% Castro::reflux() 4 1.48 1.664 1.747 0.72% Castro::FluxRegCrseInit 9 1.474 1.611 1.679 0.69% FillPatchInterp(MF) 30 1.226 1.331 1.436 0.59% Amr::InitAmr() 1 1.194 1.194 1.194 0.49% Castro::initMFs() 5 1.151 1.158 1.172 0.48% amrex::Add() 136 0.9803 1.086 1.123 0.46% Castro::construct_old_gravity_source() 18 0.9482 1.028 1.066 0.44% MLPoisson::Fapply() 318 0.9009 0.9998 1.037 0.43% FillBoundary_finish() 1553 0.2912 0.4926 0.9222 0.38% Castro::construct_new_gravity_source() 9 0.7666 0.8535 0.9161 0.38% FillPatchIterator::Initialize 62 0.7617 0.8332 0.8861 0.36% Castro::problem_post_timestep() 10 0.7495 0.8181 0.8574 0.35% FabArray::Saxpy() 182 0.7327 0.8088 0.8365 0.34% Castro::volInBoundary() 42 0.6011 0.7357 0.8075 0.33% VisMF::Write(FabArray) 3 0.5699 0.6582 0.7205 0.30% StateData::FillBoundary(geom) 50 0.5821 0.5935 0.6079 0.25% Castro::construct_new_source() 63 0.5195 0.5486 0.5636 0.23% MLCellLinOp::applyBC() 1742 0.359 0.4284 0.4573 0.19% MultiFab::contains_nan() 18 0.3831 0.4277 0.445 0.18% FillBoundary_nowait() 1553 0.3205 0.3797 0.4348 0.18% FabArray::Xpay() 283 0.3502 0.3921 0.4096 0.17% MultiFab::max() 18 0.02689 0.1177 0.3813 0.16% Castro::initialize_do_advance() 9 0.2384 0.2564 0.2652 0.11% Amr::writePlotFile() 2 0.07838 0.1444 0.2495 0.10% FabArray::setDomainBndry() 92 0.1937 0.2161 0.232 0.10% Castro::construct_old_source() 63 0.2002 0.2067 0.2188 0.09% amrex::average_down 233 0.1967 0.2055 0.2088 0.09% Castro::fill_hybrid_hydro_source() 27 0.1722 0.1885 0.1978 0.08% MLCellLinOp::defineAuxData() 11 0.1515 0.1658 0.1725 0.07% MLPoisson::FFlux() 150 0.0046 0.04194 0.1538 0.06% Castro::initData() 1 0.1359 0.1448 0.1504 0.06% MLCellLinOp::compGrad() 18 0.1274 0.1408 0.1461 0.06% FabArray::mult() 56 0.1221 0.1368 0.1423 0.06% MLMG::addInterpCorrection() 155 0.11 0.1218 0.1263 0.05% StateData::define() 25 0.1045 0.1179 0.1254 0.05% Castro::evaluate_source_change() 193 0.1026 0.1095 0.1138 0.05% Castro::swap_state_time_levels() 9 0.08368 0.09741 0.1123 0.05% Castro::initialize_advance() 9 0.06615 0.08425 0.1066 0.04% Gravity::get_new_grav_vector() 10 0.08935 0.09797 0.1044 0.04% Gravity::get_old_grav_vector() 9 0.07854 0.08656 0.08998 0.04% MLMG::prepareForSolve() 10 0.06427 0.06767 0.07179 0.03% MLCellLinOp::updateSolBC() 25 0.06633 0.06674 0.06699 0.03% MLMG::solve() 10 0.0005962 0.04216 0.06663 0.03% FabArray::norminf(mask) 21 0.05584 0.0617 0.06442 0.03% AmrLevel::FillPatch() 62 0.04923 0.05712 0.06214 0.03% Castro::apply_tagging_restrictions() 6 0.0488 0.05309 0.05527 0.02% AmrMesh::MakeNewGrids() 6 0.04703 0.04881 0.05266 0.02% MLMG::ResNormInf() 54 0.003546 0.01517 0.0462 0.02% MLCellLinOp::updateCorBC() 17 0.04503 0.04537 0.04567 0.02% BndryData::define() 27 0.04064 0.0416 0.0424 0.02% FabArray::norminf() 147 0.03494 0.03841 0.04171 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.02775 0.03234 0.0357 0.01% MLMG::MLResNormInf() 15 0.0001095 0.02423 0.03512 0.01% MLCellLinOp::reflux() 30 0.004222 0.01169 0.03393 0.01% Castro::FluxRegFineAdd() 9 0.02907 0.02937 0.02972 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01905 0.02261 0.02917 0.01% MLCellLinOp::defineBC() 11 0.02564 0.02581 0.02595 0.01% MultiFab::Multiply() 8 0.02135 0.02409 0.02579 0.01% StateDataPhysBCFunct::() 95 0.00228 0.012 0.02248 0.01% MLLinOp::makeSubCommunicator() 11 0.001901 0.009631 0.02073 0.01% Castro::construct_old_gravity() 9 0.0004284 0.00985 0.01955 0.01% Castro::build_fine_mask() 44 0.01704 0.01764 0.01893 0.01% MLMG::interpCorrection_1 17 0.01817 0.01841 0.01864 0.01% Castro::do_new_sources() 9 0.0005795 0.004963 0.01796 0.01% FabArrayBase::CPC::define() 416 0.01304 0.0147 0.01775 0.01% Castro::linear_to_hybrid_momentum() 1 0.01532 0.01644 0.01752 0.01% Castro::Castro() 5 0.01477 0.01626 0.0171 0.01% Castro::buildMetrics() 5 0.01492 0.01635 0.01692 0.01% FillPatchTwoLevels 31 0.01192 0.01268 0.01376 0.01% MLCellLinOp::prepareForSolve() 11 0.01155 0.01174 0.0121 0.00% Castro::enforce_consistent_e() 1 0.009394 0.01056 0.01096 0.00% AMRErrorTag::operator() 6 0.008638 0.00975 0.01019 0.00% Gravity::actual_multilevel_solve() 10 0.007744 0.009073 0.01002 0.00% FabArrayBase::getCPC() 1440 0.009746 0.009813 0.009989 0.00% Gravity::actual_solve_with_mlmg() 11 0.00914 0.009345 0.009591 0.00% Castro::finalize_do_advance() 9 0.00757 0.008585 0.009094 0.00% Gravity::update_max_rhs() 10 0.007943 0.008311 0.008762 0.00% Castro::apply_problem_tags() 6 0.006261 0.007058 0.007342 0.00% average_down_faces 48 0.005547 0.005645 0.005745 0.00% MLCellLinOp::setLevelBC() 19 0.003767 0.004789 0.005345 0.00% AmrLevel::derive() 6 0.00366 0.004298 0.004979 0.00% FabArray::FillBoundary() 1553 0.004285 0.004455 0.004823 0.00% MLCellLinOp::smooth() 688 0.003798 0.003921 0.004046 0.00% amrex::Dot() 115 0 0.0009465 0.003786 0.00% MLCGSolver::bicgstab 7 0 0.0009431 0.003773 0.00% Castro::construct_new_gravity() 9 0.0004814 0.001824 0.0037 0.00% FabArrayBase::getFB() 1553 0.003088 0.003227 0.003539 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0001293 0.001489 0.003379 0.00% FabArrayBase::FB::FB() 19 0.002481 0.002831 0.003173 0.00% FabArray::ParallelCopy() 830 0.002914 0.002978 0.003056 0.00% Gravity::test_composite_phi() 1 0.002136 0.00244 0.002585 0.00% Castro::do_old_sources() 9 0.0006274 0.0009635 0.001906 0.00% AmrLevel::FillCoarsePatch() 5 0.0008828 0.001187 0.001745 0.00% main() 1 0.001505 0.001604 0.001696 0.00% MLLinOp::defineGrids() 11 0.001339 0.001426 0.001546 0.00% MLCellLinOp::apply() 318 0.0008867 0.001051 0.001521 0.00% Amr::regrid() 5 0.0009148 0.001073 0.001479 0.00% Amr::coarseTimeStep() 5 0.0006002 0.001109 0.001304 0.00% Castro::clean_state() 87 0.001236 0.001256 0.001282 0.00% TagBoxArray::collate() 6 0.000531 0.000925 0.001244 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002799 0.001119 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009048 0.0009142 0.0009225 0.00% OwnerMask() 6 0.000724 0.0008257 0.0009089 0.00% MLCellLinOp::correctionResidual() 189 0.0006743 0.00068 0.0006917 0.00% MLMG::mgVcycle() 65 0.000594 0.000602 0.0006103 0.00% Castro::do_advance_ctu() 5 0.0005146 0.000545 0.0005853 0.00% FillPatchSingleLevel 92 0.0005126 0.0005265 0.0005481 0.00% Amr::grid_places() 6 0.0003778 0.0004593 0.000535 0.00% Castro::finalize_advance() 9 0.0002789 0.000372 0.0005028 0.00% Castro::subcycle_advance_ctu() 5 6.552e-05 0.0001792 0.0004824 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003988 0.0004368 0.0004752 0.00% TagBoxArray::mapPRD 6 0.00042 0.0004329 0.0004453 0.00% MLMG:computeResOfCorrection() 155 0.0003776 0.0003873 0.0004015 0.00% MLMG::oneIter() 31 0.000338 0.000348 0.0003548 0.00% MLCellLinOp::solutionResidual() 72 0.0003445 0.0003471 0.000349 0.00% MLMG::actualBottomSolve() 31 4.038e-05 0.0001181 0.0003478 0.00% Amr::timeStep() 9 0.0001952 0.0002449 0.0003475 0.00% Castro::computeNewDt() 8 0.0001139 0.0001714 0.0003397 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002933 0.0003032 0.0003208 0.00% MLMG::mgVcycle_down::0 31 0.0002756 0.0002858 0.0002916 0.00% Castro::enforce_speed_limit() 87 0.0002528 0.0002651 0.0002779 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002388 0.000249 0.0002649 0.00% MLMG::mgVcycle_down::1 31 0.0002373 0.0002438 0.0002547 0.00% Amr::defBaseLevel() 1 0.000227 0.0002301 0.000236 0.00% Castro::post_regrid() 10 0.0001932 0.0002089 0.0002196 0.00% AmrLevel::AmrLevel(dm) 5 0.0001819 0.0001971 0.0002151 0.00% MLMG::mgVcycle_bottom 65 0.0002051 0.0002097 0.0002123 0.00% MLMG::mgVcycle_down::3 31 0.0001879 0.0001912 0.0001992 0.00% MLMG::mgVcycle_up::0 31 0.0001811 0.0001875 0.0001955 0.00% MLMG::mgVcycle_down::2 31 0.0001907 0.0001925 0.0001954 0.00% makeSFC 33 0.0001604 0.0001767 0.000194 0.00% Amr::FinalizeInit() 1 3.358e-05 7.629e-05 0.0001887 0.00% Castro::react_state() 18 0.0001318 0.0001457 0.000185 0.00% MLMG::mgVcycle_down::4 31 0.0001608 0.0001672 0.0001773 0.00% FabArrayBase::TheFPinfo() 25 0.0001705 0.0001718 0.0001753 0.00% Castro::advance() 5 8.792e-05 0.0001048 0.0001499 0.00% Castro::check_for_nan() 18 0.0001305 0.0001377 0.0001439 0.00% FPinfo::FPinfo() 3 0.00011 0.0001225 0.0001413 0.00% MLMG::mgVcycle_up::1 31 0.0001364 0.0001373 0.0001385 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001287 0.0001317 0.000134 0.00% MLMG::mgVcycle_up::2 31 0.0001272 0.000129 0.000132 0.00% MLMG::mgVcycle_up::3 31 0.0001192 0.0001228 0.0001249 0.00% MLMG::mgVcycle_up::4 31 0.0001119 0.0001166 0.0001239 0.00% Castro::expand_state() 13 0.0001101 0.0001141 0.000119 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001116 0.0001151 0.0001182 0.00% Amr::writeSmallPlotFile() 1 9.718e-05 0.0001082 0.0001162 0.00% MLPoisson::define() 11 0.0001025 0.0001075 0.0001148 0.00% Castro::apply_source_to_state() 18 9.948e-05 0.0001039 0.0001146 0.00% Castro::errorEst() 6 0.0001024 0.0001064 0.0001112 0.00% FabArray::RecvLayoutMask() 8 9.074e-05 9.651e-05 0.0001112 0.00% Gravity::swapTimeLevels() 9 8.295e-05 9.114e-05 0.0001111 0.00% DistributionMapping::Distribute() 35 0.0001068 0.0001087 0.0001105 0.00% Castro::post_timestep() 9 7.795e-05 8.47e-05 9.79e-05 0.00% MLMG::computeResidual() 31 8.255e-05 8.535e-05 8.846e-05 0.00% MLMG::computeMLResidual() 15 7.742e-05 8.053e-05 8.384e-05 0.00% ClusterList::intersect() 1 0 2.02e-05 8.08e-05 0.00% knapsack() 3 6.151e-05 6.888e-05 7.959e-05 0.00% Castro::avgDown(state_indx) 20 6.525e-05 6.96e-05 7.339e-05 0.00% MLMG::miniCycle() 34 6.734e-05 6.861e-05 7.006e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.156e-05 6.316e-05 6.584e-05 0.00% MLMG::getGradSolution() 10 5.908e-05 6.189e-05 6.533e-05 0.00% MLLinOp::define() 11 5.735e-05 6.085e-05 6.303e-05 0.00% Castro::create_source_corrector() 9 5.606e-05 5.922e-05 6.101e-05 0.00% amrex::average_down_w_geom 20 5.042e-05 5.268e-05 5.434e-05 0.00% MLPoisson::prepareForSolve() 11 4.489e-05 4.867e-05 5.401e-05 0.00% Amr::initSubcycle() 1 4.706e-05 4.92e-05 5.102e-05 0.00% BoxList::parallelComplementIn() 12 2.739e-05 4.025e-05 5.015e-05 0.00% Castro::construct_new_hybrid_source() 9 4.677e-05 4.798e-05 5.004e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.407e-05 4.7e-05 4.95e-05 0.00% Castro::init(old) 3 4.349e-05 4.475e-05 4.76e-05 0.00% Castro::post_init() 1 2.848e-05 3.6e-05 4.638e-05 0.00% AmrMesh-cluster 1 0 1.019e-05 4.076e-05 0.00% Castro::construct_old_hybrid_source() 9 3.548e-05 3.776e-05 4.045e-05 0.00% Gravity::set_mass_offset() 6 3.575e-05 3.649e-05 3.818e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.898e-05 3.248e-05 3.614e-05 0.00% MLMG::compResidual() 1 2.669e-05 2.955e-05 3.397e-05 0.00% Castro::retry_advance_ctu() 5 2.291e-05 2.681e-05 3.355e-05 0.00% MLMG::MLRhsNormInf() 10 3.185e-05 3.248e-05 3.325e-05 0.00% Castro::avgDown() 4 2.825e-05 2.951e-05 3.209e-05 0.00% Amr::bldFineLevels() 1 2.074e-05 2.451e-05 2.996e-05 0.00% BoxList::complementIn 12 2.199e-05 2.38e-05 2.694e-05 0.00% Castro::construct_old_rotation_source() 9 2.009e-05 2.384e-05 2.671e-05 0.00% Castro::construct_new_rotation_source() 9 2.191e-05 2.311e-05 2.475e-05 0.00% Castro::computeInitialDt() 2 1.628e-05 1.92e-05 2.235e-05 0.00% Castro::derive() 6 1.887e-05 1.966e-05 2.14e-05 0.00% Castro::init() 1 1.572e-05 1.682e-05 1.847e-05 0.00% Amr::init() 1 6.653e-06 9.212e-06 1.266e-05 0.00% Amr::InitializeInit() 1 7.217e-06 8.572e-06 1.221e-05 0.00% ClusterList::chop() 1 0 2.975e-06 1.19e-05 0.00% Amr::initialInit() 1 5.243e-06 7.508e-06 1.085e-05 0.00% knapsack()swap 1 3.697e-06 4.305e-06 4.537e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 243.9 243.9 243.9 100.00% Amr::coarseTimeStep() 5 231.8 231.8 231.8 95.06% Amr::timeStep() 9 230.5 230.5 230.5 94.51% Castro::advance() 5 202.9 202.9 202.9 83.19% Castro::subcycle_advance_ctu() 5 194.1 194.5 195.3 80.07% Castro::do_advance_ctu() 5 194.1 194.5 195.3 80.07% Castro::construct_ctu_hydro_source() 9 119.2 119.4 119.4 48.98% Castro::clean_state() 87 72.35 80.93 84.46 34.63% Castro::computeTemp() 88 46.08 51.52 53.75 22.04% Castro::reset_internal_energy(MultiFab) 88 30.86 34.52 35.96 14.75% Castro::evaluate_source_change() 193 18.2 20.18 20.88 8.56% Castro::enforce_min_density() 87 16.92 18.91 19.78 8.11% Castro::post_regrid() 10 16.04 17.47 18.08 7.41% Amr::regrid() 5 15.94 17.22 17.79 7.29% check_for_negative_density() 9 0.4789 5.697 16.21 6.65% Castro::initialize_do_advance() 9 14.95 15.32 15.55 6.37% amrex::Copy() 679 13.05 14.51 15.07 6.18% AmrLevel::FillPatch() 62 8.321 9.852 13.92 5.71% FabArray::ParallelCopy() 830 4.363 6.642 12.63 5.18% MultiFab::AddProduct() 4205 10.88 12.07 12.5 5.12% FillPatchIterator::Initialize 62 6.576 8.181 12.4 5.09% Castro::normalize_species() 87 10.2 11.4 11.86 4.86% Gravity::solve_phi_with_mlmg() 11 11.28 11.42 11.71 4.80% Gravity::multilevel_solve_for_new_phi() 10 10.99 11.12 11.4 4.67% Gravity::actual_multilevel_solve() 10 10.95 11.09 11.37 4.66% Castro::do_old_sources() 9 11.16 11.21 11.29 4.63% Castro::do_new_sources() 9 9.363 10.47 10.92 4.48% FabArray::ParallelCopy_finish() 830 1.586 4.286 10.45 4.28% FillPatchSingleLevel 92 4.316 6.017 10.37 4.25% Amr::init() 1 10.31 10.31 10.31 4.23% Castro::post_timestep() 9 10.19 10.19 10.19 4.18% FillPatchIterator::FillFromLevel0() 31 3.987 5.647 9.976 4.09% Castro::reset_internal_energy(Fab) 1476 8.466 9.396 9.784 4.01% Amr::initialInit() 1 9.759 9.759 9.759 4.00% Castro::reflux() 4 8.902 8.956 8.976 3.68% Castro::initialize_advance() 9 7.592 8.369 8.755 3.59% Castro::estTimeStep() 26 4.036 5.381 8.66 3.55% FabArray::setVal() 5070 7.533 8.292 8.62 3.53% MultiFab::Subtract() 175 6.593 7.344 7.635 3.13% Gravity::actual_solve_with_mlmg() 11 6.901 6.908 6.915 2.84% Castro::construct_new_gravity() 9 6.407 6.528 6.797 2.79% Castro::expand_state() 13 5.306 5.686 6.451 2.65% MLMG::solve() 10 6.389 6.404 6.424 2.63% Amr::FinalizeInit() 1 5.66 5.726 5.915 2.43% MLMG::oneIter() 31 5.186 5.204 5.215 2.14% Gravity::fill_multipole_BCs() 11 4.356 4.489 4.775 1.96% MLMG::mgVcycle() 65 4.362 4.391 4.402 1.81% FluxRegister::Reflux() 24 4.364 4.385 4.397 1.80% Castro::construct_old_gravity() 9 0.7155 1.768 4.099 1.68% Amr::InitializeInit() 1 3.843 4.033 4.099 1.68% Amr::defBaseLevel() 1 3.843 4.033 4.099 1.68% Gravity::get_old_grav_vector() 9 0.6959 1.759 4.099 1.68% Castro::problem_post_timestep() 10 2.877 3.244 4.067 1.67% Castro::finalize_do_advance() 9 1.36 2.115 4.035 1.65% FabArray::sum() 4205 3.426 3.796 3.927 1.61% Castro::computeNewDt() 8 2.178 2.701 3.87 1.59% MLCellLinOp::smooth() 688 3.317 3.341 3.363 1.38% FabArray::ParallelCopy_nowait() 830 2.09 2.352 2.912 1.19% Castro::post_init() 1 2.548 2.635 2.897 1.19% Castro::initData() 1 2.375 2.557 2.623 1.08% Castro::FluxRegCrseInit 9 1.715 1.952 2.544 1.04% Castro::wd_update() 6 1.326 1.694 2.517 1.03% MLPoisson::Fsmooth() 1376 2.145 2.404 2.51 1.03% Castro::construct_new_source() 63 1.85 2.023 2.13 0.87% MLMG::mgVcycle_down::0 31 1.93 1.994 2.021 0.83% MLCellLinOp::applyBC() 1742 1.167 1.311 1.709 0.70% FillPatchIterator::FillFromTwoLevels() 31 1.474 1.484 1.498 0.61% FillPatchTwoLevels 31 1.473 1.484 1.497 0.61% Amr::bldFineLevels() 1 1.337 1.453 1.496 0.61% Castro::Castro() 5 1.427 1.434 1.443 0.59% MLMG::mgVcycle_up::0 31 1.405 1.432 1.442 0.59% FillPatchInterp(MF) 30 1.226 1.331 1.436 0.59% FabArray::FillBoundary() 1553 0.7093 0.8828 1.35 0.55% MLCellLinOp::apply() 318 1.293 1.312 1.326 0.54% Amr::InitAmr() 1 1.194 1.194 1.194 0.49% Castro::initMFs() 5 1.156 1.163 1.177 0.48% amrex::Add() 136 0.9803 1.086 1.123 0.46% Amr::writePlotFile() 2 1.09 1.09 1.091 0.45% Castro::construct_old_source() 63 0.9639 1.038 1.084 0.44% Castro::construct_old_gravity_source() 18 0.9482 1.028 1.066 0.44% MLCellLinOp::solutionResidual() 72 0.9718 1.023 1.045 0.43% MLPoisson::Fapply() 318 0.9009 0.9998 1.037 0.43% FillBoundary_finish() 1553 0.2912 0.4926 0.9222 0.38% Castro::construct_new_gravity_source() 9 0.7666 0.8535 0.9161 0.38% FabArray::Saxpy() 182 0.7327 0.8088 0.8365 0.34% Gravity::get_new_grav_vector() 10 0.8149 0.8217 0.8302 0.34% MLCellLinOp::correctionResidual() 189 0.8067 0.8126 0.8151 0.33% Castro::volInBoundary() 42 0.6012 0.7358 0.8075 0.33% Castro::computeInitialDt() 2 0.5071 0.5734 0.7629 0.31% VisMF::Write(FabArray) 3 0.5699 0.6582 0.7205 0.30% MLMG:computeResOfCorrection() 155 0.6577 0.6859 0.7017 0.29% StateDataPhysBCFunct::() 95 0.5843 0.6055 0.623 0.26% StateData::FillBoundary(geom) 50 0.5821 0.5935 0.6079 0.25% MLMG::computeResWithCrseSolFineCor() 17 0.5744 0.5754 0.5771 0.24% MLMG::computeMLResidual() 15 0.412 0.4356 0.4939 0.20% Castro::init() 1 0.4759 0.4801 0.4848 0.20% AmrLevel::FillCoarsePatch() 5 0.4759 0.4801 0.4848 0.20% Castro::check_for_nan() 18 0.3832 0.4279 0.4452 0.18% MultiFab::contains_nan() 18 0.3831 0.4277 0.445 0.18% FillBoundary_nowait() 1553 0.326 0.3858 0.4413 0.18% Gravity::update_max_rhs() 10 0.08927 0.1776 0.4367 0.18% Gravity::test_composite_phi() 1 0.4116 0.4122 0.4139 0.17% FabArray::Xpay() 283 0.3502 0.3921 0.4096 0.17% amrex::average_down 233 0.2773 0.3583 0.3981 0.16% MLMG::mgVcycle_down::1 31 0.3101 0.3338 0.3922 0.16% MultiFab::max() 18 0.02689 0.1177 0.3813 0.16% MLMG::computeResidual() 31 0.3504 0.3599 0.3661 0.15% MLCellLinOp::reflux() 30 0.1783 0.2298 0.3635 0.15% Castro::init(old) 3 0.3571 0.3586 0.3594 0.15% Castro::apply_source_to_state() 18 0.2923 0.3243 0.3356 0.14% MLPoisson::define() 11 0.284 0.2921 0.2995 0.12% Castro::react_state() 18 0.2242 0.2528 0.2699 0.11% FabArray::setDomainBndry() 92 0.1937 0.2161 0.232 0.10% MLMG::prepareForSolve() 10 0.2165 0.2256 0.2317 0.10% MLMG::mgVcycle_up::1 31 0.221 0.2235 0.2261 0.09% MLCellLinOp::defineAuxData() 11 0.1775 0.1953 0.208 0.09% Castro::fill_hybrid_hydro_source() 27 0.1722 0.1885 0.1978 0.08% Amr::grid_places() 6 0.191 0.1912 0.1913 0.08% AmrMesh::MakeNewGrids() 6 0.1905 0.1908 0.1909 0.08% MLMG::addInterpCorrection() 155 0.1316 0.1692 0.1873 0.08% MLMG::mgVcycle_bottom 65 0.1562 0.1637 0.1852 0.08% MLMG::getGradSolution() 10 0.1725 0.1761 0.178 0.07% MLCellLinOp::compGrad() 18 0.1724 0.176 0.1779 0.07% MLMG::miniCycle() 34 0.1562 0.1565 0.1569 0.06% MLPoisson::FFlux() 150 0.0046 0.04194 0.1538 0.06% FabArray::mult() 56 0.1221 0.1368 0.1423 0.06% Castro::construct_new_hybrid_source() 9 0.1143 0.1253 0.1333 0.05% Castro::avgDown() 4 0.12 0.1224 0.1261 0.05% Castro::avgDown(state_indx) 20 0.1199 0.1224 0.1261 0.05% amrex::average_down_w_geom 20 0.1199 0.1223 0.126 0.05% AmrLevel::AmrLevel(dm) 5 0.1047 0.1181 0.1255 0.05% StateData::define() 25 0.1045 0.1179 0.1254 0.05% Castro::errorEst() 6 0.1052 0.1162 0.1208 0.05% Castro::swap_state_time_levels() 9 0.08368 0.09741 0.1123 0.05% MLMG::ResNormInf() 54 0.07541 0.0844 0.1092 0.04% MLCellLinOp::updateSolBC() 25 0.07688 0.08381 0.09301 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08503 0.08641 0.08869 0.04% MLCellLinOp::defineBC() 11 0.08362 0.08513 0.08757 0.04% MLMG::mgVcycle_up::3 31 0.02593 0.06677 0.08071 0.03% MLMG::mgVcycle_down::2 31 0.07161 0.07457 0.08005 0.03% MLMG::MLResNormInf() 15 0.04382 0.06735 0.07603 0.03% Castro::construct_old_hybrid_source() 9 0.05799 0.06325 0.06522 0.03% FabArray::norminf(mask) 21 0.05584 0.0617 0.06442 0.03% BndryData::define() 27 0.05646 0.05805 0.06066 0.02% MLMG::interpCorrection_1 17 0.02738 0.03709 0.05967 0.02% Castro::apply_tagging_restrictions() 6 0.0488 0.05309 0.05527 0.02% MLMG::mgVcycle_up::2 31 0.05293 0.05345 0.05377 0.02% Castro::buildMetrics() 5 0.04244 0.04823 0.05323 0.02% MLCellLinOp::updateCorBC() 17 0.04952 0.05007 0.05151 0.02% Castro::derive() 6 0.04145 0.04624 0.04944 0.02% AmrLevel::derive() 6 0.04143 0.04622 0.04942 0.02% Gravity::swapTimeLevels() 9 0.03475 0.03812 0.04329 0.02% FabArray::norminf() 147 0.03494 0.03841 0.04171 0.02% MLMG::mgVcycle_down::3 31 0.03155 0.03257 0.035 0.01% Castro::FluxRegFineAdd() 9 0.02907 0.02937 0.02972 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01905 0.02261 0.02917 0.01% MLMG::MLRhsNormInf() 10 0.025 0.02782 0.0291 0.01% MLMG::actualBottomSolve() 31 4.038e-05 0.007287 0.02902 0.01% MLCGSolver::bicgstab 7 0 0.007075 0.0283 0.01% FabArrayBase::getCPC() 1440 0.0228 0.02451 0.02774 0.01% TagBoxArray::mapPRD 6 0.004884 0.01317 0.02691 0.01% MultiFab::Multiply() 8 0.02135 0.02409 0.02579 0.01% MLLinOp::define() 11 0.003685 0.01151 0.02275 0.01% MLLinOp::defineGrids() 11 0.003622 0.01145 0.02269 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01992 0.0206 0.02216 0.01% MLMG::compResidual() 1 0.02076 0.02124 0.02141 0.01% MLLinOp::makeSubCommunicator() 11 0.001901 0.009631 0.02073 0.01% MLMG::mgVcycle_down::4 31 0.005355 0.008844 0.01897 0.01% Castro::build_fine_mask() 44 0.01704 0.01764 0.01893 0.01% FabArrayBase::CPC::define() 416 0.01304 0.0147 0.01775 0.01% Castro::linear_to_hybrid_momentum() 1 0.01532 0.01644 0.01752 0.01% MLMG::mgVcycle_up::4 31 0.004431 0.006897 0.0141 0.01% MLPoisson::prepareForSolve() 11 0.0116 0.01179 0.01214 0.00% MLCellLinOp::prepareForSolve() 11 0.01155 0.01174 0.0121 0.00% Castro::enforce_consistent_e() 1 0.009394 0.01056 0.01096 0.00% AMRErrorTag::operator() 6 0.008638 0.00975 0.01019 0.00% average_down_faces 48 0.009135 0.009389 0.009937 0.00% Castro::apply_problem_tags() 6 0.006261 0.007058 0.007342 0.00% FabArrayBase::getFB() 1553 0.005592 0.006058 0.006468 0.00% MLCellLinOp::setLevelBC() 19 0.003767 0.004789 0.005345 0.00% amrex::Dot() 115 0 0.0009465 0.003786 0.00% FabArrayBase::TheFPinfo() 25 0.0004585 0.001818 0.003741 0.00% FPinfo::FPinfo() 3 0.0002876 0.001646 0.00357 0.00% DistributionMapping::KnapSackDoIt() 3 0.0001738 0.001523 0.003429 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0001293 0.001489 0.003379 0.00% FabArrayBase::FB::FB() 19 0.002481 0.002831 0.003173 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007843 0.003137 0.00% TagBoxArray::collate() 6 0.000531 0.000925 0.001244 0.00% OwnerMask() 6 0.000724 0.0008257 0.0009089 0.00% Castro::finalize_advance() 9 0.0002789 0.000372 0.0005028 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003744 0.0003897 0.00041 0.00% makeSFC 33 0.0002562 0.0002747 0.0002939 0.00% Castro::enforce_speed_limit() 87 0.0002528 0.0002651 0.0002779 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001051 0.0001291 0.0001428 0.00% AmrMesh-cluster 1 0 3.337e-05 0.0001335 0.00% Amr::writeSmallPlotFile() 1 9.718e-05 0.0001082 0.0001162 0.00% FabArray::RecvLayoutMask() 8 9.074e-05 9.651e-05 0.0001112 0.00% DistributionMapping::Distribute() 35 0.0001068 0.0001087 0.0001105 0.00% knapsack() 3 6.521e-05 7.319e-05 8.412e-05 0.00% ClusterList::intersect() 1 0 2.02e-05 8.08e-05 0.00% BoxList::parallelComplementIn() 12 4.937e-05 6.405e-05 7.373e-05 0.00% Castro::create_source_corrector() 9 5.606e-05 5.922e-05 6.101e-05 0.00% Amr::initSubcycle() 1 4.706e-05 4.92e-05 5.102e-05 0.00% Gravity::set_mass_offset() 6 3.575e-05 3.649e-05 3.818e-05 0.00% Castro::retry_advance_ctu() 5 2.291e-05 2.681e-05 3.355e-05 0.00% BoxList::complementIn 12 2.199e-05 2.38e-05 2.694e-05 0.00% Castro::construct_old_rotation_source() 9 2.009e-05 2.384e-05 2.671e-05 0.00% Castro::construct_new_rotation_source() 9 2.191e-05 2.311e-05 2.475e-05 0.00% ClusterList::chop() 1 0 2.975e-06 1.19e-05 0.00% knapsack()swap 1 3.697e-06 4.305e-06 4.537e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-32-g33701ffd5611) finalized