Initializing AMReX (v2024-42-g7437aa17d36f)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-42-g7437aa17d36f) initialized Starting run at 18:54:51 UTC on 2024-03-28. Successfully read inputs file ... Castro git describe: 24.03-37-g6b90dd1c0 AMReX git describe: v2024-42-g7437aa17d Microphysics git describe: 24.03-23-g3112c13a reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.034313265 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031279591 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.371636354 Gravity::multilevel_solve_for_new_phi() time = 1.778813256 Gravity::fill_multipole_BCs() time = 0.370016299 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.537384923 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.892748194 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793356e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.38568603 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.22723194 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.614366742 Gravity::multilevel_solve_for_new_phi() time = 1.693004164 Castro::construct_new_gravity() time = 1.815182922 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207916e+26 ymom added: 1.108730173e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.459269383 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02426760795 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.70994203 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032648784 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.413505301 Gravity::multilevel_solve_for_new_phi() time = 2.000080466 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.867121397 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.085451466 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.80221698 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.034195498 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.574322738e+22 ymom added: 1.475739526e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.197323267 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.867678492 on level 1 Castro::reflux() at level 0 : time = 2.408823951 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.46919673 Gravity::multilevel_solve_for_new_phi() time = 1.443285024 Castro::construct_new_gravity() time = 1.561380011 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.995904937 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.031331477 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764863347e+18 ymom added: -1.361438167e+19 zmom added: -8.219114228e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.164600713 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02235601872 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 50.87482682 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031794284 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.403750578 Gravity::multilevel_solve_for_new_phi() time = 0.535211325 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.7406124 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.009760159 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.8178015 on level 0 Castro::construct_old_gravity() time = 0.035192322 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.443552482e+22 ymom added: 5.312662293e+21 zmom added: 7.10314939e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.200072318 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.912208166 on level 1 Castro::reflux() at level 0 : time = 2.370527865 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.435487692 Gravity::multilevel_solve_for_new_phi() time = 1.004108985 Castro::construct_new_gravity() time = 1.12358218 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.034231944 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029820204 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.819512651e+17 ymom added: 2.352230085e+19 zmom added: 9.539391408e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.143114431 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02283778 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 48.09155823 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.033137741 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.418335771 Gravity::multilevel_solve_for_new_phi() time = 0.55461393 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.874699816 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.036533403 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.7322479 on level 0 Castro::construct_old_gravity() time = 0.030592269 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.200593843e+22 ymom added: 1.180591621e+22 zmom added: 8.023901326e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.19865202 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.866306765 on level 1 Castro::reflux() at level 0 : time = 2.381897686 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.424687228 Gravity::multilevel_solve_for_new_phi() time = 0.987431996 Castro::construct_new_gravity() time = 1.104341878 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 2.003140103 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029925878 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598693397e+18 ymom added: -2.568853227e+19 zmom added: -1.33695875e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.146927363 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02275127732 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 48.09210273 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.032836155 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.40899008 Gravity::multilevel_solve_for_new_phi() time = 0.538522768 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.717264012 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 1.985912074 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.57094788 on level 0 Castro::construct_old_gravity() time = 0.030518462 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013085085e+23 ymom added: 1.888946593e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.194348186 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.82390409 on level 1 Castro::reflux() at level 0 : time = 2.319643086 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.433108372 Gravity::multilevel_solve_for_new_phi() time = 0.990901526 Castro::construct_new_gravity() time = 1.110192686 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.056919817 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.030022371 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.726917289e+18 ymom added: -2.54093091e+19 zmom added: -1.252339827e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.143709106 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02303961506 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.3947772 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.538197234 seconds Ending run at 18:58:57 UTC on 2024-03-28. Run time = 245.344678 Run time without initialization = 233.7033912 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 245.3 ... 245.3 ... 245.3 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 94.9579 105.1202 108.7650 44.33% Castro::computeTemp() 88 14.73 16.53 17.19 7.01% amrex::Copy() 679 12.74 14.27 14.85 6.05% check_for_negative_density() 9 0.6696 4.319 14.57 5.94% MultiFab::AddProduct() 4205 10.54 11.8 12.28 5.01% Castro::normalize_species() 87 10.08 11.35 11.85 4.83% FabArray::ParallelCopy_finish() 830 1.808 4.488 11.33 4.62% Castro::reset_internal_energy(Fab) 1476 8.315 9.266 9.654 3.93% Castro::estTimeStep() 26 4.05 5.506 9.309 3.79% FabArray::setVal() 5070 7.537 8.368 8.709 3.55% MultiFab::Subtract() 175 7.052 7.907 8.223 3.35% Castro::reset_internal_energy(MultiFab) 88 3.399 4.297 5.132 2.09% Gravity::fill_multipole_BCs() 11 4.327 4.447 4.689 1.91% FabArray::sum() 4205 3.507 3.916 4.072 1.66% FluxRegister::Reflux() 24 3.068 3.43 3.588 1.46% FabArray::ParallelCopy_nowait() 830 2.147 2.394 2.963 1.21% MLPoisson::Fsmooth() 1376 2.175 2.426 2.527 1.03% Castro::enforce_min_density() 87 1.57 1.929 2.376 0.97% Castro::reflux() 4 1.523 1.725 1.827 0.74% Castro::FluxRegCrseInit 9 1.503 1.65 1.748 0.71% Castro::wd_update() 6 0.5408 0.8615 1.726 0.70% FillPatchInterp(MF) 30 1.261 1.361 1.459 0.59% Amr::InitAmr() 1 1.208 1.208 1.208 0.49% Castro::initMFs() 5 1.092 1.127 1.199 0.49% Castro::construct_old_gravity_source() 18 0.992 1.117 1.188 0.48% amrex::Add() 136 0.9889 1.108 1.15 0.47% Castro::construct_new_gravity_source() 9 0.8867 1.002 1.095 0.45% MLPoisson::Fapply() 318 0.9343 1.038 1.078 0.44% FabArray::Saxpy() 182 0.7169 0.8607 1.067 0.43% Castro::problem_post_timestep() 10 0.8554 0.9334 1.003 0.41% Castro::volInBoundary() 42 0.7768 0.8489 0.9277 0.38% FillPatchIterator::Initialize 62 0.7577 0.8527 0.9087 0.37% FillBoundary_finish() 1553 0.282 0.463 0.875 0.36% VisMF::Write(FabArray) 3 0.5799 0.6637 0.6994 0.29% StateData::FillBoundary(geom) 50 0.5926 0.611 0.626 0.26% Castro::construct_new_source() 63 0.4852 0.5336 0.5624 0.23% MultiFab::max() 18 0.02999 0.1751 0.5028 0.20% MLCellLinOp::applyBC() 1742 0.3694 0.446 0.4956 0.20% FillBoundary_nowait() 1553 0.364 0.4317 0.4838 0.20% MultiFab::contains_nan() 18 0.3984 0.4476 0.4671 0.19% FabArray::Xpay() 283 0.3526 0.391 0.405 0.17% Castro::initialize_do_advance() 9 0.2423 0.2754 0.2925 0.12% FabArray::setDomainBndry() 92 0.2035 0.2332 0.2549 0.10% Amr::writePlotFile() 2 0.08499 0.1313 0.2398 0.10% amrex::average_down 233 0.1996 0.2087 0.2124 0.09% Castro::construct_old_source() 63 0.1828 0.198 0.2117 0.09% Castro::fill_hybrid_hydro_source() 27 0.1778 0.1932 0.1998 0.08% MLCellLinOp::defineAuxData() 11 0.1613 0.1757 0.182 0.07% Castro::initData() 1 0.1259 0.1447 0.1603 0.07% MLPoisson::FFlux() 149 0.004833 0.04326 0.1583 0.06% MLCellLinOp::compGrad() 18 0.1244 0.138 0.1432 0.06% StateData::define() 25 0.1087 0.1336 0.143 0.06% Castro::swap_state_time_levels() 9 0.08392 0.1037 0.137 0.06% FabArray::mult() 56 0.1149 0.1276 0.1324 0.05% MLMG::addInterpCorrection() 155 0.1107 0.1235 0.1285 0.05% Castro::evaluate_source_change() 193 0.1068 0.1135 0.1177 0.05% Castro::initialize_advance() 9 0.07046 0.08497 0.1046 0.04% Gravity::get_new_grav_vector() 10 0.0893 0.09735 0.1013 0.04% Amr::coarseTimeStep() 5 0.001043 0.06989 0.09285 0.04% Gravity::get_old_grav_vector() 9 0.08318 0.08917 0.09127 0.04% MLMG::prepareForSolve() 10 0.06875 0.07053 0.07191 0.03% MLCellLinOp::updateSolBC() 25 0.06777 0.06837 0.06917 0.03% MLMG::solve() 10 0.0005901 0.04233 0.06685 0.03% FabArray::norminf(mask) 21 0.05749 0.06376 0.06605 0.03% AmrLevel::FillPatch() 62 0.05033 0.05626 0.06231 0.03% Castro::apply_tagging_restrictions() 6 0.04903 0.05422 0.05605 0.02% AmrMesh::MakeNewGrids() 6 0.04755 0.04954 0.05419 0.02% MLCellLinOp::updateCorBC() 17 0.04624 0.04674 0.04744 0.02% BndryData::define() 27 0.04244 0.04312 0.04364 0.02% MLMG::ResNormInf() 54 0.0008459 0.01256 0.04267 0.02% FabArray::norminf() 147 0.0373 0.03951 0.04237 0.02% StateDataPhysBCFunct::() 95 0.002235 0.01291 0.04022 0.02% MLMG::MLResNormInf() 15 0.0001144 0.0256 0.03701 0.02% MLCellLinOp::reflux() 30 0.004665 0.01241 0.03498 0.01% Gravity::multilevel_solve_for_new_phi() 10 0.03057 0.03321 0.03445 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.0208 0.02448 0.0309 0.01% Castro::FluxRegFineAdd() 9 0.0271 0.02761 0.02797 0.01% MLCellLinOp::defineBC() 11 0.02623 0.02655 0.02674 0.01% MultiFab::Multiply() 8 0.02219 0.02468 0.02553 0.01% Castro::do_old_sources() 9 0.0006138 0.005988 0.02185 0.01% Castro::construct_old_gravity() 9 0.0003869 0.0103 0.0203 0.01% Castro::buildMetrics() 5 0.01471 0.01828 0.01967 0.01% MLMG::interpCorrection_1 17 0.01938 0.01945 0.01953 0.01% Castro::build_fine_mask() 44 0.01741 0.01809 0.01943 0.01% Castro::Castro() 5 0.01463 0.01709 0.01859 0.01% FabArrayBase::CPC::define() 416 0.01318 0.01488 0.01799 0.01% MLLinOp::makeSubCommunicator() 11 0.001421 0.007881 0.01722 0.01% Castro::linear_to_hybrid_momentum() 1 0.01407 0.01573 0.01635 0.01% FillPatchTwoLevels 31 0.01268 0.01335 0.01437 0.01% MLCellLinOp::prepareForSolve() 11 0.01184 0.01206 0.01225 0.00% Castro::do_new_sources() 9 0.0006406 0.004258 0.01129 0.00% Castro::enforce_consistent_e() 1 0.009394 0.01069 0.01125 0.00% FabArrayBase::getCPC() 1440 0.01029 0.01051 0.01079 0.00% AMRErrorTag::operator() 6 0.008881 0.01001 0.0105 0.00% Gravity::actual_solve_with_mlmg() 11 0.009923 0.01012 0.01027 0.00% Gravity::actual_multilevel_solve() 10 0.008169 0.009467 0.01008 0.00% Castro::finalize_do_advance() 9 0.008119 0.009233 0.01004 0.00% Gravity::update_max_rhs() 10 0.008357 0.008792 0.009838 0.00% Castro::apply_problem_tags() 6 0.006383 0.007361 0.008243 0.00% AmrLevel::derive() 6 0.004397 0.005221 0.006302 0.00% MLCellLinOp::setLevelBC() 19 0.004149 0.005096 0.006221 0.00% average_down_faces 48 0.005933 0.005974 0.006028 0.00% DistributionMapping::LeastUsedCPUs() 5 0.002501 0.00411 0.005723 0.00% FabArray::FillBoundary() 1553 0.004208 0.004552 0.004969 0.00% amrex::Dot() 115 0 0.001036 0.004145 0.00% MLCellLinOp::smooth() 688 0.00395 0.004051 0.004125 0.00% MLCGSolver::bicgstab 7 0 0.001004 0.004018 0.00% FabArrayBase::getFB() 1553 0.00312 0.003278 0.003594 0.00% FabArrayBase::FB::FB() 19 0.002488 0.002833 0.003191 0.00% FabArray::ParallelCopy() 830 0.002958 0.003059 0.003152 0.00% Gravity::test_composite_phi() 1 0.002576 0.002644 0.002672 0.00% main() 1 0.001559 0.001686 0.001805 0.00% Castro::construct_new_gravity() 9 0.0004574 0.0008006 0.001681 0.00% MLLinOp::defineGrids() 11 0.001378 0.001494 0.001634 0.00% MLCellLinOp::apply() 318 0.0009338 0.00111 0.001581 0.00% TagBoxArray::collate() 6 0.0007945 0.001177 0.001476 0.00% Amr::regrid() 5 0.001031 0.001164 0.001412 0.00% Castro::clean_state() 87 0.001263 0.00134 0.001404 0.00% MLCGSolver::ParallelAllReduce 183 0 0.000289 0.001156 0.00% AmrLevel::FillCoarsePatch() 5 0.0008902 0.0009522 0.001031 0.00% OwnerMask() 6 0.0007683 0.0009049 0.001004 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009216 0.0009294 0.0009358 0.00% MLCellLinOp::correctionResidual() 189 0.0007283 0.0007512 0.0007667 0.00% MLMG::mgVcycle() 65 0.000621 0.0006453 0.0006638 0.00% FillPatchSingleLevel 92 0.0005315 0.0005506 0.0005803 0.00% Castro::do_advance_ctu() 5 0.0004821 0.0005271 0.0005802 0.00% Amr::grid_places() 6 0.0004307 0.0005167 0.0005791 0.00% TagBoxArray::mapPRD 6 0.0004471 0.0004764 0.0005407 0.00% Castro::finalize_advance() 9 0.0003105 0.0004305 0.000502 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003941 0.0004167 0.0004431 0.00% MLMG:computeResOfCorrection() 155 0.0004064 0.0004121 0.0004186 0.00% Castro::subcycle_advance_ctu() 5 0.0001413 0.0002799 0.0004099 0.00% MLCellLinOp::solutionResidual() 72 0.000361 0.0003763 0.0003875 0.00% MLMG::actualBottomSolve() 31 4.158e-05 0.0001279 0.000384 0.00% MLMG::oneIter() 31 0.0003589 0.0003649 0.0003676 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002898 0.0003077 0.0003218 0.00% MLMG::mgVcycle_down::0 31 0.0002925 0.0002976 0.0003031 0.00% Amr::defBaseLevel() 1 0.0002391 0.0002542 0.0002824 0.00% Castro::enforce_speed_limit() 87 0.0002546 0.000265 0.0002749 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002313 0.0002482 0.0002656 0.00% Amr::timeStep() 9 0.0001968 0.0002296 0.0002615 0.00% Castro::computeNewDt() 8 0.000121 0.0001578 0.0002566 0.00% MLMG::mgVcycle_down::1 31 0.0002396 0.0002467 0.0002514 0.00% Castro::post_regrid() 10 0.0002062 0.0002272 0.0002427 0.00% MLMG::mgVcycle_bottom 65 0.0002165 0.0002185 0.0002231 0.00% MLMG::mgVcycle_down::3 31 0.0001862 0.0001947 0.0002019 0.00% AmrLevel::AmrLevel(dm) 5 0.0001899 0.0001949 0.0001992 0.00% MLMG::mgVcycle_down::2 31 0.0001946 0.0001969 0.0001982 0.00% MLMG::mgVcycle_up::0 31 0.0001918 0.0001945 0.0001968 0.00% makeSFC 33 0.0001621 0.0001736 0.0001854 0.00% MLMG::mgVcycle_down::4 31 0.0001631 0.0001749 0.0001842 0.00% FabArrayBase::TheFPinfo() 25 0.000164 0.0001753 0.0001802 0.00% Castro::check_for_nan() 18 0.0001458 0.0001527 0.0001682 0.00% Castro::react_state() 18 0.000145 0.0001478 0.0001527 0.00% MLMG::mgVcycle_up::1 31 0.0001451 0.0001494 0.0001524 0.00% MLMG::mgVcycle_up::2 31 0.0001276 0.0001376 0.0001455 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001383 0.0001418 0.0001453 0.00% Castro::expand_state() 13 0.0001133 0.0001234 0.0001367 0.00% FPinfo::FPinfo() 3 0.0001027 0.0001182 0.000131 0.00% MLMG::mgVcycle_up::4 31 0.0001137 0.0001206 0.0001298 0.00% MLMG::mgVcycle_up::3 31 0.0001209 0.0001263 0.000129 0.00% Castro::errorEst() 6 0.0001083 0.0001138 0.0001219 0.00% MLLinOp::makeAgglomeratedDMap 11 0.000106 0.0001115 0.0001199 0.00% FabArray::RecvLayoutMask() 8 9.945e-05 0.0001067 0.0001182 0.00% Castro::apply_source_to_state() 18 0.0001067 0.0001117 0.0001178 0.00% DistributionMapping::Distribute() 35 0.0001111 0.0001134 0.0001168 0.00% MLPoisson::define() 11 9.874e-05 0.0001078 0.0001146 0.00% Amr::writeSmallPlotFile() 1 9.253e-05 0.0001039 0.000114 0.00% Castro::post_timestep() 9 8.002e-05 9.09e-05 0.0001121 0.00% Castro::advance() 5 7.547e-05 9.238e-05 0.0001101 0.00% Gravity::swapTimeLevels() 9 8.421e-05 9.084e-05 0.0001048 0.00% MLLinOp::define() 11 6.656e-05 7.761e-05 9.775e-05 0.00% Amr::FinalizeInit() 1 4.414e-05 6.253e-05 9.318e-05 0.00% MLMG::computeResidual() 31 8.722e-05 8.877e-05 9.024e-05 0.00% MLMG::computeMLResidual() 15 8.309e-05 8.496e-05 8.805e-05 0.00% ClusterList::intersect() 1 0 2.122e-05 8.488e-05 0.00% Castro::avgDown(state_indx) 20 6.873e-05 7.441e-05 7.864e-05 0.00% MLMG::miniCycle() 34 6.963e-05 7.243e-05 7.735e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.765e-05 7.017e-05 7.549e-05 0.00% knapsack() 3 6.36e-05 6.906e-05 7.493e-05 0.00% MLMG::getGradSolution() 10 6.07e-05 6.496e-05 7.055e-05 0.00% Castro::create_source_corrector() 9 5.455e-05 5.979e-05 6.715e-05 0.00% Gravity::set_mass_offset() 6 4.178e-05 5.426e-05 5.951e-05 0.00% Castro::construct_new_hybrid_source() 9 4.812e-05 5.311e-05 5.754e-05 0.00% amrex::average_down_w_geom 20 5.405e-05 5.53e-05 5.621e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.92e-05 5.285e-05 5.586e-05 0.00% Amr::initSubcycle() 1 4.987e-05 5.035e-05 5.073e-05 0.00% MLPoisson::prepareForSolve() 11 4.547e-05 4.77e-05 5.066e-05 0.00% Castro::init(old) 3 4.476e-05 4.643e-05 4.823e-05 0.00% Castro::retry_advance_ctu() 5 2.752e-05 3.352e-05 4.799e-05 0.00% BoxList::parallelComplementIn() 12 2.822e-05 3.257e-05 4.455e-05 0.00% Castro::computeInitialDt() 2 2.818e-05 3.56e-05 4.298e-05 0.00% Castro::avgDown() 4 3.171e-05 3.473e-05 4.138e-05 0.00% AmrMesh-cluster 1 0 9.954e-06 3.982e-05 0.00% BoxList::complementIn 12 2.295e-05 3.108e-05 3.966e-05 0.00% Castro::construct_old_hybrid_source() 9 3.325e-05 3.561e-05 3.692e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.911e-05 3.221e-05 3.441e-05 0.00% MLMG::MLRhsNormInf() 10 3.368e-05 3.404e-05 3.425e-05 0.00% Amr::InitializeInit() 1 7.363e-06 1.566e-05 3.276e-05 0.00% Castro::post_init() 1 2.396e-05 3.012e-05 3.26e-05 0.00% Castro::construct_new_rotation_source() 9 2.501e-05 2.728e-05 3.051e-05 0.00% Amr::bldFineLevels() 1 2.1e-05 2.456e-05 2.998e-05 0.00% MLMG::compResidual() 1 2.716e-05 2.802e-05 2.946e-05 0.00% Castro::construct_old_rotation_source() 9 2.222e-05 2.506e-05 2.938e-05 0.00% Castro::derive() 6 2.17e-05 2.286e-05 2.431e-05 0.00% Amr::init() 1 8.36e-06 1.3e-05 2.394e-05 0.00% Castro::init() 1 1.602e-05 1.665e-05 1.722e-05 0.00% Amr::initialInit() 1 5.502e-06 8.206e-06 1.505e-05 0.00% ClusterList::chop() 1 0 2.604e-06 1.042e-05 0.00% knapsack()swap 1 3.495e-06 3.908e-06 4.412e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 245.3 245.3 245.3 100.00% Amr::coarseTimeStep() 5 233.2 233.2 233.2 95.04% Amr::timeStep() 9 231.7 231.7 231.8 94.49% Castro::advance() 5 203.6 203.6 203.6 82.98% Castro::subcycle_advance_ctu() 5 194.8 195.2 196.2 79.96% Castro::do_advance_ctu() 5 194.8 195.2 196.2 79.96% Castro::construct_ctu_hydro_source() 9 119.4 119.5 119.5 48.70% Castro::clean_state() 87 71.75 81.06 84.68 34.52% Castro::computeTemp() 88 45.48 51.43 53.84 21.94% Castro::reset_internal_energy(MultiFab) 88 30.75 34.9 36.65 14.94% Castro::evaluate_source_change() 193 18 20.12 20.93 8.53% Castro::enforce_min_density() 87 17 19.21 20.22 8.24% Castro::post_regrid() 10 15.99 17.67 18.35 7.48% Amr::regrid() 5 15.92 17.42 18.03 7.35% Castro::initialize_do_advance() 9 15.01 15.31 15.5 6.32% amrex::Copy() 679 12.74 14.27 14.85 6.05% check_for_negative_density() 9 0.6696 4.319 14.57 5.94% AmrLevel::FillPatch() 62 8.407 9.981 14.48 5.90% FabArray::ParallelCopy() 830 4.596 6.946 13.59 5.54% FillPatchIterator::Initialize 62 6.736 8.35 13.02 5.31% MultiFab::AddProduct() 4205 10.54 11.8 12.28 5.01% Castro::normalize_species() 87 10.08 11.35 11.85 4.83% Gravity::solve_phi_with_mlmg() 11 11.47 11.59 11.83 4.82% Gravity::multilevel_solve_for_new_phi() 10 11.18 11.3 11.52 4.70% Gravity::actual_multilevel_solve() 10 11.14 11.27 11.49 4.68% FabArray::ParallelCopy_finish() 830 1.808 4.488 11.33 4.62% Castro::do_old_sources() 9 11.15 11.21 11.29 4.60% Castro::do_new_sources() 9 9.461 10.68 11.14 4.54% FillPatchSingleLevel 92 4.486 6.124 10.94 4.46% Castro::post_timestep() 9 10.45 10.48 10.54 4.30% FillPatchIterator::FillFromLevel0() 31 4.101 5.747 10.54 4.30% Amr::init() 1 10.43 10.43 10.43 4.25% Amr::initialInit() 1 9.895 9.895 9.895 4.03% Castro::reset_internal_energy(Fab) 1476 8.315 9.266 9.654 3.93% Castro::reflux() 4 9.369 9.434 9.47 3.86% Castro::estTimeStep() 26 4.05 5.506 9.309 3.79% Castro::initialize_advance() 9 7.417 8.365 8.745 3.56% FabArray::setVal() 5070 7.537 8.368 8.709 3.55% MultiFab::Subtract() 175 7.052 7.907 8.223 3.35% Gravity::actual_solve_with_mlmg() 11 7.042 7.047 7.053 2.87% Castro::construct_new_gravity() 9 6.501 6.622 6.833 2.79% Castro::expand_state() 13 5.412 5.822 6.813 2.78% MLMG::solve() 10 6.519 6.531 6.55 2.67% Amr::FinalizeInit() 1 5.728 5.91 6.319 2.58% MLMG::oneIter() 31 5.292 5.31 5.319 2.17% Gravity::fill_multipole_BCs() 11 4.409 4.527 4.761 1.94% FluxRegister::Reflux() 24 4.626 4.645 4.663 1.90% MLMG::mgVcycle() 65 4.444 4.472 4.484 1.83% Castro::problem_post_timestep() 10 3.16 3.466 4.235 1.73% Castro::construct_old_gravity() 9 0.726 1.727 4.221 1.72% Gravity::get_old_grav_vector() 9 0.7057 1.717 4.221 1.72% Amr::InitializeInit() 1 3.575 3.984 4.167 1.70% Amr::defBaseLevel() 1 3.575 3.984 4.167 1.70% Castro::finalize_do_advance() 9 1.346 2.086 4.134 1.69% Castro::computeNewDt() 8 2.223 2.756 4.101 1.67% FabArray::sum() 4205 3.507 3.916 4.072 1.66% MLCellLinOp::smooth() 688 3.375 3.393 3.419 1.39% Castro::post_init() 1 2.557 2.704 3.031 1.24% FabArray::ParallelCopy_nowait() 830 2.207 2.455 3.027 1.23% Castro::FluxRegCrseInit 9 1.806 2.089 2.781 1.13% Castro::initData() 1 2.235 2.56 2.752 1.12% MLPoisson::Fsmooth() 1376 2.175 2.426 2.527 1.03% Castro::wd_update() 6 1.335 1.711 2.503 1.02% Castro::construct_new_source() 63 1.936 2.188 2.284 0.93% MLMG::mgVcycle_down::0 31 1.964 2.031 2.054 0.84% MLCellLinOp::applyBC() 1742 1.214 1.351 1.739 0.71% FillPatchIterator::FillFromTwoLevels() 31 1.513 1.517 1.522 0.62% FillPatchTwoLevels 31 1.513 1.517 1.521 0.62% Amr::bldFineLevels() 1 1.273 1.432 1.505 0.61% MLMG::mgVcycle_up::0 31 1.428 1.454 1.464 0.60% FillPatchInterp(MF) 30 1.261 1.361 1.459 0.59% Castro::Castro() 5 1.313 1.37 1.453 0.59% MLCellLinOp::apply() 318 1.328 1.358 1.375 0.56% FabArray::FillBoundary() 1553 0.7181 0.9054 1.37 0.56% Amr::InitAmr() 1 1.208 1.208 1.208 0.49% Castro::initMFs() 5 1.098 1.133 1.205 0.49% Castro::construct_old_gravity_source() 18 0.992 1.117 1.188 0.48% amrex::Add() 136 0.9889 1.108 1.15 0.47% Castro::construct_new_gravity_source() 9 0.8867 1.002 1.095 0.45% Castro::construct_old_source() 63 0.9264 1.036 1.09 0.44% Castro::computeInitialDt() 2 0.4914 0.6738 1.083 0.44% MLPoisson::Fapply() 318 0.9343 1.038 1.078 0.44% Amr::writePlotFile() 2 1.076 1.076 1.076 0.44% MLCellLinOp::solutionResidual() 72 0.9993 1.05 1.071 0.44% FabArray::Saxpy() 182 0.7169 0.8607 1.067 0.43% Castro::volInBoundary() 42 0.7769 0.849 0.9277 0.38% FillBoundary_finish() 1553 0.282 0.463 0.875 0.36% MLCellLinOp::correctionResidual() 189 0.8291 0.8339 0.8409 0.34% Gravity::get_new_grav_vector() 10 0.8049 0.815 0.8262 0.34% MLMG:computeResOfCorrection() 155 0.6767 0.7075 0.7209 0.29% VisMF::Write(FabArray) 3 0.5799 0.6637 0.6994 0.29% StateDataPhysBCFunct::() 95 0.5948 0.6239 0.6398 0.26% StateData::FillBoundary(geom) 50 0.5926 0.611 0.626 0.26% MLMG::computeResWithCrseSolFineCor() 17 0.5913 0.5925 0.5943 0.24% Gravity::update_max_rhs() 10 0.08922 0.235 0.5591 0.23% MLMG::computeMLResidual() 15 0.4239 0.4452 0.5031 0.21% MultiFab::max() 18 0.02999 0.1751 0.5028 0.20% Castro::init() 1 0.4913 0.4951 0.4974 0.20% AmrLevel::FillCoarsePatch() 5 0.4913 0.495 0.4973 0.20% FillBoundary_nowait() 1553 0.3696 0.4378 0.4903 0.20% Castro::check_for_nan() 18 0.3985 0.4477 0.4672 0.19% MultiFab::contains_nan() 18 0.3984 0.4476 0.4671 0.19% Castro::apply_source_to_state() 18 0.2872 0.3446 0.4257 0.17% Gravity::test_composite_phi() 1 0.4147 0.4151 0.4161 0.17% FabArray::Xpay() 283 0.3526 0.391 0.405 0.17% amrex::average_down 233 0.2836 0.364 0.4042 0.16% MLMG::mgVcycle_down::1 31 0.3182 0.3408 0.4031 0.16% MLCellLinOp::reflux() 30 0.1821 0.2343 0.3743 0.15% MLMG::computeResidual() 31 0.3573 0.3663 0.3712 0.15% Castro::init(old) 3 0.3677 0.3692 0.37 0.15% MLPoisson::define() 11 0.2953 0.3041 0.3106 0.13% Castro::react_state() 18 0.2367 0.2521 0.2585 0.11% FabArray::setDomainBndry() 92 0.2035 0.2332 0.2549 0.10% MLMG::prepareForSolve() 10 0.2244 0.2291 0.2363 0.10% MLMG::mgVcycle_up::1 31 0.2272 0.2304 0.2329 0.09% MLCellLinOp::defineAuxData() 11 0.1883 0.2056 0.2172 0.09% Castro::fill_hybrid_hydro_source() 27 0.1778 0.1932 0.1998 0.08% Amr::grid_places() 6 0.196 0.1961 0.1961 0.08% AmrMesh::MakeNewGrids() 6 0.1954 0.1955 0.1956 0.08% MLMG::addInterpCorrection() 155 0.1342 0.1726 0.1909 0.08% MLMG::mgVcycle_bottom 65 0.1594 0.1676 0.1908 0.08% MLMG::getGradSolution() 10 0.1734 0.1761 0.1771 0.07% MLCellLinOp::compGrad() 18 0.1733 0.176 0.177 0.07% MLMG::miniCycle() 34 0.1594 0.1598 0.1605 0.07% MLPoisson::FFlux() 149 0.004833 0.04326 0.1583 0.06% AmrLevel::AmrLevel(dm) 5 0.1089 0.1338 0.1432 0.06% StateData::define() 25 0.1087 0.1336 0.143 0.06% Castro::swap_state_time_levels() 9 0.08392 0.1037 0.137 0.06% Castro::construct_new_hybrid_source() 9 0.1199 0.1291 0.1333 0.05% FabArray::mult() 56 0.1149 0.1276 0.1324 0.05% Castro::avgDown() 4 0.1213 0.1244 0.1278 0.05% Castro::avgDown(state_indx) 20 0.1213 0.1243 0.1278 0.05% amrex::average_down_w_geom 20 0.1212 0.1243 0.1277 0.05% Castro::errorEst() 6 0.109 0.1188 0.1232 0.05% MLMG::ResNormInf() 54 0.07443 0.08354 0.1065 0.04% MLCellLinOp::updateSolBC() 25 0.07859 0.08556 0.09529 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.0878 0.08899 0.09154 0.04% MLCellLinOp::defineBC() 11 0.0875 0.08849 0.08997 0.04% MLMG::mgVcycle_up::3 31 0.02682 0.06909 0.08384 0.03% MLMG::mgVcycle_down::2 31 0.07081 0.07409 0.07899 0.03% MLMG::MLResNormInf() 15 0.04575 0.06987 0.0785 0.03% Castro::construct_old_hybrid_source() 9 0.05799 0.06419 0.06662 0.03% FabArray::norminf(mask) 21 0.05749 0.06376 0.06605 0.03% BndryData::define() 27 0.05941 0.06061 0.06244 0.03% MLMG::interpCorrection_1 17 0.02846 0.0382 0.06049 0.02% Castro::apply_tagging_restrictions() 6 0.04903 0.05422 0.05605 0.02% MLMG::mgVcycle_up::2 31 0.05428 0.0547 0.05522 0.02% MLCellLinOp::updateCorBC() 17 0.05031 0.05151 0.05328 0.02% Castro::derive() 6 0.04459 0.04712 0.04961 0.02% AmrLevel::derive() 6 0.04457 0.0471 0.04958 0.02% Castro::buildMetrics() 5 0.03396 0.04103 0.04713 0.02% FabArray::norminf() 147 0.0373 0.03951 0.04237 0.02% Gravity::swapTimeLevels() 9 0.0333 0.03618 0.03917 0.02% MLMG::mgVcycle_down::3 31 0.03251 0.03351 0.03593 0.01% MLMG::actualBottomSolve() 31 4.158e-05 0.007847 0.03126 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.0208 0.02448 0.0309 0.01% MLCGSolver::bicgstab 7 0 0.007622 0.03049 0.01% MLMG::MLRhsNormInf() 10 0.02633 0.02888 0.0298 0.01% FabArrayBase::getCPC() 1440 0.02396 0.02538 0.02855 0.01% Castro::FluxRegFineAdd() 9 0.0271 0.02761 0.02797 0.01% TagBoxArray::mapPRD 6 0.007069 0.01399 0.02618 0.01% MultiFab::Multiply() 8 0.02219 0.02468 0.02553 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.02045 0.02113 0.02265 0.01% MLMG::compResidual() 1 0.01932 0.02088 0.02174 0.01% MLMG::mgVcycle_down::4 31 0.005966 0.009536 0.01947 0.01% Castro::build_fine_mask() 44 0.01741 0.01809 0.01943 0.01% MLLinOp::define() 11 0.003236 0.00984 0.01936 0.01% MLLinOp::defineGrids() 11 0.003169 0.009763 0.01926 0.01% FabArrayBase::CPC::define() 416 0.01318 0.01488 0.01799 0.01% MLLinOp::makeSubCommunicator() 11 0.001421 0.007881 0.01722 0.01% Castro::linear_to_hybrid_momentum() 1 0.01407 0.01573 0.01635 0.01% MLMG::mgVcycle_up::4 31 0.004808 0.007354 0.01452 0.01% MLPoisson::prepareForSolve() 11 0.01189 0.01211 0.0123 0.01% MLCellLinOp::prepareForSolve() 11 0.01184 0.01206 0.01225 0.00% Castro::enforce_consistent_e() 1 0.009394 0.01069 0.01125 0.00% AMRErrorTag::operator() 6 0.008881 0.01001 0.0105 0.00% average_down_faces 48 0.009453 0.009809 0.01029 0.00% Castro::apply_problem_tags() 6 0.006383 0.007361 0.008243 0.00% FabArrayBase::getFB() 1553 0.005609 0.00611 0.006558 0.00% MLCellLinOp::setLevelBC() 19 0.004149 0.005096 0.006221 0.00% FabArrayBase::TheFPinfo() 25 0.002646 0.004215 0.005827 0.00% DistributionMapping::LeastUsedCPUs() 5 0.002501 0.00411 0.005723 0.00% FPinfo::FPinfo() 3 0.002468 0.004039 0.005646 0.00% DistributionMapping::KnapSackDoIt() 3 0.002353 0.003921 0.005523 0.00% amrex::Dot() 115 0 0.001036 0.004145 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0008481 0.003393 0.00% FabArrayBase::FB::FB() 19 0.002488 0.002833 0.003191 0.00% TagBoxArray::collate() 6 0.0007945 0.001177 0.001476 0.00% OwnerMask() 6 0.0007683 0.0009049 0.001004 0.00% Castro::finalize_advance() 9 0.0003105 0.0004305 0.000502 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003683 0.0003875 0.0004078 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0003156 0.0003583 0.0003772 0.00% makeSFC 33 0.0002617 0.0002759 0.0002903 0.00% Castro::enforce_speed_limit() 87 0.0002546 0.000265 0.0002749 0.00% AmrMesh-cluster 1 0 3.378e-05 0.0001351 0.00% FabArray::RecvLayoutMask() 8 9.945e-05 0.0001067 0.0001182 0.00% DistributionMapping::Distribute() 35 0.0001111 0.0001134 0.0001168 0.00% Amr::writeSmallPlotFile() 1 9.253e-05 0.0001039 0.000114 0.00% ClusterList::intersect() 1 0 2.122e-05 8.488e-05 0.00% BoxList::parallelComplementIn() 12 5.153e-05 6.366e-05 8.42e-05 0.00% knapsack() 3 6.726e-05 7.297e-05 7.934e-05 0.00% Castro::create_source_corrector() 9 5.455e-05 5.979e-05 6.715e-05 0.00% Gravity::set_mass_offset() 6 4.178e-05 5.426e-05 5.951e-05 0.00% Amr::initSubcycle() 1 4.987e-05 5.035e-05 5.073e-05 0.00% Castro::retry_advance_ctu() 5 2.752e-05 3.352e-05 4.799e-05 0.00% BoxList::complementIn 12 2.295e-05 3.108e-05 3.966e-05 0.00% Castro::construct_new_rotation_source() 9 2.501e-05 2.728e-05 3.051e-05 0.00% Castro::construct_old_rotation_source() 9 2.222e-05 2.506e-05 2.938e-05 0.00% ClusterList::chop() 1 0 2.604e-06 1.042e-05 0.00% knapsack()swap 1 3.495e-06 3.908e-06 4.412e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-42-g7437aa17d36f) finalized