MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (23.08-19-ge7d3d5a4fad6) initialized Starting run at 07:02:39 UTC on 2023-08-07. Successfully read inputs file ... Castro git describe: 23.08 AMReX git describe: 23.08-19-ge7d3d5a4f Microphysics git describe: 23.08-3-g3d2714f8 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.034279418 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.030309203 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.367903763 Gravity::multilevel_solve_for_new_phi() time = 1.758209121 Gravity::fill_multipole_BCs() time = 0.374632719 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.60041199 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.881760134 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.363324583 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 20.29917051 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.657791323 Gravity::multilevel_solve_for_new_phi() time = 1.701201417 Castro::construct_new_gravity() time = 1.82502299 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207933e+26 ymom added: 1.075144428e+26 zmom added: -5.613375295e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.348566411 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.184461852e+27 ener added: 2.243834889e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.0249658652 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.190352593e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 37.65158066 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.031768468 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.409779294 Gravity::multilevel_solve_for_new_phi() time = 1.945606335 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849569753e+31 ener added: 2.785937322e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.840802683 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158191e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222073e+26 zmom added: -2.866993882e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.086588516 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.42923539 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.551111111e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.037367648 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253775e+40 ener added: 1.79168512e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.570335476e+22 ymom added: 1.534769107e+22 zmom added: 6.441948907e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.200226748 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.797800841 on level 1 Castro::reflux() at level 0 : time = 2.321515059 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.590375675 Gravity::multilevel_solve_for_new_phi() time = 1.53725885 Castro::construct_new_gravity() time = 1.655837477 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.864072325e+21 ymom added: -6.244635709e+21 zmom added: -1.246148803e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 2.067026145 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.14873037e+26 ener added: 4.251851852e+22 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029891032 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 5.112263512e+18 ymom added: -1.460517359e+19 zmom added: -8.220975323e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.145913344 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281876859e+29 ener added: 3.281875295e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02257831853 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.036240741e+24 ener added: 5.917722222e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 50.35148278 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031642235 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.412477338 Gravity::multilevel_solve_for_new_phi() time = 0.54350514 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.843654641 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157842738e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.071599004 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.35499134 on level 0 Castro::construct_old_gravity() time = 0.03993339 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691488e+40 ener added: 1.212217726e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.443638501e+22 ymom added: 5.312662293e+21 zmom added: 7.103005275e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.198288926 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.795012639 on level 1 Castro::reflux() at level 0 : time = 2.292090599 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.647437618 Gravity::multilevel_solve_for_new_phi() time = 1.217224669 Castro::construct_new_gravity() time = 1.331662529 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.108598872e+21 ymom added: -7.90284783e+21 zmom added: 1.10682106e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.04194399 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029042906 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -5.607146119e+17 ymom added: 2.17388754e+19 zmom added: 9.538641252e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.141801546 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.495038611e+26 ener added: 1.500090463e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02295010219 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.283851852e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.73256792 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.031785096 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.410033405 Gravity::multilevel_solve_for_new_phi() time = 0.538586883 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.861218168 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471615689e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.05557828 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.30061074 on level 0 Castro::construct_old_gravity() time = 0.039755683 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425501e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.203147947e+22 ymom added: 1.180591621e+22 zmom added: 8.023757211e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.198474875 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.885988615 on level 1 Castro::reflux() at level 0 : time = 2.380258417 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.658230511 Gravity::multilevel_solve_for_new_phi() time = 1.223264277 Castro::construct_new_gravity() time = 1.336346899 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405469507e+21 ymom added: -9.735672131e+21 zmom added: -7.437548959e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 2.113465122 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029516615 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.521656476e+18 ymom added: -2.667031699e+19 zmom added: -1.338681644e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.139902116 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.851958796e+25 ener added: 2.794684722e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02289359944 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.81008068 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.032220607 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.401474261 Gravity::multilevel_solve_for_new_phi() time = 0.537934957 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.852407393 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819051789e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.124818652 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.37324073 on level 0 Castro::construct_old_gravity() time = 0.044934226 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109508e+40 ener added: 1.662186972e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013636596e+23 ymom added: 1.829917012e+22 zmom added: 8.797079311e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.201285842 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.833333359 on level 1 Castro::reflux() at level 0 : time = 2.351504515 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.632634055 Gravity::multilevel_solve_for_new_phi() time = 1.187681115 Castro::construct_new_gravity() time = 1.302328854 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.798425632e+21 ymom added: -1.178643014e+22 zmom added: -5.143756067e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.010973273 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.0289507 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 2.013228434e+18 ymom added: -2.459866116e+19 zmom added: -1.244730437e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.142037945 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.005277778e+24 ener added: 3.882013889e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02296543507 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.814813006e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.65736794 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.577449006 seconds Ending run at 07:06:43 UTC on 2023-08-07. Run time = 243.7360104 Run time without initialization = 231.7826429 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 243.7 ... 243.7 ... 243.7 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 93.2045 102.4977 106.0004 43.49% Castro::computeTemp() 88 15.13 16.72 17.33 7.11% amrex::Copy() 736 13.09 14.4 14.97 6.14% check_for_negative_density() 9 0.7112 4.245 13.61 5.58% MultiFab::AddProduct() 4205 10.6 11.71 12.21 5.01% Castro::normalize_species() 87 10.07 11.22 11.68 4.79% FabArray::ParallelCopy_finish() 830 1.527 4.121 10.29 4.22% Castro::reset_internal_energy(Fab) 1476 8.413 9.257 9.684 3.97% Castro::estTimeStep() 26 4.01 5.614 9.235 3.79% FabArray::setVal() 5078 7.628 8.435 8.817 3.62% MultiFab::Subtract() 175 6.955 7.694 8.04 3.30% Castro::reset_internal_energy(MultiFab) 88 3.774 4.82 6.295 2.58% FabArray::sum() 4205 3.407 4.331 6.213 2.55% Gravity::fill_multipole_BCs() 11 4.348 4.705 5.482 2.25% Castro::enforce_min_density() 87 1.778 2.462 3.861 1.58% FluxRegister::Reflux() 24 3.023 3.408 3.619 1.48% FabArray::ParallelCopy_nowait() 830 2.097 2.248 2.576 1.06% MLPoisson::Fsmooth() 1376 2.133 2.358 2.46 1.01% Castro::FluxRegCrseInit 9 1.57 1.789 1.883 0.77% Castro::reflux() 4 1.557 1.744 1.823 0.75% Castro::wd_update() 6 0.5432 0.8995 1.787 0.73% FillPatchInterp(MF) 30 1.2 1.307 1.412 0.58% Amr::InitAmr() 1 1.228 1.228 1.228 0.50% amrex::Add() 143 1.096 1.164 1.218 0.50% Castro::initMFs() 5 1.115 1.147 1.167 0.48% Castro::construct_old_gravity_source() 18 0.9319 1.013 1.061 0.44% MLPoisson::Fapply() 326 0.9014 1.001 1.04 0.43% FillBoundary_finish() 1561 0.3526 0.5542 0.957 0.39% Castro::problem_post_timestep() 10 0.8532 0.8987 0.9449 0.39% FillPatchIterator::Initialize 62 0.8022 0.8951 0.9338 0.38% Castro::construct_new_gravity_source() 9 0.7414 0.8288 0.8656 0.36% Castro::volInBoundary() 42 0.6651 0.7327 0.8217 0.34% FabArray::Saxpy() 125 0.7114 0.7826 0.8119 0.33% VisMF::Write(FabArray) 3 0.6518 0.7307 0.786 0.32% StateData::FillBoundary(geom) 50 0.6026 0.6098 0.6206 0.25% Castro::construct_new_source() 63 0.4806 0.5396 0.5648 0.23% MultiFab::max() 18 0.02518 0.1995 0.5348 0.22% MLCellLinOp::applyBC() 1750 0.3661 0.4331 0.484 0.20% FillBoundary_nowait() 1561 0.3502 0.4014 0.4537 0.19% MultiFab::contains_nan() 18 0.3812 0.4261 0.4473 0.18% FabArray::Xpay() 291 0.3611 0.4002 0.4135 0.17% Castro::initialize_do_advance() 9 0.2442 0.2748 0.2904 0.12% Amr::writePlotFile() 2 0.1008 0.1596 0.2616 0.11% FabArray::setDomainBndry() 92 0.2072 0.2229 0.2328 0.10% Castro::construct_old_source() 63 0.1923 0.2086 0.2231 0.09% amrex::average_down 233 0.2004 0.2094 0.2132 0.09% Castro::fill_hybrid_hydro_source() 27 0.1729 0.1901 0.1975 0.08% MLCellLinOp::defineAuxData() 11 0.1578 0.1713 0.1774 0.07% MLPoisson::FFlux() 150 0.004567 0.04185 0.1536 0.06% Castro::initData() 1 0.1307 0.1447 0.152 0.06% MLCellLinOp::compGrad() 18 0.1269 0.1393 0.1448 0.06% FabArray::mult() 56 0.1211 0.1346 0.1404 0.06% StateData::define() 25 0.1196 0.1291 0.1371 0.06% Castro::evaluate_source_change() 193 0.1109 0.1181 0.1232 0.05% MLMG::addInterpCorrection() 155 0.1068 0.1182 0.1229 0.05% Castro::swap_state_time_levels() 9 0.09601 0.1128 0.1228 0.05% Gravity::get_old_grav_vector() 9 0.07994 0.0914 0.1027 0.04% AmrLevel::FillPatch() 62 0.08287 0.09142 0.1002 0.04% Castro::initialize_advance() 9 0.08347 0.09002 0.09869 0.04% Gravity::get_new_grav_vector() 10 0.08735 0.09477 0.0979 0.04% MLMG::prepareForSolve() 10 0.06953 0.08075 0.08712 0.04% MLCellLinOp::updateSolBC() 25 0.06699 0.06812 0.06935 0.03% Castro::do_new_sources() 9 0.0005875 0.01819 0.0686 0.03% MLMG::solve() 10 0.002862 0.04158 0.06602 0.03% FabArray::norminf(mask) 21 0.05386 0.06004 0.06255 0.03% Castro::apply_tagging_restrictions() 6 0.05403 0.05561 0.05619 0.02% AmrMesh::MakeNewGrids() 6 0.04654 0.04812 0.05182 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.03946 0.04321 0.04771 0.02% MLMG::ResNormInf() 54 0.001882 0.01684 0.04755 0.02% Castro::do_old_sources() 9 0.0006131 0.0168 0.04752 0.02% BndryData::define() 27 0.04466 0.04591 0.0473 0.02% MLCellLinOp::updateCorBC() 17 0.04547 0.04624 0.04702 0.02% FabArray::norminf() 117 0.03496 0.0389 0.04172 0.02% MLMG::MLResNormInf() 15 8.808e-05 0.0239 0.03467 0.01% MLCellLinOp::reflux() 30 0.004338 0.01159 0.03297 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.02045 0.02356 0.02898 0.01% MLCellLinOp::defineBC() 11 0.02613 0.02649 0.02696 0.01% MultiFab::Multiply() 8 0.02255 0.02547 0.02687 0.01% Castro::FluxRegFineAdd() 9 0.02586 0.02616 0.02638 0.01% Castro::build_fine_mask() 44 0.01941 0.02046 0.0224 0.01% StateDataPhysBCFunct::() 95 0.002397 0.008423 0.02139 0.01% MLMG::interpCorrection_1 17 0.01821 0.01886 0.02042 0.01% FabArrayBase::CPC::define() 416 0.01415 0.01605 0.01957 0.01% Castro::buildMetrics() 5 0.0155 0.01754 0.01903 0.01% Castro::Castro() 5 0.01634 0.01797 0.0188 0.01% Castro::linear_to_hybrid_momentum() 1 0.0142 0.01563 0.01616 0.01% Castro::construct_old_gravity() 9 0.0003833 0.004818 0.01587 0.01% MLLinOp::makeSubCommunicator() 11 0.0003725 0.007759 0.01576 0.01% Gravity::actual_solve_with_mlmg() 11 0.00948 0.01123 0.01542 0.01% FillPatchTwoLevels 31 0.0128 0.014 0.01524 0.01% Gravity::actual_multilevel_solve() 10 0.01071 0.01232 0.01306 0.01% Castro::finalize_do_advance() 9 0.009323 0.0112 0.01237 0.01% MLCellLinOp::prepareForSolve() 11 0.01152 0.01162 0.01169 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0001095 0.007385 0.01166 0.00% Castro::enforce_consistent_e() 1 0.009385 0.01043 0.01093 0.00% FabArrayBase::getCPC() 1440 0.01015 0.0104 0.01074 0.00% AMRErrorTag::operator() 6 0.008813 0.009901 0.01028 0.00% Gravity::update_max_rhs() 10 0.00815 0.008941 0.01011 0.00% Castro::apply_problem_tags() 6 0.006383 0.007145 0.007528 0.00% MLCellLinOp::setLevelBC() 19 0.004548 0.005658 0.007143 0.00% AmrLevel::derive() 6 0.004953 0.005535 0.006185 0.00% average_down_faces 48 0.00577 0.005836 0.005899 0.00% MLCGSolver::bicgstab 7 0 0.001291 0.005165 0.00% FabArray::FillBoundary() 1561 0.004492 0.004684 0.005146 0.00% Castro::finalize_advance() 9 0.000324 0.001344 0.004176 0.00% MLCellLinOp::smooth() 688 0.003994 0.004011 0.004024 0.00% Castro::construct_new_gravity() 9 0.0004305 0.001777 0.003838 0.00% FabArrayBase::getFB() 1561 0.003143 0.003303 0.003652 0.00% FabArrayBase::FB::FB() 19 0.002781 0.003096 0.003477 0.00% amrex::Dot() 115 0 0.000831 0.003324 0.00% FabArray::ParallelCopy() 830 0.003093 0.003155 0.003245 0.00% Gravity::test_composite_phi() 1 0.00225 0.002522 0.002724 0.00% main() 1 0.001591 0.001802 0.001997 0.00% FabArray::LinComb() 57 0 0.0004635 0.001854 0.00% MLCellLinOp::apply() 326 0.0009348 0.001104 0.001602 0.00% MLLinOp::defineGrids() 11 0.001417 0.001488 0.001563 0.00% Amr::regrid() 5 0.001035 0.001188 0.001424 0.00% Castro::clean_state() 87 0.00123 0.001293 0.001363 0.00% AmrLevel::FillCoarsePatch() 5 0.000925 0.001074 0.001243 0.00% Amr::coarseTimeStep() 5 0.0005094 0.001 0.001167 0.00% MLCGSolver::ParallelAllReduce 153 0 0.0002663 0.001065 0.00% TagBoxArray::collate() 6 0.0004893 0.0008429 0.001064 0.00% OwnerMask() 6 0.0007703 0.0008678 0.0009508 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009227 0.000934 0.0009396 0.00% MLCellLinOp::correctionResidual() 196 0.0007695 0.0007991 0.0008569 0.00% MLMG::mgVcycle() 65 0.0006053 0.0006219 0.0006428 0.00% Castro::do_advance_ctu() 5 0.0005931 0.0006087 0.0006332 0.00% FillPatchSingleLevel 92 0.0005337 0.0005525 0.0005761 0.00% Amr::grid_places() 6 0.0004045 0.0004624 0.0005243 0.00% TagBoxArray::mapPRD 6 0.0004559 0.0004654 0.0004786 0.00% FillPatchIterator::FillFromLevel0() 31 0.000408 0.0004349 0.0004707 0.00% Castro::subcycle_advance_ctu() 5 7.336e-05 0.0002017 0.0004524 0.00% MLMG:computeResOfCorrection() 155 0.0004028 0.0004108 0.0004224 0.00% MLMG::actualBottomSolve() 31 4.028e-05 0.0001344 0.0004143 0.00% MLCellLinOp::solutionResidual() 72 0.0003797 0.0003866 0.0003941 0.00% MLMG::oneIter() 31 0.0003523 0.0003638 0.0003738 0.00% Castro::computeNewDt() 8 0.0001102 0.0001751 0.0003458 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0003094 0.000321 0.0003309 0.00% MLMG::mgVcycle_down::0 31 0.0002899 0.0002935 0.000297 0.00% Castro::enforce_speed_limit() 87 0.0002428 0.0002579 0.0002667 0.00% Amr::defBaseLevel() 1 0.000236 0.0002507 0.0002651 0.00% Gravity::solve_phi_with_mlmg() 11 0.000238 0.0002459 0.0002584 0.00% MLMG::mgVcycle_down::1 31 0.0002511 0.0002546 0.0002561 0.00% Amr::timeStep() 9 0.0001829 0.0002081 0.0002235 0.00% MLMG::mgVcycle_bottom 65 0.0002167 0.0002186 0.0002211 0.00% Castro::post_regrid() 10 0.0001811 0.0001967 0.0002112 0.00% MLMG::mgVcycle_down::2 31 0.0002053 0.0002074 0.0002094 0.00% MLMG::mgVcycle_down::3 31 0.0001987 0.0002052 0.0002091 0.00% MLMG::mgVcycle_up::0 31 0.0001904 0.0001953 0.0002017 0.00% AmrLevel::AmrLevel(dm) 5 0.0001896 0.0001951 0.0002002 0.00% Castro::check_for_nan() 18 0.0001786 0.0001868 0.0001984 0.00% makeSFC 33 0.0001812 0.0001864 0.0001919 0.00% MLMG::mgVcycle_down::4 31 0.0001761 0.0001808 0.0001898 0.00% FabArrayBase::TheFPinfo() 25 0.0001787 0.0001836 0.000187 0.00% Castro::react_state() 18 0.0001337 0.0001514 0.0001778 0.00% Amr::FinalizeInit() 1 2.952e-05 6.758e-05 0.0001689 0.00% MLMG::mgVcycle_up::1 31 0.0001381 0.000146 0.0001515 0.00% Castro::post_timestep() 9 7.605e-05 9.789e-05 0.000148 0.00% FPinfo::FPinfo() 3 0.0001254 0.0001345 0.0001476 0.00% MLMG::mgVcycle_up::2 31 0.0001322 0.0001383 0.000144 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001349 0.0001381 0.0001419 0.00% MLMG::mgVcycle_up::4 31 0.0001159 0.0001219 0.0001328 0.00% MLMG::mgVcycle_up::3 31 0.0001294 0.0001314 0.0001323 0.00% DistributionMapping::Distribute() 35 0.000114 0.0001191 0.0001249 0.00% Castro::expand_state() 13 9.98e-05 0.0001097 0.0001245 0.00% Castro::advance() 5 8.965e-05 0.0001011 0.0001199 0.00% Castro::errorEst() 6 0.0001065 0.0001117 0.0001196 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001152 0.0001171 0.000119 0.00% MLPoisson::define() 11 0.0001083 0.0001126 0.0001185 0.00% FabArray::RecvLayoutMask() 8 9.523e-05 0.0001044 0.0001163 0.00% Castro::apply_source_to_state() 18 0.0001008 0.0001064 0.0001102 0.00% Amr::writeSmallPlotFile() 1 9.547e-05 9.913e-05 0.0001019 0.00% ClusterList::intersect() 1 0 2.471e-05 9.885e-05 0.00% Gravity::swapTimeLevels() 9 8.378e-05 8.892e-05 9.802e-05 0.00% MLMG::computeResidual() 31 8.473e-05 8.702e-05 8.929e-05 0.00% MLLinOp::define() 11 6.634e-05 7.138e-05 8.449e-05 0.00% MLMG::computeMLResidual() 15 7.841e-05 7.99e-05 8.143e-05 0.00% Castro::avgDown(state_indx) 20 6.833e-05 7.333e-05 8.137e-05 0.00% knapsack() 3 6.839e-05 7.207e-05 7.76e-05 0.00% Castro::create_source_corrector() 9 5.132e-05 6.151e-05 7.317e-05 0.00% MLMG::miniCycle() 34 6.925e-05 6.996e-05 7.066e-05 0.00% Gravity::set_mass_offset() 6 3.176e-05 4.686e-05 7.029e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.308e-05 6.616e-05 6.816e-05 0.00% MLMG::getGradSolution() 10 6.066e-05 6.226e-05 6.686e-05 0.00% amrex::average_down_w_geom 20 5.054e-05 5.431e-05 5.983e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.661e-05 5.112e-05 5.53e-05 0.00% Amr::initSubcycle() 1 4.846e-05 5.217e-05 5.408e-05 0.00% Castro::construct_old_hybrid_source() 9 3.278e-05 4.213e-05 5.141e-05 0.00% MLPoisson::prepareForSolve() 11 4.715e-05 4.868e-05 5.121e-05 0.00% BoxList::parallelComplementIn() 12 2.59e-05 3.698e-05 4.99e-05 0.00% Castro::construct_new_hybrid_source() 9 4.704e-05 4.756e-05 4.793e-05 0.00% Castro::init(old) 3 4.252e-05 4.46e-05 4.585e-05 0.00% AmrMesh-cluster 1 0 1.128e-05 4.511e-05 0.00% Castro::post_init() 1 2.852e-05 3.314e-05 4.176e-05 0.00% MLMG::compResidual() 1 2.951e-05 3.162e-05 3.775e-05 0.00% DistributionMapping::KnapSackDoIt() 3 3.217e-05 3.355e-05 3.518e-05 0.00% MLMG::MLRhsNormInf() 10 3.216e-05 3.347e-05 3.499e-05 0.00% Castro::avgDown() 4 2.975e-05 3.15e-05 3.331e-05 0.00% Castro::construct_old_rotation_source() 9 2.362e-05 2.522e-05 2.871e-05 0.00% BoxList::complementIn 12 2.274e-05 2.474e-05 2.807e-05 0.00% Amr::bldFineLevels() 1 1.67e-05 2.187e-05 2.737e-05 0.00% Castro::computeInitialDt() 2 1.857e-05 2.206e-05 2.679e-05 0.00% Castro::retry_advance_ctu() 5 2.085e-05 2.27e-05 2.588e-05 0.00% Castro::derive() 6 1.821e-05 1.999e-05 2.427e-05 0.00% Castro::construct_new_rotation_source() 9 2.088e-05 2.205e-05 2.368e-05 0.00% Amr::InitializeInit() 1 7.514e-06 1.115e-05 1.777e-05 0.00% Castro::init() 1 1.669e-05 1.708e-05 1.746e-05 0.00% ClusterList::chop() 1 0 2.885e-06 1.154e-05 0.00% Amr::init() 1 6.35e-06 7.26e-06 8.652e-06 0.00% Amr::initialInit() 1 4.768e-06 6.206e-06 7.933e-06 0.00% knapsack()swap 1 3.5e-06 3.712e-06 3.962e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 243.7 243.7 243.7 100.00% Amr::coarseTimeStep() 5 231.2 231.2 231.2 94.86% Amr::timeStep() 9 229.9 229.9 229.9 94.30% Castro::advance() 5 201.8 201.8 201.8 82.80% Castro::subcycle_advance_ctu() 5 192.8 193.2 194.1 79.65% Castro::do_advance_ctu() 5 192.8 193.2 194.1 79.65% Castro::construct_ctu_hydro_source() 9 116.7 116.9 117 47.99% Castro::clean_state() 87 72.88 82.23 86.45 35.47% Castro::computeTemp() 88 46.38 52.15 54.66 22.43% Castro::reset_internal_energy(MultiFab) 88 31.25 35.43 37.33 15.32% Castro::evaluate_source_change() 193 17.95 20.38 22.95 9.42% Castro::enforce_min_density() 87 17.21 19.75 21.2 8.70% Castro::post_regrid() 10 16.32 17.82 18.54 7.61% Amr::regrid() 5 16.13 17.47 18.1 7.43% Castro::initialize_do_advance() 9 14.96 15.36 15.54 6.37% amrex::Copy() 736 13.09 14.4 14.97 6.14% AmrLevel::FillPatch() 62 7.93 9.91 14.51 5.95% check_for_negative_density() 9 0.7112 4.245 13.61 5.58% FillPatchIterator::Initialize 62 6.172 8.212 12.94 5.31% FabArray::ParallelCopy() 830 3.737 6.433 12.53 5.14% Gravity::solve_phi_with_mlmg() 11 11.36 11.72 12.49 5.12% MultiFab::AddProduct() 4205 10.6 11.71 12.21 5.01% Gravity::multilevel_solve_for_new_phi() 10 11.07 11.42 12.19 5.00% Gravity::actual_multilevel_solve() 10 11.03 11.38 12.14 4.98% Castro::normalize_species() 87 10.07 11.22 11.68 4.79% Castro::do_old_sources() 9 11.33 11.4 11.49 4.71% Castro::do_new_sources() 9 9.365 10.6 11.14 4.57% FillPatchSingleLevel 92 3.851 5.966 10.85 4.45% Amr::init() 1 10.73 10.73 10.73 4.40% Castro::post_timestep() 9 10.38 10.38 10.38 4.26% FabArray::ParallelCopy_finish() 830 1.527 4.121 10.29 4.22% FillPatchIterator::FillFromLevel0() 31 3.475 5.523 10.29 4.22% Amr::initialInit() 1 10.12 10.12 10.12 4.15% Castro::reset_internal_energy(Fab) 1476 8.413 9.257 9.684 3.97% Castro::reflux() 4 8.595 9.133 9.344 3.83% Castro::estTimeStep() 26 4.01 5.614 9.235 3.79% Castro::initialize_advance() 9 7.678 8.595 9.026 3.70% FabArray::setVal() 5078 7.628 8.435 8.817 3.62% MultiFab::Subtract() 175 6.955 7.694 8.04 3.30% Castro::construct_new_gravity() 9 6.48 6.816 7.567 3.10% Gravity::actual_solve_with_mlmg() 11 6.901 6.909 6.914 2.84% Castro::expand_state() 13 5.166 5.687 6.58 2.70% MLMG::solve() 10 6.386 6.394 6.412 2.63% Amr::FinalizeInit() 1 5.875 6.016 6.379 2.62% FabArray::sum() 4205 3.407 4.331 6.213 2.55% Gravity::fill_multipole_BCs() 11 4.433 4.787 5.556 2.28% MLMG::oneIter() 31 5.158 5.178 5.189 2.13% FluxRegister::Reflux() 24 3.869 4.412 4.666 1.91% MLMG::mgVcycle() 65 4.366 4.435 4.475 1.84% Castro::construct_old_gravity() 9 0.7165 1.792 4.44 1.82% Gravity::get_old_grav_vector() 9 0.7086 1.787 4.44 1.82% Castro::finalize_do_advance() 9 1.327 2.234 4.32 1.77% Amr::InitializeInit() 1 3.746 4.108 4.25 1.74% Amr::defBaseLevel() 1 3.746 4.108 4.25 1.74% Castro::problem_post_timestep() 10 2.819 3.242 4.07 1.67% Castro::computeNewDt() 8 2.176 2.733 3.903 1.60% MLCellLinOp::smooth() 688 3.302 3.375 3.427 1.41% Castro::post_init() 1 2.521 2.69 3.022 1.24% Castro::FluxRegCrseInit 9 1.881 2.209 2.856 1.17% Castro::initData() 1 2.294 2.632 2.79 1.14% FabArray::ParallelCopy_nowait() 830 2.158 2.309 2.64 1.08% MLPoisson::Fsmooth() 1376 2.133 2.358 2.46 1.01% Castro::wd_update() 6 1.209 1.632 2.46 1.01% MLMG::mgVcycle_down::0 31 2.004 2.061 2.112 0.87% Castro::construct_new_source() 63 1.772 1.98 2.063 0.85% MLCellLinOp::applyBC() 1750 1.265 1.4 1.788 0.73% FillPatchIterator::FillFromTwoLevels() 31 1.499 1.572 1.646 0.68% FillPatchTwoLevels 31 1.499 1.572 1.645 0.67% Amr::bldFineLevels() 1 1.324 1.497 1.573 0.65% Castro::Castro() 5 1.414 1.432 1.455 0.60% FabArray::FillBoundary() 1561 0.7807 0.9667 1.422 0.58% MLMG::mgVcycle_up::0 31 1.381 1.407 1.421 0.58% FillPatchInterp(MF) 30 1.2 1.307 1.412 0.58% MLCellLinOp::apply() 326 1.294 1.316 1.343 0.55% Amr::InitAmr() 1 1.228 1.228 1.228 0.50% amrex::Add() 143 1.096 1.164 1.218 0.50% Amr::writePlotFile() 2 1.178 1.178 1.178 0.48% Castro::initMFs() 5 1.12 1.152 1.173 0.48% Castro::construct_old_source() 63 0.9446 1.028 1.076 0.44% Castro::construct_old_gravity_source() 18 0.9319 1.013 1.061 0.44% MLCellLinOp::solutionResidual() 72 1.013 1.035 1.043 0.43% MLPoisson::Fapply() 326 0.9014 1.001 1.04 0.43% Castro::computeInitialDt() 2 0.5175 0.6588 1.021 0.42% FillBoundary_finish() 1561 0.3526 0.5542 0.957 0.39% Castro::construct_new_gravity_source() 9 0.7414 0.8288 0.8656 0.36% MLCellLinOp::correctionResidual() 196 0.8142 0.824 0.8319 0.34% Gravity::get_new_grav_vector() 10 0.7905 0.8016 0.8267 0.34% Castro::volInBoundary() 42 0.6652 0.7328 0.8217 0.34% FabArray::Saxpy() 125 0.7114 0.7826 0.8119 0.33% VisMF::Write(FabArray) 3 0.6518 0.7307 0.786 0.32% MLMG:computeResOfCorrection() 155 0.6627 0.6939 0.7081 0.29% StateDataPhysBCFunct::() 95 0.605 0.6182 0.6252 0.26% StateData::FillBoundary(geom) 50 0.6026 0.6098 0.6206 0.25% Gravity::update_max_rhs() 10 0.08762 0.2605 0.5919 0.24% MLMG::computeResWithCrseSolFineCor() 17 0.4658 0.4975 0.5791 0.24% MultiFab::max() 18 0.02518 0.1995 0.5348 0.22% MLMG::computeMLResidual() 15 0.4249 0.4565 0.5111 0.21% Castro::init() 1 0.4657 0.4694 0.4745 0.19% AmrLevel::FillCoarsePatch() 5 0.4657 0.4694 0.4745 0.19% FillBoundary_nowait() 1561 0.3562 0.4078 0.4603 0.19% Castro::check_for_nan() 18 0.3814 0.4263 0.4475 0.18% MultiFab::contains_nan() 18 0.3812 0.4261 0.4473 0.18% Gravity::test_composite_phi() 1 0.4192 0.4196 0.4199 0.17% FabArray::Xpay() 291 0.3611 0.4002 0.4135 0.17% MLMG::mgVcycle_down::1 31 0.3188 0.342 0.3995 0.16% MLCellLinOp::reflux() 30 0.1833 0.2355 0.3717 0.15% Castro::init(old) 3 0.3607 0.3667 0.3713 0.15% MLMG::computeResidual() 31 0.3473 0.3548 0.3586 0.15% Castro::apply_source_to_state() 18 0.288 0.315 0.326 0.13% MLPoisson::define() 11 0.2928 0.3017 0.3109 0.13% Castro::react_state() 18 0.2481 0.2748 0.2886 0.12% amrex::average_down 233 0.2598 0.2712 0.287 0.12% MLMG::prepareForSolve() 10 0.2127 0.2261 0.2348 0.10% FabArray::setDomainBndry() 92 0.2072 0.2229 0.2328 0.10% MLMG::mgVcycle_up::1 31 0.2168 0.2217 0.2246 0.09% MLCellLinOp::defineAuxData() 11 0.185 0.2011 0.2099 0.09% Castro::fill_hybrid_hydro_source() 27 0.1729 0.1901 0.1975 0.08% Amr::grid_places() 6 0.192 0.192 0.1921 0.08% AmrMesh::MakeNewGrids() 6 0.1915 0.1916 0.1917 0.08% MLMG::mgVcycle_bottom 65 0.1554 0.1638 0.1869 0.08% MLMG::addInterpCorrection() 155 0.1263 0.1664 0.1853 0.08% MLMG::getGradSolution() 10 0.1735 0.1752 0.1759 0.07% MLCellLinOp::compGrad() 18 0.1735 0.1751 0.1758 0.07% MLMG::miniCycle() 34 0.1549 0.1559 0.1569 0.06% MLPoisson::FFlux() 150 0.004567 0.04185 0.1536 0.06% FabArray::mult() 56 0.1211 0.1346 0.1404 0.06% AmrLevel::AmrLevel(dm) 5 0.1198 0.1293 0.1373 0.06% StateData::define() 25 0.1196 0.1291 0.1371 0.06% Castro::construct_new_hybrid_source() 9 0.1154 0.1272 0.1323 0.05% Castro::avgDown() 4 0.1201 0.1241 0.1262 0.05% Castro::avgDown(state_indx) 20 0.1201 0.1241 0.1262 0.05% amrex::average_down_w_geom 20 0.12 0.124 0.1261 0.05% Castro::errorEst() 6 0.1126 0.1205 0.1243 0.05% Castro::swap_state_time_levels() 9 0.09601 0.1128 0.1228 0.05% MLCellLinOp::updateSolBC() 25 0.07697 0.09226 0.1199 0.05% MLMG::ResNormInf() 54 0.07323 0.08567 0.1096 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08853 0.09025 0.09188 0.04% MLCellLinOp::defineBC() 11 0.08988 0.09076 0.09157 0.04% MLMG::mgVcycle_up::3 31 0.02475 0.0666 0.08102 0.03% MLMG::mgVcycle_down::2 31 0.06802 0.07244 0.08085 0.03% MLMG::MLResNormInf() 15 0.04301 0.06585 0.07401 0.03% Castro::construct_old_hybrid_source() 9 0.0576 0.06302 0.06532 0.03% BndryData::define() 27 0.06155 0.06297 0.06363 0.03% FabArray::norminf(mask) 21 0.05386 0.06004 0.06255 0.03% MLMG::interpCorrection_1 17 0.02569 0.03858 0.05928 0.02% Castro::apply_tagging_restrictions() 6 0.05403 0.05561 0.05619 0.02% MLMG::mgVcycle_up::2 31 0.05032 0.05172 0.05284 0.02% MLCellLinOp::updateCorBC() 17 0.05027 0.05116 0.05162 0.02% Castro::buildMetrics() 5 0.0387 0.04392 0.05144 0.02% Castro::derive() 6 0.04323 0.04775 0.05045 0.02% AmrLevel::derive() 6 0.04321 0.04773 0.05043 0.02% Gravity::swapTimeLevels() 9 0.03679 0.03978 0.04479 0.02% FabArray::norminf() 117 0.03496 0.0389 0.04172 0.02% MLMG::mgVcycle_down::3 31 0.02977 0.03117 0.03408 0.01% MLMG::actualBottomSolve() 31 4.028e-05 0.008051 0.03208 0.01% MLCGSolver::bicgstab 7 0 0.00783 0.03132 0.01% FabArrayBase::getCPC() 1440 0.02431 0.02644 0.03003 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.02045 0.02356 0.02898 0.01% MLMG::MLRhsNormInf() 10 0.02486 0.02753 0.02855 0.01% MultiFab::Multiply() 8 0.02255 0.02547 0.02687 0.01% Castro::FluxRegFineAdd() 9 0.02586 0.02616 0.02638 0.01% Castro::build_fine_mask() 44 0.01941 0.02046 0.0224 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01467 0.01657 0.02145 0.01% MLMG::compResidual() 1 0.01992 0.02051 0.02073 0.01% TagBoxArray::mapPRD 6 0.005805 0.0101 0.02011 0.01% FabArrayBase::CPC::define() 416 0.01415 0.01605 0.01957 0.01% MLMG::mgVcycle_down::4 31 0.005658 0.008991 0.01839 0.01% MLLinOp::define() 11 0.002372 0.009728 0.01782 0.01% MLLinOp::defineGrids() 11 0.002305 0.009656 0.01774 0.01% Castro::linear_to_hybrid_momentum() 1 0.0142 0.01563 0.01616 0.01% MLLinOp::makeSubCommunicator() 11 0.0003725 0.007759 0.01576 0.01% MLMG::mgVcycle_up::4 31 0.004636 0.007017 0.01388 0.01% FabArrayBase::TheFPinfo() 25 0.0004809 0.007755 0.01203 0.00% FPinfo::FPinfo() 3 0.0002964 0.007571 0.01185 0.00% MLPoisson::prepareForSolve() 11 0.01157 0.01166 0.01174 0.00% DistributionMapping::KnapSackDoIt() 3 0.0001689 0.007437 0.0117 0.00% MLCellLinOp::prepareForSolve() 11 0.01152 0.01162 0.01169 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0001095 0.007385 0.01166 0.00% Castro::enforce_consistent_e() 1 0.009385 0.01043 0.01093 0.00% AMRErrorTag::operator() 6 0.008813 0.009901 0.01028 0.00% average_down_faces 48 0.009017 0.009496 0.009923 0.00% Castro::apply_problem_tags() 6 0.006383 0.007145 0.007528 0.00% MLCellLinOp::setLevelBC() 19 0.004548 0.005658 0.007143 0.00% FabArrayBase::getFB() 1561 0.005997 0.006399 0.006829 0.00% Castro::finalize_advance() 9 0.000324 0.001344 0.004176 0.00% FabArrayBase::FB::FB() 19 0.002781 0.003096 0.003477 0.00% amrex::Dot() 115 0 0.000831 0.003324 0.00% MLCGSolver::ParallelAllReduce 153 0 0.0007034 0.002814 0.00% FabArray::LinComb() 57 0 0.0004635 0.001854 0.00% TagBoxArray::collate() 6 0.0004893 0.0008429 0.001064 0.00% OwnerMask() 6 0.0007703 0.0008678 0.0009508 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0004034 0.0004097 0.0004168 0.00% makeSFC 33 0.0002883 0.0002925 0.0002984 0.00% Castro::enforce_speed_limit() 87 0.0002428 0.0002579 0.0002667 0.00% AmrMesh-cluster 1 0 3.888e-05 0.0001555 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001116 0.0001218 0.0001426 0.00% DistributionMapping::Distribute() 35 0.000114 0.0001191 0.0001249 0.00% FabArray::RecvLayoutMask() 8 9.523e-05 0.0001044 0.0001163 0.00% Amr::writeSmallPlotFile() 1 9.547e-05 9.913e-05 0.0001019 0.00% ClusterList::intersect() 1 0 2.471e-05 9.885e-05 0.00% knapsack() 3 7.189e-05 7.578e-05 8.124e-05 0.00% BoxList::parallelComplementIn() 12 4.864e-05 6.172e-05 7.495e-05 0.00% Castro::create_source_corrector() 9 5.132e-05 6.151e-05 7.317e-05 0.00% Gravity::set_mass_offset() 6 3.176e-05 4.686e-05 7.029e-05 0.00% Amr::initSubcycle() 1 4.846e-05 5.217e-05 5.408e-05 0.00% Castro::construct_old_rotation_source() 9 2.362e-05 2.522e-05 2.871e-05 0.00% BoxList::complementIn 12 2.274e-05 2.474e-05 2.807e-05 0.00% Castro::retry_advance_ctu() 5 2.085e-05 2.27e-05 2.588e-05 0.00% Castro::construct_new_rotation_source() 9 2.088e-05 2.205e-05 2.368e-05 0.00% ClusterList::chop() 1 0 2.885e-06 1.154e-05 0.00% knapsack()swap 1 3.5e-06 3.712e-06 3.962e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (23.08-19-ge7d3d5a4fad6) finalized