Initializing AMReX (v2024-42-g7437aa17d36f)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-42-g7437aa17d36f) initialized Starting run at 06:54:37 UTC on 2024-03-29. Successfully read inputs file ... Castro git describe: 24.03-37-g6b90dd1c0 AMReX git describe: v2024-42-g7437aa17d Microphysics git describe: 24.03-23-g3112c13a reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.032920848 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031855654 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.371274614 Gravity::multilevel_solve_for_new_phi() time = 1.743459815 Gravity::fill_multipole_BCs() time = 0.370871579 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.533281292 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.93634865 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.378884759 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.11330545 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.610133535 Gravity::multilevel_solve_for_new_phi() time = 1.643581853 Castro::construct_new_gravity() time = 1.7587635 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207933e+26 ymom added: 1.108730196e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.368210545 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02449353702 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.3426259 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032564133 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.413735349 Gravity::multilevel_solve_for_new_phi() time = 1.962416714 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.795991682 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.057508614 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.69706622 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.034641645 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.571742739e+22 ymom added: 1.505254316e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.198842396 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.803741667 on level 1 Castro::reflux() at level 0 : time = 2.311219809 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.431711533 Gravity::multilevel_solve_for_new_phi() time = 1.403412873 Castro::construct_new_gravity() time = 1.51642094 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 2.016247666 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029962935 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764863347e+18 ymom added: -1.361438167e+19 zmom added: -8.219114469e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.143953607 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02260318548 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 50.32729065 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.03188873 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402084096 Gravity::multilevel_solve_for_new_phi() time = 0.533115844 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.749618204 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 1.997689577 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.67241998 on level 0 Castro::construct_old_gravity() time = 0.032265175 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.443251529e+22 ymom added: 5.017514388e+21 zmom added: 7.103005275e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.196897662 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.848209866 on level 1 Castro::reflux() at level 0 : time = 2.314861663 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.445539878 Gravity::multilevel_solve_for_new_phi() time = 1.010184219 Castro::construct_new_gravity() time = 1.122847363 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.032560768 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029883753 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.81826608e+17 ymom added: 2.351779725e+19 zmom added: 9.539392295e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.153142314 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02296980634 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.68165505 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032926561 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.408019862 Gravity::multilevel_solve_for_new_phi() time = 0.538265362 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.77613439 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.001487187 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.29692046 on level 0 Castro::construct_old_gravity() time = 0.034610625 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.197093646e+22 ymom added: 1.239621202e+22 zmom added: 8.023901326e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.194373875 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.84284242 on level 1 Castro::reflux() at level 0 : time = 2.283706775 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.424326156 Gravity::multilevel_solve_for_new_phi() time = 0.97105099 Castro::construct_new_gravity() time = 1.084329905 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.980576917 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.033723433 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598361396e+18 ymom added: -2.567952508e+19 zmom added: -1.336958118e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.161100651 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02323170775 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.13966912 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.03210662 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.404843233 Gravity::multilevel_solve_for_new_phi() time = 0.533987811 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.704980427 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 1.999550084 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.18796046 on level 0 Castro::construct_old_gravity() time = 0.036121943 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013667806e+23 ymom added: 1.888946593e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.197495782 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.910398957 on level 1 Castro::reflux() at level 0 : time = 2.344145461 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.431236823 Gravity::multilevel_solve_for_new_phi() time = 0.994820688 Castro::construct_new_gravity() time = 1.108617803 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.002303499 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029654979 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.726901398e+18 ymom added: -2.54093091e+19 zmom added: -1.252338095e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.140231352 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02326181321 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 46.98330213 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.531533381 seconds Ending run at 06:58:39 UTC on 2024-03-29. Run time = 242.6225857 Run time without initialization = 231.0083022 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 242.6 ... 242.6 ... 242.6 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 93.6673 103.7298 107.5595 44.33% Castro::computeTemp() 88 14.69 16.47 17.19 7.08% check_for_negative_density() 9 0.7927 4.62 14.77 6.09% amrex::Copy() 679 12.55 14.13 14.72 6.07% MultiFab::AddProduct() 4205 10.46 11.73 12.2 5.03% Castro::normalize_species() 87 9.99 11.26 11.74 4.84% FabArray::ParallelCopy_finish() 830 1.652 4.149 10.89 4.49% Castro::reset_internal_energy(Fab) 1476 8.42 9.27 9.562 3.94% Castro::estTimeStep() 26 4.012 5.382 8.97 3.70% FabArray::setVal() 5070 7.397 8.335 8.713 3.59% MultiFab::Subtract() 175 6.996 7.839 8.16 3.36% Castro::reset_internal_energy(MultiFab) 88 3.367 4.2 4.693 1.93% Gravity::fill_multipole_BCs() 11 4.287 4.395 4.643 1.91% FabArray::sum() 4205 3.472 3.894 4.043 1.67% FluxRegister::Reflux() 24 2.963 3.358 3.516 1.45% FabArray::ParallelCopy_nowait() 830 2.096 2.299 2.659 1.10% Castro::enforce_min_density() 87 1.539 2.008 2.644 1.09% MLPoisson::Fsmooth() 1376 2.139 2.405 2.527 1.04% Castro::wd_update() 6 0.5667 1.008 1.989 0.82% Castro::FluxRegCrseInit 9 1.497 1.673 1.79 0.74% Castro::reflux() 4 1.504 1.695 1.772 0.73% FillPatchInterp(MF) 30 1.259 1.37 1.476 0.61% Castro::construct_old_gravity_source() 18 1.016 1.101 1.178 0.49% amrex::Add() 136 0.9842 1.108 1.16 0.48% Amr::InitAmr() 1 1.153 1.153 1.153 0.48% Castro::initMFs() 5 1.069 1.116 1.147 0.47% Castro::construct_new_gravity_source() 9 0.8624 0.9928 1.076 0.44% MLPoisson::Fapply() 318 0.9181 1.03 1.073 0.44% Castro::problem_post_timestep() 10 0.8009 0.8532 0.9352 0.39% FillPatchIterator::Initialize 62 0.7677 0.8505 0.9179 0.38% Castro::volInBoundary() 42 0.656 0.7777 0.8896 0.37% FillBoundary_finish() 1553 0.2785 0.4584 0.8886 0.37% FabArray::Saxpy() 182 0.7102 0.7973 0.8298 0.34% VisMF::Write(FabArray) 3 0.5848 0.6568 0.6907 0.28% StateData::FillBoundary(geom) 50 0.5886 0.6008 0.6104 0.25% Castro::construct_new_source() 63 0.4753 0.5268 0.5458 0.22% MultiFab::max() 18 0.0182 0.1609 0.507 0.21% MLCellLinOp::applyBC() 1742 0.3599 0.4295 0.4851 0.20% MultiFab::contains_nan() 18 0.3984 0.4453 0.4616 0.19% FillBoundary_nowait() 1553 0.3429 0.378 0.4317 0.18% FabArray::Xpay() 283 0.3464 0.3869 0.4022 0.17% Castro::initialize_do_advance() 9 0.2432 0.272 0.2891 0.12% FabArray::setDomainBndry() 92 0.1984 0.222 0.2374 0.10% Amr::writePlotFile() 2 0.08614 0.1322 0.2343 0.10% amrex::average_down 233 0.1965 0.2061 0.2105 0.09% Castro::construct_old_source() 63 0.18 0.1973 0.2065 0.09% Castro::fill_hybrid_hydro_source() 27 0.1707 0.192 0.201 0.08% MLCellLinOp::defineAuxData() 11 0.1575 0.1728 0.1786 0.07% Castro::initData() 1 0.1252 0.1504 0.1607 0.07% MLPoisson::FFlux() 150 0.004709 0.04274 0.1566 0.06% MLCellLinOp::compGrad() 18 0.1221 0.136 0.1408 0.06% StateData::define() 25 0.1037 0.1192 0.1341 0.06% FabArray::mult() 56 0.1116 0.1259 0.1312 0.05% MLMG::addInterpCorrection() 155 0.1092 0.1221 0.127 0.05% Castro::evaluate_source_change() 193 0.1028 0.1118 0.1159 0.05% Gravity::get_new_grav_vector() 10 0.08707 0.09692 0.1023 0.04% Castro::swap_state_time_levels() 9 0.09163 0.09541 0.09956 0.04% Gravity::get_old_grav_vector() 9 0.08233 0.08836 0.09164 0.04% Castro::initialize_advance() 9 0.068 0.07894 0.08714 0.04% MLMG::prepareForSolve() 10 0.06112 0.06863 0.07451 0.03% MLMG::solve() 10 0.001188 0.04289 0.06921 0.03% MLCellLinOp::updateSolBC() 25 0.06763 0.0682 0.06909 0.03% FabArray::norminf(mask) 21 0.05551 0.06294 0.0662 0.03% Castro::apply_tagging_restrictions() 6 0.04788 0.05386 0.05655 0.02% AmrLevel::FillPatch() 62 0.04957 0.05285 0.05451 0.02% AmrMesh::MakeNewGrids() 6 0.04712 0.04912 0.05354 0.02% MLCellLinOp::updateCorBC() 17 0.04579 0.04621 0.04685 0.02% BndryData::define() 27 0.04113 0.04263 0.04393 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.0267 0.03411 0.04346 0.02% MLMG::ResNormInf() 54 0.0007724 0.01418 0.04295 0.02% FabArray::norminf() 147 0.03663 0.03888 0.04134 0.02% MLMG::MLResNormInf() 15 0.0001271 0.02443 0.03561 0.01% MLCellLinOp::reflux() 30 0.004433 0.01188 0.03373 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01971 0.02356 0.02955 0.01% StateDataPhysBCFunct::() 95 0.002191 0.009805 0.029 0.01% Castro::FluxRegFineAdd() 9 0.0259 0.02645 0.02679 0.01% MLCellLinOp::defineBC() 11 0.0261 0.02633 0.02653 0.01% MultiFab::Multiply() 8 0.02165 0.0245 0.02563 0.01% Castro::construct_old_gravity() 9 0.0004194 0.008248 0.02129 0.01% MLMG::interpCorrection_1 17 0.01884 0.01945 0.02064 0.01% MLLinOp::makeSubCommunicator() 11 0.0006502 0.008456 0.01958 0.01% Castro::build_fine_mask() 44 0.01672 0.01824 0.01902 0.01% FabArrayBase::CPC::define() 416 0.01285 0.01477 0.01805 0.01% Castro::linear_to_hybrid_momentum() 1 0.01354 0.01581 0.01697 0.01% Castro::Castro() 5 0.01431 0.0162 0.01696 0.01% Castro::buildMetrics() 5 0.01424 0.01609 0.01681 0.01% FillPatchTwoLevels 31 0.013 0.01463 0.01635 0.01% MLCellLinOp::prepareForSolve() 11 0.01165 0.01182 0.01197 0.00% Gravity::actual_multilevel_solve() 10 0.007643 0.009263 0.01118 0.00% Castro::enforce_consistent_e() 1 0.009452 0.01067 0.01115 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0007803 0.00567 0.01103 0.00% Gravity::update_max_rhs() 10 0.007856 0.008918 0.01046 0.00% FabArrayBase::getCPC() 1440 0.01003 0.01023 0.01043 0.00% AMRErrorTag::operator() 6 0.008822 0.009909 0.01043 0.00% Gravity::actual_solve_with_mlmg() 11 0.00922 0.009555 0.00994 0.00% Castro::finalize_do_advance() 9 0.006041 0.007723 0.009076 0.00% Castro::do_new_sources() 9 0.0006133 0.002345 0.007437 0.00% Castro::apply_problem_tags() 6 0.006331 0.007111 0.007411 0.00% average_down_faces 48 0.005805 0.005891 0.00598 0.00% AmrLevel::derive() 6 0.003757 0.004544 0.005794 0.00% MLCellLinOp::setLevelBC() 19 0.003911 0.004752 0.005657 0.00% FabArray::FillBoundary() 1553 0.00425 0.004423 0.004747 0.00% MLCellLinOp::smooth() 688 0.003776 0.003919 0.004141 0.00% MLCGSolver::bicgstab 7 0 0.0009451 0.00378 0.00% FabArrayBase::getFB() 1553 0.003128 0.003265 0.003621 0.00% Gravity::test_composite_phi() 1 0.002115 0.002662 0.003437 0.00% amrex::Dot() 115 0 0.0008425 0.00337 0.00% FabArrayBase::FB::FB() 19 0.002523 0.002842 0.003099 0.00% FabArray::ParallelCopy() 830 0.002959 0.00303 0.003095 0.00% Castro::do_old_sources() 9 0.0005728 0.001044 0.002306 0.00% main() 1 0.00167 0.001976 0.002094 0.00% Castro::construct_new_gravity() 9 0.000443 0.000752 0.001557 0.00% MLLinOp::defineGrids() 11 0.001329 0.00143 0.001552 0.00% MLCellLinOp::apply() 318 0.0008849 0.001068 0.001485 0.00% Amr::regrid() 5 0.0009983 0.001182 0.001346 0.00% Castro::clean_state() 87 0.001286 0.001316 0.00134 0.00% Amr::coarseTimeStep() 5 0.0006358 0.001052 0.001258 0.00% TagBoxArray::collate() 6 0.0006422 0.0009073 0.001244 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002859 0.001144 0.00% AmrLevel::FillCoarsePatch() 5 0.0008961 0.0009365 0.0009732 0.00% OwnerMask() 6 0.0007396 0.0008522 0.0009224 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008954 0.0009119 0.0009215 0.00% MLCellLinOp::correctionResidual() 189 0.0007046 0.0007249 0.0007628 0.00% MLMG::mgVcycle() 65 0.0005972 0.0006221 0.0006391 0.00% Amr::grid_places() 6 0.0004304 0.0005391 0.0006195 0.00% FillPatchSingleLevel 92 0.0005335 0.0005611 0.0005807 0.00% Castro::do_advance_ctu() 5 0.0005123 0.0005387 0.0005632 0.00% Castro::finalize_advance() 9 0.0002867 0.0004595 0.0005538 0.00% TagBoxArray::mapPRD 6 0.000434 0.0004436 0.0004561 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003967 0.0004235 0.0004542 0.00% Castro::subcycle_advance_ctu() 5 6.84e-05 0.00018 0.0004189 0.00% MLMG:computeResOfCorrection() 155 0.0003762 0.0003882 0.0004097 0.00% MLCellLinOp::solutionResidual() 72 0.0003686 0.0003763 0.000383 0.00% MLMG::actualBottomSolve() 31 4.093e-05 0.0001261 0.0003761 0.00% MLMG::oneIter() 31 0.0003478 0.0003527 0.0003595 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002908 0.0003034 0.0003128 0.00% MLMG::mgVcycle_down::0 31 0.0002714 0.0002836 0.0002977 0.00% Castro::enforce_speed_limit() 87 0.0002616 0.0002694 0.0002775 0.00% Castro::computeNewDt() 8 0.0001353 0.0001714 0.0002703 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002318 0.0002423 0.0002599 0.00% MLMG::mgVcycle_down::1 31 0.000229 0.0002391 0.0002528 0.00% Amr::timeStep() 9 0.0002061 0.0002231 0.0002462 0.00% Amr::defBaseLevel() 1 0.0002353 0.0002377 0.0002397 0.00% AmrLevel::AmrLevel(dm) 5 0.0002013 0.0002104 0.0002319 0.00% Castro::post_regrid() 10 0.0002175 0.0002233 0.0002319 0.00% MLMG::mgVcycle_down::2 31 0.0001885 0.0001987 0.0002162 0.00% MLMG::mgVcycle_bottom 65 0.0002077 0.0002123 0.0002148 0.00% MLMG::mgVcycle_down::3 31 0.0001845 0.0001917 0.000206 0.00% FabArrayBase::TheFPinfo() 25 0.0001713 0.0001826 0.0001957 0.00% makeSFC 33 0.0001658 0.0001749 0.0001948 0.00% MLMG::mgVcycle_up::0 31 0.0001777 0.0001885 0.0001947 0.00% Amr::FinalizeInit() 1 3.744e-05 7.841e-05 0.0001824 0.00% MLMG::mgVcycle_down::4 31 0.0001626 0.0001728 0.0001787 0.00% Castro::check_for_nan() 18 0.0001415 0.000155 0.0001643 0.00% Castro::react_state() 18 0.0001323 0.0001394 0.0001502 0.00% MLMG::mgVcycle_up::1 31 0.0001353 0.0001394 0.0001479 0.00% Castro::expand_state() 13 0.0001133 0.0001229 0.0001428 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001334 0.0001368 0.0001409 0.00% MLMG::mgVcycle_up::2 31 0.0001251 0.0001289 0.0001346 0.00% FPinfo::FPinfo() 3 0.000112 0.0001201 0.0001304 0.00% MLMG::mgVcycle_up::3 31 0.0001199 0.0001236 0.000126 0.00% MLMG::mgVcycle_up::4 31 0.0001115 0.0001177 0.0001244 0.00% Castro::apply_source_to_state() 18 0.0001107 0.0001134 0.0001181 0.00% MLLinOp::makeAgglomeratedDMap 11 0.000104 0.0001105 0.0001169 0.00% FabArray::RecvLayoutMask() 8 9.631e-05 0.0001035 0.0001163 0.00% DistributionMapping::Distribute() 35 0.0001063 0.00011 0.0001159 0.00% MLPoisson::define() 11 0.0001052 0.0001094 0.0001147 0.00% Castro::errorEst() 6 0.0001017 0.0001098 0.0001143 0.00% Gravity::swapTimeLevels() 9 8.195e-05 9.07e-05 9.786e-05 0.00% Amr::writeSmallPlotFile() 1 9.06e-05 9.469e-05 9.723e-05 0.00% Castro::post_timestep() 9 8.518e-05 8.894e-05 9.512e-05 0.00% MLMG::computeResidual() 31 8.165e-05 8.666e-05 9.076e-05 0.00% MLMG::computeMLResidual() 15 7.711e-05 8.203e-05 8.717e-05 0.00% Castro::advance() 5 7.597e-05 8.352e-05 8.693e-05 0.00% ClusterList::intersect() 1 0 2.126e-05 8.504e-05 0.00% Gravity::set_mass_offset() 6 4.397e-05 5.735e-05 8.464e-05 0.00% knapsack() 3 6.609e-05 7.109e-05 7.768e-05 0.00% Castro::create_source_corrector() 9 5.635e-05 6.643e-05 7.673e-05 0.00% Castro::avgDown(state_indx) 20 6.799e-05 7.043e-05 7.36e-05 0.00% MLMG::miniCycle() 34 7.011e-05 7.126e-05 7.279e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.293e-05 6.482e-05 6.897e-05 0.00% MLMG::getGradSolution() 10 5.624e-05 6.182e-05 6.721e-05 0.00% MLLinOp::define() 11 5.604e-05 6.182e-05 6.442e-05 0.00% amrex::average_down_w_geom 20 5.091e-05 5.318e-05 5.655e-05 0.00% Castro::retry_advance_ctu() 5 2.274e-05 3.395e-05 5.583e-05 0.00% Castro::construct_new_hybrid_source() 9 5.111e-05 5.274e-05 5.394e-05 0.00% Castro::init(old) 3 4.563e-05 4.939e-05 5.352e-05 0.00% MLPoisson::prepareForSolve() 11 4.599e-05 4.829e-05 5.288e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.53e-05 4.818e-05 5.186e-05 0.00% Amr::initSubcycle() 1 4.929e-05 4.996e-05 5.069e-05 0.00% DistributionMapping::KnapSackDoIt() 3 3.023e-05 3.607e-05 4.56e-05 0.00% Castro::computeInitialDt() 2 1.879e-05 2.43e-05 3.774e-05 0.00% AmrMesh-cluster 1 0 9.366e-06 3.746e-05 0.00% Castro::construct_old_hybrid_source() 9 3.537e-05 3.584e-05 3.643e-05 0.00% MLMG::MLRhsNormInf() 10 3.233e-05 3.321e-05 3.431e-05 0.00% Castro::post_init() 1 2.435e-05 2.897e-05 3.387e-05 0.00% Castro::avgDown() 4 2.851e-05 3.083e-05 3.347e-05 0.00% BoxList::parallelComplementIn() 12 2.466e-05 2.794e-05 3.078e-05 0.00% Castro::construct_new_rotation_source() 9 2.415e-05 2.728e-05 2.983e-05 0.00% MLMG::compResidual() 1 2.604e-05 2.732e-05 2.922e-05 0.00% BoxList::complementIn 12 2.238e-05 2.409e-05 2.862e-05 0.00% Amr::InitializeInit() 1 7.941e-06 1.316e-05 2.629e-05 0.00% Castro::construct_old_rotation_source() 9 2.057e-05 2.401e-05 2.602e-05 0.00% Castro::derive() 6 2.049e-05 2.295e-05 2.602e-05 0.00% Amr::bldFineLevels() 1 2.131e-05 2.296e-05 2.556e-05 0.00% Castro::init() 1 1.55e-05 1.658e-05 1.795e-05 0.00% ClusterList::chop() 1 0 2.717e-06 1.087e-05 0.00% Amr::init() 1 7.227e-06 8.215e-06 1.038e-05 0.00% Amr::initialInit() 1 5.499e-06 7.346e-06 1.025e-05 0.00% knapsack()swap 1 3.647e-06 3.889e-06 4.202e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 242.6 242.6 242.6 100.00% Amr::coarseTimeStep() 5 230.5 230.5 230.5 94.99% Amr::timeStep() 9 229.1 229.1 229.1 94.45% Castro::advance() 5 201.3 201.3 201.3 82.96% Castro::subcycle_advance_ctu() 5 192.6 193 193.9 79.91% Castro::do_advance_ctu() 5 192.6 193 193.9 79.91% Castro::construct_ctu_hydro_source() 9 118.1 118.2 118.3 48.75% Castro::clean_state() 87 71.36 80.59 84.42 34.79% Castro::computeTemp() 88 45.34 51.13 53.4 22.01% Castro::reset_internal_energy(MultiFab) 88 30.65 34.66 36.21 14.92% Castro::evaluate_source_change() 193 17.84 20.01 20.78 8.56% Castro::enforce_min_density() 87 16.82 19.13 20.38 8.40% Castro::post_regrid() 10 15.88 17.53 18.15 7.48% Amr::regrid() 5 15.78 17.29 17.86 7.36% Castro::initialize_do_advance() 9 14.79 15.08 15.27 6.29% check_for_negative_density() 9 0.7927 4.62 14.77 6.09% amrex::Copy() 679 12.55 14.13 14.72 6.07% AmrLevel::FillPatch() 62 7.963 9.568 14.03 5.78% FabArray::ParallelCopy() 830 3.95 6.511 13.16 5.42% FillPatchIterator::Initialize 62 6.278 7.95 12.59 5.19% MultiFab::AddProduct() 4205 10.46 11.73 12.2 5.03% Castro::normalize_species() 87 9.99 11.26 11.74 4.84% Gravity::solve_phi_with_mlmg() 11 11.29 11.4 11.64 4.80% Gravity::multilevel_solve_for_new_phi() 10 10.99 11.1 11.33 4.67% Gravity::actual_multilevel_solve() 10 10.96 11.07 11.3 4.66% Castro::do_old_sources() 9 11.1 11.14 11.22 4.62% Castro::do_new_sources() 9 9.473 10.55 10.99 4.53% FabArray::ParallelCopy_finish() 830 1.652 4.149 10.89 4.49% Amr::init() 1 10.46 10.46 10.46 4.31% FillPatchSingleLevel 92 3.944 5.709 10.46 4.31% Castro::post_timestep() 9 10.43 10.43 10.43 4.30% FillPatchIterator::FillFromLevel0() 31 3.581 5.329 10.05 4.14% Amr::initialInit() 1 9.927 9.927 9.927 4.09% Castro::reset_internal_energy(Fab) 1476 8.42 9.27 9.562 3.94% Castro::reflux() 4 9.166 9.221 9.241 3.81% Castro::estTimeStep() 26 4.012 5.382 8.97 3.70% FabArray::setVal() 5070 7.397 8.335 8.713 3.59% Castro::initialize_advance() 9 7.405 8.317 8.648 3.56% MultiFab::Subtract() 175 6.996 7.839 8.16 3.36% Gravity::actual_solve_with_mlmg() 11 6.9 6.906 6.913 2.85% Castro::construct_new_gravity() 9 6.388 6.493 6.713 2.77% Castro::expand_state() 13 5.14 5.544 6.519 2.69% MLMG::solve() 10 6.39 6.401 6.422 2.65% Amr::FinalizeInit() 1 5.741 5.92 6.322 2.61% MLMG::oneIter() 31 5.179 5.197 5.205 2.15% Gravity::fill_multipole_BCs() 11 4.367 4.473 4.713 1.94% FluxRegister::Reflux() 24 4.521 4.541 4.557 1.88% MLMG::mgVcycle() 65 4.338 4.371 4.385 1.81% Castro::problem_post_timestep() 10 2.984 3.431 4.29 1.77% Amr::InitializeInit() 1 3.605 4.007 4.186 1.73% Amr::defBaseLevel() 1 3.605 4.007 4.186 1.73% Castro::construct_old_gravity() 9 0.6964 1.657 4.1 1.69% Gravity::get_old_grav_vector() 9 0.6751 1.649 4.1 1.69% FabArray::sum() 4205 3.472 3.894 4.043 1.67% Castro::computeNewDt() 8 2.156 2.633 3.944 1.63% Castro::finalize_do_advance() 9 1.342 2.06 3.933 1.62% MLCellLinOp::smooth() 688 3.3 3.316 3.339 1.38% Castro::post_init() 1 2.518 2.656 2.997 1.24% Castro::initData() 1 2.235 2.6 2.831 1.17% Castro::FluxRegCrseInit 9 1.826 2.081 2.732 1.13% FabArray::ParallelCopy_nowait() 830 2.156 2.359 2.722 1.12% Castro::wd_update() 6 1.338 1.786 2.646 1.09% MLPoisson::Fsmooth() 1376 2.139 2.405 2.527 1.04% Castro::construct_new_source() 63 2.045 2.14 2.262 0.93% MLMG::mgVcycle_down::0 31 1.92 1.989 2.021 0.83% MLCellLinOp::applyBC() 1742 1.102 1.276 1.691 0.70% FillPatchIterator::FillFromTwoLevels() 31 1.531 1.549 1.574 0.65% FillPatchTwoLevels 31 1.53 1.549 1.574 0.65% Amr::bldFineLevels() 1 1.254 1.453 1.554 0.64% FillPatchInterp(MF) 30 1.259 1.37 1.476 0.61% Castro::Castro() 5 1.308 1.37 1.428 0.59% MLMG::mgVcycle_up::0 31 1.385 1.413 1.424 0.59% MLCellLinOp::apply() 318 1.302 1.331 1.358 0.56% FabArray::FillBoundary() 1553 0.6315 0.847 1.331 0.55% Castro::construct_old_gravity_source() 18 1.016 1.101 1.178 0.49% amrex::Add() 136 0.9842 1.108 1.16 0.48% Castro::initMFs() 5 1.075 1.121 1.153 0.48% Amr::InitAmr() 1 1.153 1.153 1.153 0.48% Castro::computeInitialDt() 2 0.5189 0.698 1.099 0.45% Castro::construct_new_gravity_source() 9 0.8624 0.9928 1.076 0.44% Castro::construct_old_source() 63 0.9185 1.022 1.073 0.44% MLPoisson::Fapply() 318 0.9181 1.03 1.073 0.44% Amr::writePlotFile() 2 1.065 1.066 1.066 0.44% MLCellLinOp::solutionResidual() 72 0.9766 1.029 1.049 0.43% Castro::volInBoundary() 42 0.6561 0.7777 0.8896 0.37% FillBoundary_finish() 1553 0.2785 0.4584 0.8886 0.37% MLCellLinOp::correctionResidual() 189 0.8117 0.8226 0.833 0.34% FabArray::Saxpy() 182 0.7102 0.7973 0.8298 0.34% Gravity::get_new_grav_vector() 10 0.783 0.7933 0.8014 0.33% MLMG:computeResOfCorrection() 155 0.6599 0.6924 0.7107 0.29% VisMF::Write(FabArray) 3 0.5848 0.6568 0.6907 0.28% StateDataPhysBCFunct::() 95 0.5908 0.6106 0.6235 0.26% StateData::FillBoundary(geom) 50 0.5886 0.6008 0.6104 0.25% MLMG::computeResWithCrseSolFineCor() 17 0.5849 0.5862 0.5886 0.24% Gravity::update_max_rhs() 10 0.07925 0.2189 0.5593 0.23% MultiFab::max() 18 0.0182 0.1609 0.507 0.21% MLMG::computeMLResidual() 15 0.4169 0.4424 0.5032 0.21% Castro::init() 1 0.4914 0.4937 0.4956 0.20% AmrLevel::FillCoarsePatch() 5 0.4913 0.4937 0.4956 0.20% Castro::check_for_nan() 18 0.3985 0.4454 0.4618 0.19% MultiFab::contains_nan() 18 0.3984 0.4453 0.4616 0.19% FillBoundary_nowait() 1553 0.3486 0.3841 0.4379 0.18% amrex::average_down 233 0.2824 0.3643 0.4195 0.17% Gravity::test_composite_phi() 1 0.4121 0.4134 0.415 0.17% MLMG::mgVcycle_down::1 31 0.3151 0.3421 0.4039 0.17% FabArray::Xpay() 283 0.3464 0.3869 0.4022 0.17% Castro::init(old) 3 0.3655 0.3835 0.3902 0.16% MLCellLinOp::reflux() 30 0.1806 0.2333 0.3687 0.15% MLMG::computeResidual() 31 0.3476 0.355 0.3593 0.15% Castro::apply_source_to_state() 18 0.2855 0.3195 0.3321 0.14% MLPoisson::define() 11 0.2901 0.2998 0.3106 0.13% Castro::react_state() 18 0.2219 0.2525 0.2709 0.11% FabArray::setDomainBndry() 92 0.1984 0.222 0.2374 0.10% MLMG::prepareForSolve() 10 0.2182 0.2259 0.2294 0.09% MLMG::mgVcycle_up::1 31 0.216 0.221 0.2245 0.09% MLCellLinOp::defineAuxData() 11 0.1835 0.2021 0.2121 0.09% Castro::fill_hybrid_hydro_source() 27 0.1707 0.192 0.201 0.08% Amr::grid_places() 6 0.1942 0.1943 0.1944 0.08% AmrMesh::MakeNewGrids() 6 0.1936 0.1938 0.1939 0.08% MLMG::mgVcycle_bottom 65 0.1595 0.1668 0.1865 0.08% MLMG::addInterpCorrection() 155 0.132 0.1674 0.1846 0.08% MLMG::getGradSolution() 10 0.1683 0.1712 0.1725 0.07% MLCellLinOp::compGrad() 18 0.1682 0.1712 0.1725 0.07% MLMG::miniCycle() 34 0.1591 0.1599 0.161 0.07% MLPoisson::FFlux() 150 0.004709 0.04274 0.1566 0.06% Castro::avgDown() 4 0.1243 0.1284 0.1378 0.06% Castro::avgDown(state_indx) 20 0.1243 0.1284 0.1377 0.06% amrex::average_down_w_geom 20 0.1242 0.1283 0.1377 0.06% AmrLevel::AmrLevel(dm) 5 0.1039 0.1194 0.1343 0.06% Castro::construct_new_hybrid_source() 9 0.1131 0.1278 0.1342 0.06% StateData::define() 25 0.1037 0.1192 0.1341 0.06% FabArray::mult() 56 0.1116 0.1259 0.1312 0.05% Castro::errorEst() 6 0.1035 0.117 0.1243 0.05% MLMG::ResNormInf() 54 0.07328 0.08428 0.1053 0.04% Castro::swap_state_time_levels() 9 0.09163 0.09541 0.09956 0.04% MLCellLinOp::updateSolBC() 25 0.0787 0.08422 0.09779 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08712 0.08806 0.08981 0.04% MLCellLinOp::defineBC() 11 0.08491 0.08724 0.08861 0.04% MLMG::mgVcycle_down::2 31 0.0689 0.07347 0.08197 0.03% MLMG::mgVcycle_up::3 31 0.02562 0.06444 0.07823 0.03% MLMG::MLResNormInf() 15 0.04515 0.06819 0.07755 0.03% Castro::construct_old_hybrid_source() 9 0.05769 0.06427 0.06704 0.03% FabArray::norminf(mask) 21 0.05551 0.06294 0.0662 0.03% BndryData::define() 27 0.05755 0.05963 0.06084 0.03% MLMG::interpCorrection_1 17 0.02741 0.03599 0.06038 0.02% Castro::apply_tagging_restrictions() 6 0.04788 0.05386 0.05655 0.02% MLMG::mgVcycle_up::2 31 0.05138 0.05243 0.0532 0.02% MLCellLinOp::updateCorBC() 17 0.04993 0.05093 0.05262 0.02% Castro::derive() 6 0.04039 0.046 0.05012 0.02% AmrLevel::derive() 6 0.04037 0.04598 0.0501 0.02% Castro::buildMetrics() 5 0.03398 0.04003 0.04582 0.02% FabArray::norminf() 147 0.03663 0.03888 0.04134 0.02% Gravity::swapTimeLevels() 9 0.0323 0.0365 0.03957 0.02% MLMG::mgVcycle_down::3 31 0.03021 0.03151 0.03484 0.01% TagBoxArray::mapPRD 6 0.006609 0.01457 0.03131 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01971 0.02356 0.02955 0.01% MLMG::MLRhsNormInf() 10 0.02531 0.02844 0.02951 0.01% FabArrayBase::getCPC() 1440 0.02328 0.025 0.02808 0.01% MLMG::actualBottomSolve() 31 4.093e-05 0.00691 0.02751 0.01% Castro::FluxRegFineAdd() 9 0.0259 0.02645 0.02679 0.01% MLCGSolver::bicgstab 7 0 0.006695 0.02678 0.01% MultiFab::Multiply() 8 0.02165 0.0245 0.02563 0.01% MLLinOp::define() 11 0.002449 0.01033 0.02161 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01951 0.02038 0.02158 0.01% MLLinOp::defineGrids() 11 0.002393 0.01027 0.02155 0.01% MLMG::compResidual() 1 0.01858 0.02026 0.02096 0.01% MLLinOp::makeSubCommunicator() 11 0.0006502 0.008456 0.01958 0.01% Castro::build_fine_mask() 44 0.01672 0.01824 0.01902 0.01% MLMG::mgVcycle_down::4 31 0.0053 0.008965 0.01863 0.01% FabArrayBase::CPC::define() 416 0.01285 0.01477 0.01805 0.01% Castro::linear_to_hybrid_momentum() 1 0.01354 0.01581 0.01697 0.01% MLMG::mgVcycle_up::4 31 0.004413 0.007002 0.01394 0.01% MLPoisson::prepareForSolve() 11 0.0117 0.01187 0.01201 0.00% MLCellLinOp::prepareForSolve() 11 0.01165 0.01182 0.01197 0.00% FabArrayBase::TheFPinfo() 25 0.001121 0.006009 0.01137 0.00% FPinfo::FPinfo() 3 0.0009254 0.005827 0.01118 0.00% Castro::enforce_consistent_e() 1 0.009452 0.01067 0.01115 0.00% DistributionMapping::KnapSackDoIt() 3 0.0008083 0.005707 0.01105 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0007803 0.00567 0.01103 0.00% average_down_faces 48 0.009066 0.009833 0.01061 0.00% AMRErrorTag::operator() 6 0.008822 0.009909 0.01043 0.00% Castro::apply_problem_tags() 6 0.006331 0.007111 0.007411 0.00% FabArrayBase::getFB() 1553 0.005696 0.006107 0.006664 0.00% MLCellLinOp::setLevelBC() 19 0.003911 0.004752 0.005657 0.00% amrex::Dot() 115 0 0.0008425 0.00337 0.00% FabArrayBase::FB::FB() 19 0.002523 0.002842 0.003099 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007381 0.002952 0.00% TagBoxArray::collate() 6 0.0006422 0.0009073 0.001244 0.00% OwnerMask() 6 0.0007396 0.0008522 0.0009224 0.00% Castro::finalize_advance() 9 0.0002867 0.0004595 0.0005538 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003691 0.0003845 0.0004163 0.00% makeSFC 33 0.0002629 0.000274 0.0002994 0.00% Castro::enforce_speed_limit() 87 0.0002616 0.0002694 0.0002775 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001099 0.000133 0.0001488 0.00% AmrMesh-cluster 1 0 3.334e-05 0.0001334 0.00% FabArray::RecvLayoutMask() 8 9.631e-05 0.0001035 0.0001163 0.00% DistributionMapping::Distribute() 35 0.0001063 0.00011 0.0001159 0.00% Amr::writeSmallPlotFile() 1 9.06e-05 9.469e-05 9.723e-05 0.00% ClusterList::intersect() 1 0 2.126e-05 8.504e-05 0.00% Gravity::set_mass_offset() 6 4.397e-05 5.735e-05 8.464e-05 0.00% knapsack() 3 6.974e-05 7.498e-05 8.159e-05 0.00% Castro::create_source_corrector() 9 5.635e-05 6.643e-05 7.673e-05 0.00% BoxList::parallelComplementIn() 12 4.725e-05 5.204e-05 5.94e-05 0.00% Castro::retry_advance_ctu() 5 2.274e-05 3.395e-05 5.583e-05 0.00% Amr::initSubcycle() 1 4.929e-05 4.996e-05 5.069e-05 0.00% Castro::construct_new_rotation_source() 9 2.415e-05 2.728e-05 2.983e-05 0.00% BoxList::complementIn 12 2.238e-05 2.409e-05 2.862e-05 0.00% Castro::construct_old_rotation_source() 9 2.057e-05 2.401e-05 2.602e-05 0.00% ClusterList::chop() 1 0 2.717e-06 1.087e-05 0.00% knapsack()swap 1 3.647e-06 3.889e-06 4.202e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 1 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-42-g7437aa17d36f) finalized