Initializing AMReX (v2024-48-g47347f785f5c)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-48-g47347f785f5c) initialized Starting run at 07:43:08 UTC on 2024-04-01. Successfully read inputs file ... Castro git describe: 24.03-38-gcd731a3af AMReX git describe: v2024-48-g47347f785 Microphysics git describe: 24.03-23-g3112c13a reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.030830301 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.030469187 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.364916464 Gravity::multilevel_solve_for_new_phi() time = 1.771870538 Gravity::fill_multipole_BCs() time = 0.370984037 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.612382624 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.874135299 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.302882322 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 20.42963859 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.645631201 Gravity::multilevel_solve_for_new_phi() time = 1.68622955 Castro::construct_new_gravity() time = 1.801309682 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207922e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.268120565 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.025175372 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 37.24158498 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032529399 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402867352 Gravity::multilevel_solve_for_new_phi() time = 1.92293252 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.845658478 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.005312263 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.25183684 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.028969324 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.575065382e+22 ymom added: 1.534769107e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.192333546 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.72804202 on level 1 Castro::reflux() at level 0 : time = 2.236631133 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.420819347 Gravity::multilevel_solve_for_new_phi() time = 1.367802514 Castro::construct_new_gravity() time = 1.481178312 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.947112618 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029291053 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764946858e+18 ymom added: -1.361438167e+19 zmom added: -8.219114345e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.141338905 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02307924001 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.15055295 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031954979 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.403066261 Gravity::multilevel_solve_for_new_phi() time = 0.532795879 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.814032797 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.035630975 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.7611466 on level 0 Castro::construct_old_gravity() time = 0.038289467 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.443896444e+22 ymom added: 3.836922767e+21 zmom added: 7.103293505e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.195078317 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.764475825 on level 1 Castro::reflux() at level 0 : time = 2.21830849 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.418097502 Gravity::multilevel_solve_for_new_phi() time = 0.971241624 Castro::construct_new_gravity() time = 1.079659731 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.946726774 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.02972724 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.819118889e+17 ymom added: 2.351779725e+19 zmom added: 9.539393112e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.14026426 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02304834422 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.2830661 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032218873 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402999157 Gravity::multilevel_solve_for_new_phi() time = 0.532197929 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.713892042 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.054742037 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.08250666 on level 0 Castro::construct_old_gravity() time = 0.032008402 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.200499268e+22 ymom added: 1.239621202e+22 zmom added: 8.024189557e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.1936225 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.761381085 on level 1 Castro::reflux() at level 0 : time = 2.250939861 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.417606465 Gravity::multilevel_solve_for_new_phi() time = 0.973495938 Castro::construct_new_gravity() time = 1.08477482 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.967744185 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029466544 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598303655e+18 ymom added: -2.568853227e+19 zmom added: -1.336958338e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.137245641 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02348086391 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 46.53048137 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.032331536 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.405031437 Gravity::multilevel_solve_for_new_phi() time = 0.534638945 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.866259578 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.000091449 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.10703177 on level 0 Castro::construct_old_gravity() time = 0.034542526 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013225564e+23 ymom added: 1.829917012e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.193453422 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.81851553 on level 1 Castro::reflux() at level 0 : time = 2.252935919 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.419789286 Gravity::multilevel_solve_for_new_phi() time = 0.976159339 Castro::construct_new_gravity() time = 1.089607951 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.960989864 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029977922 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.726313915e+18 ymom added: -2.54093091e+19 zmom added: -1.25233914e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.141662185 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02340731093 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 46.70437137 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.531412941 seconds Ending run at 07:47:07 UTC on 2024-04-01. Run time = 239.0594907 Run time without initialization = 227.4432422 Average number of zones advanced per microsecond: 0.022 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 239.1 ... 239.1 ... 239.1 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 92.4551 102.2224 106.3353 44.48% Castro::computeTemp() 88 14.65 16.5 17.33 7.25% amrex::Copy() 679 12.93 14.43 15 6.28% check_for_negative_density() 9 0.4131 4.534 14.36 6.01% MultiFab::AddProduct() 4205 10.45 11.7 12.16 5.08% Castro::normalize_species() 87 9.928 11.14 11.59 4.85% FabArray::ParallelCopy_finish() 830 1.457 4.258 10.76 4.50% Castro::reset_internal_energy(Fab) 1476 8.262 9.212 9.564 4.00% Castro::estTimeStep() 26 3.947 5.288 8.683 3.63% FabArray::setVal() 5070 7.114 7.899 8.208 3.43% MultiFab::Subtract() 175 6.685 7.525 7.851 3.28% Gravity::fill_multipole_BCs() 11 4.244 4.364 4.599 1.92% Castro::reset_internal_energy(MultiFab) 88 3.271 3.81 4.12 1.72% FabArray::sum() 4205 3.422 3.833 3.981 1.67% FluxRegister::Reflux() 24 2.958 3.284 3.41 1.43% FabArray::ParallelCopy_nowait() 830 2.101 2.266 2.661 1.11% MLPoisson::Fsmooth() 1376 2.112 2.382 2.524 1.06% Castro::enforce_min_density() 87 1.555 1.804 2.045 0.86% Castro::FluxRegCrseInit 9 1.5 1.69 1.773 0.74% Castro::wd_update() 6 0.4437 0.8899 1.745 0.73% Castro::reflux() 4 1.472 1.659 1.725 0.72% FillPatchInterp(MF) 30 1.212 1.331 1.441 0.60% Amr::InitAmr() 1 1.249 1.249 1.249 0.52% MLPoisson::Fapply() 318 0.9095 1.037 1.144 0.48% amrex::Add() 136 0.9865 1.098 1.143 0.48% Castro::initMFs() 5 1.086 1.114 1.131 0.47% Castro::construct_old_gravity_source() 18 0.8918 0.9882 1.04 0.44% FillBoundary_finish() 1553 0.3141 0.4867 0.9238 0.39% Castro::construct_new_gravity_source() 9 0.7397 0.8329 0.8741 0.37% FillPatchIterator::Initialize 62 0.7477 0.8259 0.8537 0.36% Castro::problem_post_timestep() 10 0.7441 0.7729 0.8281 0.35% FabArray::Saxpy() 182 0.6766 0.7611 0.7952 0.33% Castro::volInBoundary() 42 0.6284 0.6954 0.7712 0.32% VisMF::Write(FabArray) 3 0.5646 0.6545 0.7377 0.31% StateData::FillBoundary(geom) 50 0.5865 0.5992 0.6147 0.26% Castro::construct_new_source() 63 0.4693 0.5216 0.5458 0.23% MultiFab::max() 18 0.02142 0.1925 0.5383 0.23% MultiFab::contains_nan() 18 0.3926 0.4412 0.4599 0.19% MLCellLinOp::applyBC() 1742 0.3551 0.4145 0.4424 0.19% FillBoundary_nowait() 1553 0.3144 0.3673 0.4107 0.17% FabArray::Xpay() 283 0.3489 0.3888 0.4053 0.17% Amr::writePlotFile() 2 0.1083 0.2015 0.318 0.13% Castro::initialize_do_advance() 9 0.2299 0.2612 0.2735 0.11% FabArray::setDomainBndry() 92 0.1922 0.2138 0.2309 0.10% amrex::average_down 233 0.1935 0.2031 0.2081 0.09% Castro::construct_old_source() 63 0.1775 0.1968 0.2071 0.09% Castro::fill_hybrid_hydro_source() 27 0.1701 0.1893 0.1959 0.08% MLCellLinOp::defineAuxData() 11 0.1529 0.1663 0.1731 0.07% MLPoisson::FFlux() 150 0.004606 0.04282 0.1573 0.07% Castro::initData() 1 0.1281 0.1382 0.1489 0.06% MLCellLinOp::compGrad() 18 0.1245 0.1388 0.1457 0.06% FabArray::mult() 56 0.115 0.1291 0.1352 0.06% MLMG::addInterpCorrection() 155 0.1077 0.1202 0.126 0.05% StateData::define() 25 0.1074 0.1186 0.1237 0.05% Castro::evaluate_source_change() 193 0.1039 0.1107 0.1139 0.05% Gravity::get_new_grav_vector() 10 0.08494 0.09343 0.09654 0.04% Castro::swap_state_time_levels() 9 0.08637 0.09177 0.09574 0.04% Gravity::get_old_grav_vector() 9 0.07778 0.08558 0.09018 0.04% Castro::initialize_advance() 9 0.06874 0.07693 0.08206 0.03% MLMG::prepareForSolve() 10 0.06006 0.06547 0.06849 0.03% MLMG::solve() 10 0.004314 0.04485 0.06816 0.03% MLCellLinOp::updateSolBC() 25 0.06703 0.06735 0.06777 0.03% FabArray::norminf(mask) 21 0.05299 0.05963 0.06225 0.03% AmrLevel::FillPatch() 62 0.05224 0.05366 0.05644 0.02% Castro::apply_tagging_restrictions() 6 0.04787 0.05333 0.05567 0.02% MLMG::ResNormInf() 54 0.0005928 0.02043 0.05351 0.02% AmrMesh::MakeNewGrids() 6 0.04626 0.0479 0.05225 0.02% Castro::do_old_sources() 9 0.0005368 0.01688 0.0461 0.02% MLCellLinOp::updateCorBC() 17 0.04574 0.04586 0.04601 0.02% FabArray::norminf() 147 0.04052 0.0426 0.04489 0.02% BndryData::define() 27 0.04204 0.0424 0.04283 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.02912 0.03246 0.03666 0.02% MLMG::MLResNormInf() 15 0.000113 0.02464 0.03571 0.01% MLCellLinOp::reflux() 30 0.004135 0.01127 0.03244 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01876 0.02265 0.02884 0.01% MLLinOp::makeSubCommunicator() 11 0.001348 0.01141 0.02698 0.01% Castro::FluxRegFineAdd() 9 0.0255 0.02569 0.02604 0.01% MLCellLinOp::defineBC() 11 0.02573 0.02589 0.02599 0.01% MultiFab::Multiply() 8 0.02121 0.02381 0.02482 0.01% StateDataPhysBCFunct::() 95 0.001803 0.006879 0.01949 0.01% Castro::build_fine_mask() 44 0.0165 0.01749 0.01881 0.01% MLMG::interpCorrection_1 17 0.01845 0.01854 0.01862 0.01% FabArrayBase::CPC::define() 416 0.01285 0.01472 0.01789 0.01% Castro::construct_old_gravity() 9 0.0003862 0.008628 0.01724 0.01% Castro::buildMetrics() 5 0.01476 0.01639 0.01707 0.01% Castro::Castro() 5 0.01499 0.01625 0.01681 0.01% Castro::linear_to_hybrid_momentum() 1 0.01405 0.01576 0.01643 0.01% FillPatchTwoLevels 31 0.01253 0.01285 0.01315 0.01% MLCellLinOp::prepareForSolve() 11 0.01133 0.01153 0.01186 0.00% Gravity::actual_multilevel_solve() 10 0.007639 0.009346 0.01123 0.00% Castro::enforce_consistent_e() 1 0.009373 0.01058 0.01105 0.00% AMRErrorTag::operator() 6 0.008809 0.009854 0.0103 0.00% FabArrayBase::getCPC() 1440 0.009584 0.00984 0.01002 0.00% Gravity::actual_solve_with_mlmg() 11 0.009107 0.009398 0.009907 0.00% Castro::finalize_do_advance() 9 0.007745 0.008831 0.009491 0.00% Gravity::update_max_rhs() 10 0.007472 0.008011 0.008306 0.00% Castro::apply_problem_tags() 6 0.006726 0.007555 0.007844 0.00% MLCellLinOp::setLevelBC() 19 0.004379 0.005441 0.006454 0.00% average_down_faces 48 0.005605 0.005665 0.005706 0.00% FabArray::FillBoundary() 1553 0.004232 0.00445 0.004853 0.00% AmrLevel::derive() 6 0.00393 0.004149 0.004266 0.00% MLCellLinOp::smooth() 688 0.003782 0.003883 0.004037 0.00% MLCGSolver::bicgstab 7 0 0.000948 0.003792 0.00% FabArrayBase::getFB() 1553 0.003086 0.003242 0.003646 0.00% FabArrayBase::FB::FB() 19 0.002491 0.002965 0.003404 0.00% amrex::Dot() 115 0 0.0008471 0.003388 0.00% Castro::do_new_sources() 9 0.0005391 0.001532 0.003355 0.00% Gravity::test_composite_phi() 1 0.002497 0.002805 0.003196 0.00% FabArray::ParallelCopy() 830 0.002868 0.002991 0.003082 0.00% main() 1 0.001512 0.001624 0.001725 0.00% DistributionMapping::LeastUsedCPUs() 5 0.000609 0.001215 0.00172 0.00% MLLinOp::defineGrids() 11 0.001291 0.001392 0.001558 0.00% MLCellLinOp::apply() 318 0.0008756 0.001052 0.001529 0.00% Amr::regrid() 5 0.0009717 0.00112 0.001421 0.00% Castro::construct_new_gravity() 9 0.0003752 0.0007021 0.001306 0.00% Castro::clean_state() 87 0.001214 0.001251 0.001296 0.00% TagBoxArray::collate() 6 0.0006685 0.001033 0.001276 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002835 0.001134 0.00% Amr::coarseTimeStep() 5 0.0004769 0.0009477 0.00111 0.00% AmrLevel::FillCoarsePatch() 5 0.0009059 0.0009414 0.000983 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009101 0.0009127 0.000916 0.00% OwnerMask() 6 0.0007265 0.0008313 0.0008939 0.00% MLCellLinOp::correctionResidual() 189 0.0006759 0.0006999 0.0007331 0.00% MLMG::mgVcycle() 65 0.0005688 0.0005873 0.0006062 0.00% Amr::grid_places() 6 0.0003805 0.000472 0.000577 0.00% Castro::do_advance_ctu() 5 0.0004878 0.0005092 0.0005575 0.00% FillPatchSingleLevel 92 0.0005178 0.0005282 0.0005494 0.00% TagBoxArray::mapPRD 6 0.000438 0.0004472 0.0004726 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003505 0.0004056 0.0004602 0.00% Castro::subcycle_advance_ctu() 5 7.384e-05 0.0001939 0.0004434 0.00% MLMG:computeResOfCorrection() 155 0.0003771 0.0003878 0.0004107 0.00% Castro::finalize_advance() 9 0.0002856 0.0003385 0.0003684 0.00% MLMG::actualBottomSolve() 31 3.816e-05 0.0001216 0.0003649 0.00% MLMG::oneIter() 31 0.0003328 0.0003491 0.0003628 0.00% MLCellLinOp::solutionResidual() 72 0.0003339 0.0003526 0.0003613 0.00% Castro::computeNewDt() 8 0.0001143 0.0001668 0.0003138 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002968 0.000305 0.0003125 0.00% MLMG::mgVcycle_down::0 31 0.0002719 0.0002824 0.0002953 0.00% Castro::enforce_speed_limit() 87 0.0002535 0.0002626 0.0002737 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002282 0.000242 0.0002593 0.00% Amr::defBaseLevel() 1 0.0002351 0.0002449 0.0002502 0.00% MLMG::mgVcycle_down::1 31 0.0002251 0.0002347 0.0002422 0.00% Amr::timeStep() 9 0.0001877 0.0002041 0.0002416 0.00% MLMG::mgVcycle_bottom 65 0.0002119 0.0002182 0.0002237 0.00% Castro::post_regrid() 10 0.0001801 0.0002011 0.0002159 0.00% MLMG::mgVcycle_up::0 31 0.0001808 0.0001869 0.0001962 0.00% MLMG::mgVcycle_down::3 31 0.0001896 0.0001918 0.0001949 0.00% MLMG::mgVcycle_down::2 31 0.000187 0.0001903 0.0001943 0.00% AmrLevel::AmrLevel(dm) 5 0.0001836 0.000188 0.000194 0.00% makeSFC 33 0.0001617 0.0001721 0.00019 0.00% FabArrayBase::TheFPinfo() 25 0.0001669 0.0001814 0.0001886 0.00% MLMG::mgVcycle_down::4 31 0.0001618 0.0001669 0.0001791 0.00% Castro::react_state() 18 0.0001182 0.0001362 0.0001577 0.00% MLMG::mgVcycle_up::1 31 0.0001364 0.0001379 0.0001393 0.00% MLMG::mgVcycle_up::2 31 0.0001293 0.0001321 0.0001388 0.00% Castro::check_for_nan() 18 0.0001277 0.0001331 0.0001376 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001278 0.0001318 0.0001354 0.00% Castro::expand_state() 13 0.0001098 0.0001184 0.0001335 0.00% MLMG::mgVcycle_up::4 31 0.0001145 0.0001193 0.0001298 0.00% MLMG::mgVcycle_up::3 31 0.0001206 0.0001258 0.0001291 0.00% DistributionMapping::Distribute() 35 0.0001066 0.0001146 0.0001229 0.00% FPinfo::FPinfo() 3 0.0001167 0.0001197 0.0001228 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001066 0.0001114 0.0001185 0.00% Amr::FinalizeInit() 1 3.59e-05 6.318e-05 0.000117 0.00% Castro::apply_source_to_state() 18 9.804e-05 0.0001049 0.000116 0.00% Castro::advance() 5 9.956e-05 0.0001059 0.0001143 0.00% FabArray::RecvLayoutMask() 8 8.91e-05 9.638e-05 0.0001113 0.00% MLPoisson::define() 11 9.63e-05 0.0001037 0.0001111 0.00% Castro::errorEst() 6 0.0001006 0.0001044 0.0001099 0.00% Gravity::swapTimeLevels() 9 8.283e-05 9.386e-05 0.0001087 0.00% Amr::writeSmallPlotFile() 1 8.833e-05 9.512e-05 0.0001049 0.00% Castro::post_timestep() 9 7.764e-05 8.247e-05 8.771e-05 0.00% MLMG::computeResidual() 31 8.271e-05 8.444e-05 8.628e-05 0.00% ClusterList::intersect() 1 0 2.03e-05 8.118e-05 0.00% MLMG::computeMLResidual() 15 7.535e-05 7.778e-05 8.034e-05 0.00% Castro::avgDown(state_indx) 20 6.503e-05 6.962e-05 7.532e-05 0.00% MLLinOp::define() 11 5.825e-05 6.249e-05 7.373e-05 0.00% MLMG::miniCycle() 34 6.589e-05 6.938e-05 7.336e-05 0.00% Castro::create_source_corrector() 9 4.971e-05 6.042e-05 7.174e-05 0.00% knapsack() 3 6.429e-05 6.679e-05 6.945e-05 0.00% MLMG::getGradSolution() 10 6.107e-05 6.297e-05 6.552e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.162e-05 6.291e-05 6.359e-05 0.00% amrex::average_down_w_geom 20 5.102e-05 5.185e-05 5.394e-05 0.00% Gravity::set_mass_offset() 6 4.212e-05 4.847e-05 5.332e-05 0.00% Amr::initSubcycle() 1 4.84e-05 5.113e-05 5.223e-05 0.00% MLPoisson::prepareForSolve() 11 4.612e-05 4.782e-05 5.055e-05 0.00% Castro::construct_new_hybrid_source() 9 4.575e-05 4.794e-05 4.911e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.43e-05 4.631e-05 4.859e-05 0.00% BoxList::parallelComplementIn() 12 2.614e-05 3.175e-05 4.69e-05 0.00% Castro::init(old) 3 4.319e-05 4.436e-05 4.552e-05 0.00% Castro::construct_old_hybrid_source() 9 3.29e-05 3.543e-05 3.764e-05 0.00% AmrMesh-cluster 1 0 9.177e-06 3.671e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.979e-05 3.213e-05 3.444e-05 0.00% MLMG::MLRhsNormInf() 10 3.112e-05 3.213e-05 3.325e-05 0.00% Castro::avgDown() 4 3.079e-05 3.129e-05 3.207e-05 0.00% MLMG::compResidual() 1 2.778e-05 2.856e-05 3.008e-05 0.00% Amr::initialInit() 1 5.841e-06 1.224e-05 2.948e-05 0.00% Castro::retry_advance_ctu() 5 2.365e-05 2.53e-05 2.925e-05 0.00% Castro::post_init() 1 2.054e-05 2.605e-05 2.866e-05 0.00% Castro::construct_new_rotation_source() 9 2.039e-05 2.355e-05 2.843e-05 0.00% BoxList::complementIn 12 2.285e-05 2.361e-05 2.468e-05 0.00% Castro::computeInitialDt() 2 1.568e-05 1.898e-05 2.43e-05 0.00% Castro::construct_old_rotation_source() 9 2.042e-05 2.249e-05 2.382e-05 0.00% Amr::bldFineLevels() 1 2.046e-05 2.185e-05 2.278e-05 0.00% Castro::derive() 6 1.95e-05 1.992e-05 2.055e-05 0.00% Castro::init() 1 1.607e-05 1.633e-05 1.669e-05 0.00% Amr::InitializeInit() 1 8.366e-06 9.498e-06 1.265e-05 0.00% Amr::init() 1 6.695e-06 7.85e-06 1.084e-05 0.00% ClusterList::chop() 1 0 2.548e-06 1.019e-05 0.00% knapsack()swap 1 3.862e-06 4.175e-06 4.445e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 239.1 239.1 239.1 100.00% Amr::coarseTimeStep() 5 226.9 226.9 226.9 94.92% Amr::timeStep() 9 225.6 225.6 225.6 94.36% Castro::advance() 5 198.6 198.6 198.6 83.08% Castro::subcycle_advance_ctu() 5 190.1 190.4 191.3 80.02% Castro::do_advance_ctu() 5 190.1 190.4 191.3 80.02% Castro::construct_ctu_hydro_source() 9 116.4 116.5 116.6 48.77% Castro::clean_state() 87 70.73 79.5 83.09 34.76% Castro::computeTemp() 88 44.99 50.58 52.91 22.13% Castro::reset_internal_energy(MultiFab) 88 30.34 34.08 35.58 14.88% Castro::evaluate_source_change() 193 17.59 19.68 20.44 8.55% Castro::enforce_min_density() 87 16.6 18.67 19.56 8.18% Castro::post_regrid() 10 15.71 17.22 17.82 7.45% Amr::regrid() 5 15.6 16.97 17.49 7.31% Castro::initialize_do_advance() 9 14.56 14.97 15.26 6.38% amrex::Copy() 679 12.93 14.43 15 6.28% check_for_negative_density() 9 0.4131 4.534 14.36 6.01% AmrLevel::FillPatch() 62 8.108 9.682 13.95 5.84% FabArray::ParallelCopy() 830 4.184 6.587 12.96 5.42% FillPatchIterator::Initialize 62 6.398 8.03 12.46 5.21% MultiFab::AddProduct() 4205 10.45 11.7 12.16 5.08% Castro::normalize_species() 87 9.928 11.14 11.59 4.85% Gravity::solve_phi_with_mlmg() 11 11.23 11.35 11.58 4.84% Gravity::multilevel_solve_for_new_phi() 10 10.93 11.05 11.27 4.71% Gravity::actual_multilevel_solve() 10 10.9 11.01 11.24 4.70% Castro::do_old_sources() 9 11.06 11.09 11.16 4.67% FabArray::ParallelCopy_finish() 830 1.457 4.258 10.76 4.50% Castro::do_new_sources() 9 9.095 10.21 10.65 4.46% FillPatchSingleLevel 92 4.186 5.867 10.44 4.37% Amr::init() 1 10.37 10.37 10.37 4.34% FillPatchIterator::FillFromLevel0() 31 3.835 5.485 9.998 4.18% Castro::post_timestep() 9 9.908 9.909 9.91 4.15% Amr::initialInit() 1 9.754 9.754 9.754 4.08% Castro::reset_internal_energy(Fab) 1476 8.262 9.212 9.564 4.00% Castro::reflux() 4 8.896 8.936 8.955 3.75% Castro::estTimeStep() 26 3.947 5.288 8.683 3.63% Castro::initialize_advance() 9 7.328 8.168 8.51 3.56% FabArray::setVal() 5070 7.114 7.899 8.208 3.43% MultiFab::Subtract() 175 6.685 7.525 7.851 3.28% Gravity::actual_solve_with_mlmg() 11 6.879 6.888 6.905 2.89% Castro::construct_new_gravity() 9 6.333 6.446 6.655 2.78% MLMG::solve() 10 6.37 6.383 6.401 2.68% Castro::expand_state() 13 5.099 5.538 6.367 2.66% Amr::FinalizeInit() 1 5.702 5.843 6.155 2.57% MLMG::oneIter() 31 5.144 5.162 5.171 2.16% Gravity::fill_multipole_BCs() 11 4.324 4.441 4.669 1.95% FluxRegister::Reflux() 24 4.344 4.36 4.372 1.83% MLMG::mgVcycle() 65 4.3 4.332 4.35 1.82% Castro::construct_old_gravity() 9 0.6861 1.765 4.246 1.78% Gravity::get_old_grav_vector() 9 0.6696 1.757 4.245 1.78% Amr::InitializeInit() 1 3.598 3.91 4.052 1.69% Amr::defBaseLevel() 1 3.598 3.91 4.052 1.69% Castro::finalize_do_advance() 9 1.312 2.089 4.002 1.67% FabArray::sum() 4205 3.422 3.833 3.981 1.67% Castro::problem_post_timestep() 10 2.666 3.035 3.824 1.60% Castro::computeNewDt() 8 2.105 2.527 3.696 1.55% MLCellLinOp::smooth() 688 3.231 3.281 3.309 1.38% Castro::post_init() 1 2.524 2.694 3.034 1.27% Castro::FluxRegCrseInit 9 1.82 2.101 2.77 1.16% FabArray::ParallelCopy_nowait() 830 2.161 2.326 2.725 1.14% Castro::initData() 1 2.215 2.492 2.644 1.11% MLPoisson::Fsmooth() 1376 2.112 2.382 2.524 1.06% Castro::wd_update() 6 1.215 1.585 2.374 0.99% Castro::construct_new_source() 63 1.739 1.948 2.026 0.85% MLMG::mgVcycle_down::0 31 1.904 1.98 2.026 0.85% MLCellLinOp::applyBC() 1742 1.109 1.279 1.7 0.71% Amr::bldFineLevels() 1 1.232 1.432 1.552 0.65% FillPatchIterator::FillFromTwoLevels() 31 1.486 1.506 1.523 0.64% FillPatchTwoLevels 31 1.486 1.506 1.522 0.64% FillPatchInterp(MF) 30 1.212 1.331 1.441 0.60% MLCellLinOp::apply() 318 1.296 1.346 1.42 0.59% Castro::Castro() 5 1.348 1.374 1.415 0.59% MLMG::mgVcycle_up::0 31 1.369 1.398 1.411 0.59% FabArray::FillBoundary() 1553 0.6662 0.8647 1.345 0.56% Amr::InitAmr() 1 1.249 1.249 1.249 0.52% Amr::writePlotFile() 2 1.144 1.144 1.145 0.48% MLPoisson::Fapply() 318 0.9095 1.037 1.144 0.48% amrex::Add() 136 0.9865 1.098 1.143 0.48% Castro::initMFs() 5 1.092 1.12 1.138 0.48% MLCellLinOp::solutionResidual() 72 0.9633 1.035 1.102 0.46% Castro::construct_old_source() 63 0.8891 0.9999 1.052 0.44% Castro::construct_old_gravity_source() 18 0.8918 0.9882 1.04 0.44% Castro::computeInitialDt() 2 0.5393 0.6809 0.9929 0.42% FillBoundary_finish() 1553 0.3141 0.4867 0.9238 0.39% Castro::construct_new_gravity_source() 9 0.7397 0.8329 0.8741 0.37% MLCellLinOp::correctionResidual() 189 0.8196 0.8332 0.8522 0.36% FabArray::Saxpy() 182 0.6766 0.7611 0.7952 0.33% Gravity::get_new_grav_vector() 10 0.7733 0.7827 0.7932 0.33% Castro::volInBoundary() 42 0.6284 0.6955 0.7713 0.32% MLMG:computeResOfCorrection() 155 0.6573 0.6961 0.7389 0.31% VisMF::Write(FabArray) 3 0.5646 0.6545 0.7377 0.31% StateDataPhysBCFunct::() 95 0.5917 0.6061 0.619 0.26% StateData::FillBoundary(geom) 50 0.5865 0.5992 0.6147 0.26% MLMG::computeResWithCrseSolFineCor() 17 0.5899 0.5909 0.5926 0.25% Gravity::update_max_rhs() 10 0.08084 0.2504 0.5908 0.25% MultiFab::max() 18 0.02142 0.1925 0.5383 0.23% MLMG::computeMLResidual() 15 0.4316 0.4526 0.5104 0.21% Castro::init() 1 0.4801 0.482 0.4837 0.20% AmrLevel::FillCoarsePatch() 5 0.4801 0.482 0.4837 0.20% Castro::check_for_nan() 18 0.3927 0.4413 0.46 0.19% MultiFab::contains_nan() 18 0.3926 0.4412 0.4599 0.19% FillBoundary_nowait() 1553 0.32 0.3735 0.4171 0.17% Gravity::test_composite_phi() 1 0.4143 0.4161 0.4171 0.17% MLMG::mgVcycle_down::1 31 0.2996 0.3411 0.4091 0.17% FabArray::Xpay() 283 0.3489 0.3888 0.4053 0.17% amrex::average_down 233 0.2731 0.3573 0.4011 0.17% MLMG::computeResidual() 31 0.3469 0.3587 0.3765 0.16% MLCellLinOp::reflux() 30 0.1729 0.2374 0.3765 0.16% Castro::init(old) 3 0.3566 0.3571 0.3575 0.15% Castro::apply_source_to_state() 18 0.2713 0.3047 0.3175 0.13% MLPoisson::define() 11 0.2834 0.2951 0.3118 0.13% Castro::react_state() 18 0.2213 0.2387 0.246 0.10% MLMG::prepareForSolve() 10 0.2131 0.2212 0.2313 0.10% FabArray::setDomainBndry() 92 0.1922 0.2138 0.2309 0.10% MLMG::mgVcycle_up::1 31 0.2147 0.2184 0.2213 0.09% MLCellLinOp::defineAuxData() 11 0.1797 0.1959 0.2077 0.09% Castro::fill_hybrid_hydro_source() 27 0.1701 0.1893 0.1959 0.08% Amr::grid_places() 6 0.1903 0.1904 0.1904 0.08% AmrMesh::MakeNewGrids() 6 0.1898 0.1899 0.19 0.08% MLMG::addInterpCorrection() 155 0.1314 0.1667 0.1842 0.08% MLMG::mgVcycle_bottom 65 0.15 0.1586 0.1803 0.08% MLMG::getGradSolution() 10 0.1714 0.174 0.1762 0.07% MLCellLinOp::compGrad() 18 0.1714 0.1739 0.1762 0.07% MLPoisson::FFlux() 150 0.004606 0.04282 0.1573 0.07% MLMG::miniCycle() 34 0.1501 0.1516 0.1529 0.06% FabArray::mult() 56 0.115 0.1291 0.1352 0.06% Castro::construct_new_hybrid_source() 9 0.1122 0.1255 0.1302 0.05% Castro::avgDown() 4 0.1197 0.1221 0.1247 0.05% Castro::avgDown(state_indx) 20 0.1197 0.1221 0.1247 0.05% amrex::average_down_w_geom 20 0.1196 0.122 0.1246 0.05% AmrLevel::AmrLevel(dm) 5 0.1075 0.1188 0.1239 0.05% StateData::define() 25 0.1074 0.1186 0.1237 0.05% Castro::errorEst() 6 0.1047 0.1163 0.121 0.05% MLMG::ResNormInf() 54 0.07329 0.09109 0.1171 0.05% MLCellLinOp::updateSolBC() 25 0.0772 0.08446 0.09606 0.04% Castro::swap_state_time_levels() 9 0.08637 0.09177 0.09574 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08503 0.08636 0.08809 0.04% MLCellLinOp::defineBC() 11 0.08475 0.08591 0.08797 0.04% MLMG::mgVcycle_up::3 31 0.02566 0.06559 0.07912 0.03% MLMG::mgVcycle_down::2 31 0.06614 0.07073 0.07883 0.03% MLMG::MLResNormInf() 15 0.04357 0.06667 0.07496 0.03% Castro::construct_old_hybrid_source() 9 0.05791 0.06389 0.06649 0.03% FabArray::norminf(mask) 21 0.05299 0.05963 0.06225 0.03% MLMG::interpCorrection_1 17 0.02608 0.03628 0.06159 0.03% BndryData::define() 27 0.05756 0.05881 0.06078 0.03% Castro::apply_tagging_restrictions() 6 0.04787 0.05333 0.05567 0.02% MLMG::mgVcycle_up::2 31 0.05143 0.05212 0.05275 0.02% MLCellLinOp::updateCorBC() 17 0.04951 0.05018 0.05167 0.02% Castro::derive() 6 0.04123 0.04549 0.04708 0.02% AmrLevel::derive() 6 0.04121 0.04547 0.04706 0.02% Castro::buildMetrics() 5 0.04192 0.0434 0.04495 0.02% FabArray::norminf() 147 0.04052 0.0426 0.04489 0.02% Gravity::swapTimeLevels() 9 0.03179 0.03461 0.03604 0.02% MLMG::mgVcycle_down::3 31 0.03133 0.03228 0.03499 0.01% MLMG::MLRhsNormInf() 10 0.02496 0.02793 0.02928 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01876 0.02265 0.02884 0.01% MLLinOp::define() 11 0.003185 0.01325 0.02869 0.01% MLLinOp::defineGrids() 11 0.003112 0.01319 0.02863 0.01% FabArrayBase::getCPC() 1440 0.02243 0.02456 0.0279 0.01% MLMG::actualBottomSolve() 31 3.816e-05 0.006992 0.02785 0.01% TagBoxArray::mapPRD 6 0.003929 0.01313 0.02754 0.01% MLCGSolver::bicgstab 7 0 0.006776 0.0271 0.01% MLLinOp::makeSubCommunicator() 11 0.001348 0.01141 0.02698 0.01% Castro::FluxRegFineAdd() 9 0.0255 0.02569 0.02604 0.01% MultiFab::Multiply() 8 0.02121 0.02381 0.02482 0.01% MLMG::compResidual() 1 0.01933 0.021 0.02243 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01947 0.02011 0.02174 0.01% Castro::build_fine_mask() 44 0.0165 0.01749 0.01881 0.01% MLMG::mgVcycle_down::4 31 0.005168 0.00863 0.01868 0.01% FabArrayBase::CPC::define() 416 0.01285 0.01472 0.01789 0.01% Castro::linear_to_hybrid_momentum() 1 0.01405 0.01576 0.01643 0.01% MLMG::mgVcycle_up::4 31 0.004262 0.006757 0.01408 0.01% MLPoisson::prepareForSolve() 11 0.01138 0.01158 0.0119 0.00% MLCellLinOp::prepareForSolve() 11 0.01133 0.01153 0.01186 0.00% Castro::enforce_consistent_e() 1 0.009373 0.01058 0.01105 0.00% AMRErrorTag::operator() 6 0.008809 0.009854 0.0103 0.00% average_down_faces 48 0.008943 0.009176 0.009413 0.00% Castro::apply_problem_tags() 6 0.006726 0.007555 0.007844 0.00% FabArrayBase::getFB() 1553 0.005587 0.006207 0.006651 0.00% MLCellLinOp::setLevelBC() 19 0.004379 0.005441 0.006454 0.00% FabArrayBase::FB::FB() 19 0.002491 0.002965 0.003404 0.00% amrex::Dot() 115 0 0.0008471 0.003388 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007351 0.00294 0.00% FabArrayBase::TheFPinfo() 25 0.0009548 0.001551 0.002056 0.00% FPinfo::FPinfo() 3 0.0007674 0.00137 0.001873 0.00% DistributionMapping::KnapSackDoIt() 3 0.0006446 0.00125 0.00175 0.00% DistributionMapping::LeastUsedCPUs() 5 0.000609 0.001215 0.00172 0.00% TagBoxArray::collate() 6 0.0006685 0.001033 0.001276 0.00% OwnerMask() 6 0.0007265 0.0008313 0.0008939 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003637 0.000383 0.0004109 0.00% Castro::finalize_advance() 9 0.0002856 0.0003385 0.0003684 0.00% makeSFC 33 0.0002571 0.0002715 0.0002923 0.00% Castro::enforce_speed_limit() 87 0.0002535 0.0002626 0.0002737 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001178 0.0001288 0.0001364 0.00% AmrMesh-cluster 1 0 3.202e-05 0.0001281 0.00% DistributionMapping::Distribute() 35 0.0001066 0.0001146 0.0001229 0.00% FabArray::RecvLayoutMask() 8 8.91e-05 9.638e-05 0.0001113 0.00% Amr::writeSmallPlotFile() 1 8.833e-05 9.512e-05 0.0001049 0.00% ClusterList::intersect() 1 0 2.03e-05 8.118e-05 0.00% knapsack() 3 6.873e-05 7.096e-05 7.373e-05 0.00% Castro::create_source_corrector() 9 4.971e-05 6.042e-05 7.174e-05 0.00% BoxList::parallelComplementIn() 12 4.899e-05 5.536e-05 7.158e-05 0.00% Gravity::set_mass_offset() 6 4.212e-05 4.847e-05 5.332e-05 0.00% Amr::initSubcycle() 1 4.84e-05 5.113e-05 5.223e-05 0.00% Castro::retry_advance_ctu() 5 2.365e-05 2.53e-05 2.925e-05 0.00% Castro::construct_new_rotation_source() 9 2.039e-05 2.355e-05 2.843e-05 0.00% BoxList::complementIn 12 2.285e-05 2.361e-05 2.468e-05 0.00% Castro::construct_old_rotation_source() 9 2.042e-05 2.249e-05 2.382e-05 0.00% ClusterList::chop() 1 0 2.548e-06 1.019e-05 0.00% knapsack()swap 1 3.862e-06 4.175e-06 4.445e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-48-g47347f785f5c) finalized