Initializing AMReX (v2024-35-gb892132f6144)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-35-gb892132f6144) initialized Starting run at 07:20:08 UTC on 2024-03-27. Successfully read inputs file ... Castro git describe: 24.03-31-g760ecc416 AMReX git describe: v2024-35-gb892132f6 Microphysics git describe: 24.03-22-gf47a7fcc reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.042558358 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.041260373 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.375512484 Gravity::multilevel_solve_for_new_phi() time = 1.797337247 Gravity::fill_multipole_BCs() time = 0.377296441 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.543422654 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.945651182 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.401113451 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 20.8256885 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.694711687 Gravity::multilevel_solve_for_new_phi() time = 1.765165532 Castro::construct_new_gravity() time = 1.883629608 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207916e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.353056798 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02454391683 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.27447654 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032364075 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.414814394 Gravity::multilevel_solve_for_new_phi() time = 1.925890233 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.861821984 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.10694896 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 22.01132136 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.043770768 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.573032682e+22 ymom added: 1.593798688e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.197291587 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.852607022 on level 1 Castro::reflux() at level 0 : time = 2.284934523 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.436798634 Gravity::multilevel_solve_for_new_phi() time = 1.414340444 Castro::construct_new_gravity() time = 1.528149411 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 2.091381947 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.030130664 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764917034e+18 ymom added: -1.360987807e+19 zmom added: -8.219114359e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.151304904 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02219217351 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 51.18779993 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.032067545 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.416414079 Gravity::multilevel_solve_for_new_phi() time = 0.54874777 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.911074643 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.084420531 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.93613592 on level 0 Castro::construct_old_gravity() time = 0.03812569 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.44217652e+22 ymom added: 4.427218578e+21 zmom added: 7.103005275e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.202874939 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.820378331 on level 1 Castro::reflux() at level 0 : time = 2.26700991 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.437770801 Gravity::multilevel_solve_for_new_phi() time = 0.995920025 Castro::construct_new_gravity() time = 1.114410073 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.027988184 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.030085085 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.820365632e+17 ymom added: 2.351329365e+19 zmom added: 9.539392143e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.146163199 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02259569629 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 48.55357703 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032851634 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.413591211 Gravity::multilevel_solve_for_new_phi() time = 0.545944818 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.834425528 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.046633814 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.20855589 on level 0 Castro::construct_old_gravity() time = 0.038844119 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.199458711e+22 ymom added: 1.239621202e+22 zmom added: 8.023901326e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.19763519 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.791480886 on level 1 Castro::reflux() at level 0 : time = 2.209247831 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.44758807 Gravity::multilevel_solve_for_new_phi() time = 1.006877211 Castro::construct_new_gravity() time = 1.121630856 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 2.129659628 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.031875011 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598318086e+18 ymom added: -2.568853227e+19 zmom added: -1.336957912e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.163381455 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02307732166 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.54036685 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.03314917 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.425536318 Gravity::multilevel_solve_for_new_phi() time = 0.555119801 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.811731758 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.038419576 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.25476202 on level 0 Castro::construct_old_gravity() time = 0.037103789 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013662616e+23 ymom added: 1.770887431e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.196505173 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.781594754 on level 1 Castro::reflux() at level 0 : time = 2.231259657 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.4233317 Gravity::multilevel_solve_for_new_phi() time = 0.971778982 Castro::construct_new_gravity() time = 1.086530836 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.012600777 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.030013773 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727060174e+18 ymom added: -2.54093091e+19 zmom added: -1.252337601e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.152870177 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02314726017 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.49432751 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.529260175 seconds Ending run at 07:24:13 UTC on 2024-03-27. Run time = 245.4191414 Run time without initialization = 233.5816596 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 245.4 ... 245.4 ... 245.4 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 92.2507 103.3934 107.5497 43.82% Castro::computeTemp() 88 14.96 16.99 17.87 7.28% check_for_negative_density() 9 0.7044 4.887 16.1 6.56% amrex::Copy() 679 13.21 14.97 15.66 6.38% MultiFab::AddProduct() 4205 10.3 12.37 15.05 6.13% FabArray::ParallelCopy_finish() 830 1.498 4.911 12.66 5.16% Castro::normalize_species() 87 10.01 11.46 12.12 4.94% Castro::estTimeStep() 26 4.105 6.11 10.56 4.30% Castro::reset_internal_energy(Fab) 1476 8.512 9.463 9.83 4.01% FabArray::setVal() 5070 7.22 8.223 8.625 3.51% MultiFab::Subtract() 175 6.575 7.47 7.827 3.19% Castro::reset_internal_energy(MultiFab) 88 3.268 4.125 5.178 2.11% Gravity::fill_multipole_BCs() 11 4.358 4.514 4.781 1.95% FabArray::sum() 4205 3.189 3.599 3.742 1.52% FluxRegister::Reflux() 24 2.89 3.268 3.401 1.39% Castro::enforce_min_density() 87 1.461 2.085 3.17 1.29% FabArray::ParallelCopy_nowait() 830 2.045 2.247 2.648 1.08% MLPoisson::Fsmooth() 1376 2.085 2.4 2.581 1.05% Castro::wd_update() 6 0.5074 1.142 2.351 0.96% Castro::reflux() 4 1.462 1.661 1.737 0.71% Castro::FluxRegCrseInit 9 1.422 1.629 1.725 0.70% FillPatchInterp(MF) 30 1.224 1.333 1.437 0.59% Amr::InitAmr() 1 1.202 1.202 1.202 0.49% Castro::initMFs() 5 1.118 1.145 1.185 0.48% amrex::Add() 136 0.9381 1.066 1.115 0.45% FillBoundary_finish() 1553 0.2637 0.5278 1.07 0.44% Castro::construct_old_gravity_source() 18 0.8741 1.008 1.066 0.43% MLPoisson::Fapply() 318 0.876 0.9907 1.035 0.42% Castro::volInBoundary() 42 0.6023 0.7673 0.943 0.38% Castro::problem_post_timestep() 10 0.7931 0.8375 0.9309 0.38% Castro::construct_new_gravity_source() 9 0.7342 0.8502 0.9079 0.37% FillPatchIterator::Initialize 62 0.7224 0.821 0.8652 0.35% FabArray::Saxpy() 182 0.6807 0.7701 0.8077 0.33% VisMF::Write(FabArray) 3 0.5675 0.636 0.6718 0.27% StateData::FillBoundary(geom) 50 0.5929 0.6064 0.6253 0.25% Castro::construct_new_source() 63 0.4702 0.5347 0.5616 0.23% MultiFab::max() 18 0.02195 0.1788 0.5008 0.20% MLCellLinOp::applyBC() 1742 0.3546 0.4328 0.4751 0.19% MultiFab::contains_nan() 18 0.3721 0.42 0.4406 0.18% FillBoundary_nowait() 1553 0.3342 0.372 0.4256 0.17% FabArray::Xpay() 283 0.3579 0.4043 0.4233 0.17% Castro::initialize_do_advance() 9 0.2264 0.2564 0.269 0.11% Amr::writePlotFile() 2 0.1011 0.1494 0.2466 0.10% FabArray::setDomainBndry() 92 0.1931 0.2178 0.239 0.10% amrex::average_down 233 0.1961 0.2069 0.2115 0.09% Castro::construct_old_source() 63 0.1749 0.1981 0.2096 0.09% Castro::fill_hybrid_hydro_source() 27 0.1668 0.1878 0.1975 0.08% MLCellLinOp::defineAuxData() 11 0.151 0.1679 0.1766 0.07% Castro::initData() 1 0.1271 0.1425 0.1582 0.06% MLPoisson::FFlux() 150 0.004566 0.04225 0.1553 0.06% MLCellLinOp::compGrad() 18 0.1231 0.1399 0.1468 0.06% FabArray::mult() 56 0.1155 0.1311 0.1365 0.06% MLMG::addInterpCorrection() 155 0.1067 0.1207 0.1259 0.05% StateData::define() 25 0.1061 0.1165 0.1231 0.05% Castro::evaluate_source_change() 193 0.0989 0.1105 0.1161 0.05% Castro::swap_state_time_levels() 9 0.08296 0.09895 0.108 0.04% Gravity::get_new_grav_vector() 10 0.08232 0.09491 0.09978 0.04% Castro::initialize_advance() 9 0.066 0.08352 0.0951 0.04% Gravity::get_old_grav_vector() 9 0.07741 0.08631 0.09003 0.04% MLMG::prepareForSolve() 10 0.06042 0.06901 0.07929 0.03% MLMG::solve() 10 0.0006122 0.04318 0.0702 0.03% MLCellLinOp::updateSolBC() 25 0.06683 0.06729 0.06773 0.03% FabArray::norminf(mask) 21 0.0531 0.06061 0.06371 0.03% Castro::apply_tagging_restrictions() 6 0.05427 0.05523 0.05779 0.02% Castro::do_old_sources() 9 0.0005836 0.02743 0.05643 0.02% AmrLevel::FillPatch() 62 0.04621 0.05106 0.05616 0.02% AmrMesh::MakeNewGrids() 6 0.04687 0.05051 0.05474 0.02% Castro::do_new_sources() 9 0.0005698 0.01367 0.05285 0.02% MLMG::ResNormInf() 54 0.001888 0.01799 0.05281 0.02% MLCellLinOp::updateCorBC() 17 0.04536 0.04573 0.04651 0.02% BndryData::define() 27 0.04104 0.04317 0.0445 0.02% FabArray::norminf() 147 0.03751 0.04009 0.04288 0.02% MLMG::MLResNormInf() 15 0.0001066 0.02481 0.03631 0.01% Gravity::multilevel_solve_for_new_phi() 10 0.03049 0.03283 0.03528 0.01% MLCellLinOp::reflux() 30 0.004167 0.01184 0.03442 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01868 0.02296 0.02981 0.01% MultiFab::Multiply() 8 0.02208 0.02533 0.02667 0.01% MLCellLinOp::defineBC() 11 0.02564 0.02603 0.02629 0.01% Castro::FluxRegFineAdd() 9 0.0252 0.02539 0.02575 0.01% Castro::construct_old_gravity() 9 0.0003702 0.0122 0.0249 0.01% StateDataPhysBCFunct::() 95 0.002018 0.009057 0.02305 0.01% Castro::build_fine_mask() 44 0.01717 0.01868 0.02121 0.01% MLLinOp::makeSubCommunicator() 11 0.0006443 0.008807 0.0204 0.01% MLMG::interpCorrection_1 17 0.01828 0.01845 0.01861 0.01% FabArrayBase::CPC::define() 416 0.01289 0.01481 0.01811 0.01% DistributionMapping::LeastUsedCPUs() 5 0.0004476 0.007669 0.01737 0.01% Castro::buildMetrics() 5 0.01517 0.01634 0.01708 0.01% Castro::Castro() 5 0.015 0.01632 0.01681 0.01% Castro::linear_to_hybrid_momentum() 1 0.01436 0.01607 0.01669 0.01% FillPatchTwoLevels 31 0.01297 0.01396 0.01489 0.01% MLCellLinOp::prepareForSolve() 11 0.01144 0.01193 0.01244 0.01% Castro::enforce_consistent_e() 1 0.009318 0.01057 0.011 0.00% AMRErrorTag::operator() 6 0.009839 0.01018 0.01087 0.00% FabArrayBase::getCPC() 1440 0.009594 0.009905 0.01041 0.00% Gravity::actual_solve_with_mlmg() 11 0.008814 0.00935 0.01017 0.00% Castro::finalize_do_advance() 9 0.008074 0.008639 0.009476 0.00% Gravity::actual_multilevel_solve() 10 0.008733 0.009006 0.00938 0.00% Gravity::update_max_rhs() 10 0.007081 0.008288 0.009073 0.00% Castro::apply_problem_tags() 6 0.007311 0.007444 0.007669 0.00% average_down_faces 48 0.005627 0.005929 0.00668 0.00% MLCellLinOp::setLevelBC() 19 0.003604 0.004803 0.005402 0.00% FabArray::FillBoundary() 1553 0.004187 0.004354 0.004825 0.00% AmrLevel::derive() 6 0.003658 0.004136 0.004348 0.00% Gravity::test_composite_phi() 1 0.002085 0.002795 0.004064 0.00% MLCellLinOp::smooth() 688 0.003633 0.003808 0.003955 0.00% MLCGSolver::bicgstab 7 0 0.0009694 0.003878 0.00% amrex::Dot() 115 0 0.0009655 0.003862 0.00% FabArrayBase::getFB() 1553 0.003081 0.003247 0.003598 0.00% FabArrayBase::FB::FB() 19 0.00252 0.002828 0.003096 0.00% FabArray::ParallelCopy() 830 0.002813 0.002938 0.003044 0.00% Castro::construct_new_gravity() 9 0.0004233 0.001177 0.001941 0.00% main() 1 0.001476 0.001552 0.001667 0.00% MLCellLinOp::apply() 318 0.0008574 0.001036 0.001513 0.00% Amr::regrid() 5 0.0009544 0.001116 0.001438 0.00% MLLinOp::defineGrids() 11 0.001352 0.001392 0.001415 0.00% Castro::clean_state() 87 0.001207 0.001314 0.001383 0.00% Amr::coarseTimeStep() 5 0.0005528 0.001028 0.001208 0.00% TagBoxArray::collate() 6 0.0005991 0.0009151 0.00118 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002774 0.00111 0.00% AmrLevel::FillCoarsePatch() 5 0.0009275 0.0009502 0.0009713 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008854 0.0009073 0.0009171 0.00% OwnerMask() 6 0.0007374 0.0008466 0.0008929 0.00% MLCellLinOp::correctionResidual() 189 0.000674 0.0006816 0.0006881 0.00% MLMG::mgVcycle() 65 0.0005876 0.0005955 0.0006071 0.00% FillPatchSingleLevel 92 0.0005392 0.0005558 0.0005969 0.00% Castro::finalize_advance() 9 0.0003533 0.000477 0.0005677 0.00% Castro::do_advance_ctu() 5 0.0004729 0.0005183 0.0005635 0.00% Amr::grid_places() 6 0.0003738 0.0004874 0.0005588 0.00% TagBoxArray::mapPRD 6 0.0004379 0.0004437 0.0004493 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003906 0.0004068 0.0004367 0.00% Castro::subcycle_advance_ctu() 5 6.703e-05 0.0001573 0.0003986 0.00% MLMG:computeResOfCorrection() 155 0.0003726 0.0003854 0.0003972 0.00% MLMG::actualBottomSolve() 31 4.183e-05 0.0001239 0.0003689 0.00% MLMG::oneIter() 31 0.0003293 0.0003452 0.0003621 0.00% MLCellLinOp::solutionResidual() 72 0.0003365 0.0003436 0.0003522 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.000286 0.0003007 0.000314 0.00% Amr::timeStep() 9 0.0001936 0.0002249 0.0002939 0.00% MLMG::mgVcycle_down::0 31 0.0002584 0.0002769 0.0002892 0.00% Castro::enforce_speed_limit() 87 0.0002523 0.0002702 0.0002873 0.00% Castro::computeNewDt() 8 0.0001118 0.0001536 0.000271 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002317 0.0002403 0.0002466 0.00% Castro::post_regrid() 10 0.0001857 0.000208 0.0002458 0.00% Amr::defBaseLevel() 1 0.0002182 0.0002322 0.0002419 0.00% MLMG::mgVcycle_down::1 31 0.0002313 0.0002346 0.0002399 0.00% MLMG::mgVcycle_bottom 65 0.00021 0.0002151 0.0002224 0.00% makeSFC 33 0.0001665 0.0001749 0.0001982 0.00% AmrLevel::AmrLevel(dm) 5 0.0001878 0.0001915 0.000195 0.00% MLMG::mgVcycle_down::2 31 0.0001846 0.0001899 0.0001945 0.00% FabArrayBase::TheFPinfo() 25 0.0001807 0.0001875 0.0001939 0.00% MLMG::mgVcycle_down::3 31 0.0001855 0.0001885 0.0001915 0.00% MLMG::mgVcycle_up::0 31 0.0001738 0.0001838 0.0001891 0.00% MLMG::mgVcycle_down::4 31 0.0001674 0.0001713 0.0001811 0.00% Amr::FinalizeInit() 1 3.466e-05 7.617e-05 0.0001774 0.00% Castro::react_state() 18 0.000122 0.0001304 0.0001444 0.00% Castro::check_for_nan() 18 0.0001298 0.0001391 0.0001438 0.00% MLMG::mgVcycle_up::1 31 0.0001286 0.0001349 0.0001425 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001334 0.0001351 0.000139 0.00% MLMG::mgVcycle_up::3 31 0.0001207 0.0001261 0.000131 0.00% MLMG::mgVcycle_up::2 31 0.0001253 0.0001285 0.0001309 0.00% Castro::expand_state() 13 0.0001038 0.0001177 0.0001289 0.00% MLMG::mgVcycle_up::4 31 0.0001112 0.0001172 0.0001268 0.00% Gravity::swapTimeLevels() 9 8.618e-05 0.0001004 0.0001266 0.00% FPinfo::FPinfo() 3 0.0001115 0.0001189 0.0001262 0.00% FabArray::RecvLayoutMask() 8 9.134e-05 9.934e-05 0.0001164 0.00% MLPoisson::define() 11 0.0001017 0.0001065 0.0001141 0.00% Castro::apply_source_to_state() 18 0.0001012 0.0001054 0.000114 0.00% DistributionMapping::Distribute() 35 0.0001075 0.0001114 0.0001133 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001067 0.0001093 0.0001118 0.00% Castro::errorEst() 6 9.968e-05 0.0001059 0.0001109 0.00% Castro::advance() 5 8.392e-05 9.437e-05 0.0001074 0.00% Amr::writeSmallPlotFile() 1 8.976e-05 9.379e-05 9.672e-05 0.00% MLMG::computeResidual() 31 7.925e-05 8.401e-05 8.923e-05 0.00% Castro::post_timestep() 9 8.044e-05 8.351e-05 8.774e-05 0.00% ClusterList::intersect() 1 0 2.162e-05 8.648e-05 0.00% MLMG::computeMLResidual() 15 8.072e-05 8.315e-05 8.556e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.078e-05 6.707e-05 7.878e-05 0.00% Castro::avgDown(state_indx) 20 6.443e-05 7.151e-05 7.499e-05 0.00% MLMG::miniCycle() 34 6.528e-05 6.869e-05 7.442e-05 0.00% knapsack() 3 6.484e-05 6.739e-05 7.011e-05 0.00% MLLinOp::define() 11 5.854e-05 6.082e-05 6.255e-05 0.00% MLMG::getGradSolution() 10 5.706e-05 5.954e-05 6.183e-05 0.00% Castro::create_source_corrector() 9 6.036e-05 6.102e-05 6.166e-05 0.00% amrex::average_down_w_geom 20 5.023e-05 5.226e-05 5.457e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.709e-05 5.083e-05 5.408e-05 0.00% BoxList::parallelComplementIn() 12 2.393e-05 3.36e-05 5.337e-05 0.00% Castro::construct_new_hybrid_source() 9 4.763e-05 5.098e-05 5.291e-05 0.00% Amr::initSubcycle() 1 4.991e-05 5.056e-05 5.094e-05 0.00% MLPoisson::prepareForSolve() 11 4.456e-05 4.686e-05 4.949e-05 0.00% Castro::init(old) 3 4.344e-05 4.41e-05 4.469e-05 0.00% Gravity::set_mass_offset() 6 3.607e-05 3.802e-05 4.248e-05 0.00% Castro::construct_old_hybrid_source() 9 3.456e-05 3.746e-05 3.986e-05 0.00% AmrMesh-cluster 1 0 9.838e-06 3.935e-05 0.00% Castro::post_init() 1 2.23e-05 2.978e-05 3.9e-05 0.00% Castro::avgDown() 4 2.962e-05 3.227e-05 3.709e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.999e-05 3.206e-05 3.389e-05 0.00% MLMG::MLRhsNormInf() 10 3.152e-05 3.297e-05 3.378e-05 0.00% MLMG::compResidual() 1 2.659e-05 2.741e-05 2.827e-05 0.00% Castro::construct_new_rotation_source() 9 2.062e-05 2.355e-05 2.701e-05 0.00% Castro::retry_advance_ctu() 5 2.183e-05 2.393e-05 2.595e-05 0.00% Castro::construct_old_rotation_source() 9 2.23e-05 2.379e-05 2.592e-05 0.00% Amr::bldFineLevels() 1 1.657e-05 2.198e-05 2.556e-05 0.00% Castro::computeInitialDt() 2 1.335e-05 1.903e-05 2.354e-05 0.00% BoxList::complementIn 12 2.13e-05 2.177e-05 2.223e-05 0.00% Castro::derive() 6 1.836e-05 1.89e-05 1.97e-05 0.00% Castro::init() 1 1.573e-05 1.583e-05 1.599e-05 0.00% ClusterList::chop() 1 0 3.328e-06 1.331e-05 0.00% Amr::InitializeInit() 1 7.397e-06 8.693e-06 1.158e-05 0.00% Amr::initialInit() 1 4.934e-06 6.112e-06 7.07e-06 0.00% Amr::init() 1 6.142e-06 6.646e-06 6.948e-06 0.00% knapsack()swap 1 3.407e-06 3.617e-06 3.969e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 245.4 245.4 245.4 100.00% Amr::coarseTimeStep() 5 233.1 233.1 233.1 94.96% Amr::timeStep() 9 231.7 231.7 231.7 94.41% Castro::advance() 5 203.2 203.2 203.2 82.78% Castro::subcycle_advance_ctu() 5 194 194.6 195.9 79.81% Castro::do_advance_ctu() 5 194 194.6 195.9 79.81% Castro::construct_ctu_hydro_source() 9 118.1 118.3 118.4 48.23% Castro::clean_state() 87 71.08 82.09 86.98 35.44% Castro::computeTemp() 88 45.46 52.14 55.01 22.41% Castro::reset_internal_energy(MultiFab) 88 30.5 35.15 37.27 15.19% Castro::evaluate_source_change() 193 17.3 20.28 23.26 9.48% Castro::enforce_min_density() 87 16.4 19.39 21.02 8.56% Castro::post_regrid() 10 15.87 17.8 18.68 7.61% Amr::regrid() 5 15.78 17.54 18.33 7.47% check_for_negative_density() 9 0.7044 4.887 16.1 6.56% amrex::Copy() 679 13.21 14.97 15.66 6.38% Castro::initialize_do_advance() 9 15.46 15.53 15.66 6.38% AmrLevel::FillPatch() 62 8.021 10.24 15.46 6.30% MultiFab::AddProduct() 4205 10.3 12.37 15.05 6.13% FabArray::ParallelCopy() 830 4.176 7.223 14.86 6.05% FillPatchIterator::Initialize 62 6.24 8.525 13.91 5.67% FabArray::ParallelCopy_finish() 830 1.498 4.911 12.66 5.16% Castro::normalize_species() 87 10.01 11.46 12.12 4.94% FillPatchSingleLevel 92 4.054 6.36 11.9 4.85% Gravity::solve_phi_with_mlmg() 11 11.41 11.57 11.83 4.82% Gravity::multilevel_solve_for_new_phi() 10 11.12 11.27 11.52 4.69% Gravity::actual_multilevel_solve() 10 11.09 11.23 11.49 4.68% FillPatchIterator::FillFromLevel0() 31 3.666 5.962 11.43 4.66% Castro::do_old_sources() 9 11.36 11.39 11.43 4.66% Castro::do_new_sources() 9 9.111 10.56 11.23 4.57% Amr::init() 1 10.63 10.63 10.63 4.33% Castro::post_timestep() 9 10.59 10.59 10.59 4.32% Castro::estTimeStep() 26 4.105 6.11 10.56 4.30% Amr::initialInit() 1 10.09 10.09 10.09 4.11% Castro::reset_internal_energy(Fab) 1476 8.512 9.463 9.83 4.01% Castro::initialize_advance() 9 7.282 8.507 9.116 3.71% Castro::reflux() 4 8.934 8.97 8.989 3.66% FabArray::setVal() 5070 7.22 8.223 8.625 3.51% MultiFab::Subtract() 175 6.575 7.47 7.827 3.19% Castro::expand_state() 13 5.229 5.874 7.218 2.94% Gravity::actual_solve_with_mlmg() 11 6.946 6.955 6.963 2.84% Castro::construct_new_gravity() 9 6.472 6.613 6.855 2.79% MLMG::solve() 10 6.43 6.448 6.473 2.64% Amr::FinalizeInit() 1 5.877 6.051 6.39 2.60% MLMG::oneIter() 31 5.201 5.226 5.24 2.13% Castro::finalize_do_advance() 9 1.368 2.483 4.972 2.03% Gravity::fill_multipole_BCs() 11 4.444 4.594 4.854 1.98% Castro::computeNewDt() 8 2.243 2.959 4.583 1.87% Castro::construct_old_gravity() 9 0.729 1.863 4.522 1.84% Gravity::get_old_grav_vector() 9 0.7059 1.85 4.522 1.84% Castro::problem_post_timestep() 10 2.881 3.446 4.491 1.83% MLMG::mgVcycle() 65 4.373 4.41 4.433 1.81% FluxRegister::Reflux() 24 4.368 4.382 4.391 1.79% Amr::InitializeInit() 1 3.7 4.04 4.213 1.72% Amr::defBaseLevel() 1 3.7 4.04 4.213 1.72% FabArray::sum() 4205 3.189 3.599 3.742 1.52% MLCellLinOp::smooth() 688 3.333 3.357 3.375 1.38% Castro::post_init() 1 2.616 2.771 3.085 1.26% Castro::FluxRegCrseInit 9 1.777 2.127 2.96 1.21% Castro::wd_update() 6 1.343 1.909 2.954 1.20% Castro::initData() 1 2.235 2.563 2.735 1.11% FabArray::ParallelCopy_nowait() 830 2.106 2.31 2.715 1.11% MLPoisson::Fsmooth() 1376 2.085 2.4 2.581 1.05% Castro::construct_new_source() 63 1.737 1.989 2.103 0.86% MLMG::mgVcycle_down::0 31 1.934 2.007 2.034 0.83% MLCellLinOp::applyBC() 1742 1.093 1.343 1.818 0.74% FillPatchIterator::FillFromTwoLevels() 31 1.501 1.525 1.563 0.64% FillPatchTwoLevels 31 1.501 1.524 1.563 0.64% Amr::bldFineLevels() 1 1.355 1.492 1.561 0.64% Castro::Castro() 5 1.39 1.436 1.489 0.61% FabArray::FillBoundary() 1553 0.6184 0.9102 1.464 0.60% MLMG::mgVcycle_up::0 31 1.403 1.436 1.454 0.59% FillPatchInterp(MF) 30 1.224 1.333 1.437 0.59% MLCellLinOp::apply() 318 1.289 1.31 1.335 0.54% Amr::InitAmr() 1 1.203 1.203 1.203 0.49% Castro::initMFs() 5 1.125 1.15 1.191 0.49% amrex::Add() 136 0.9381 1.066 1.115 0.45% Amr::writePlotFile() 2 1.073 1.073 1.073 0.44% FillBoundary_finish() 1553 0.2637 0.5278 1.07 0.44% Castro::construct_old_gravity_source() 18 0.8741 1.008 1.066 0.43% MLCellLinOp::solutionResidual() 72 0.9808 1.033 1.059 0.43% Castro::construct_old_source() 63 0.8931 1.011 1.052 0.43% MLPoisson::Fapply() 318 0.876 0.9907 1.035 0.42% Castro::computeInitialDt() 2 0.5032 0.6769 1.016 0.41% Castro::volInBoundary() 42 0.6024 0.7674 0.9431 0.38% Castro::construct_new_gravity_source() 9 0.7342 0.8502 0.9079 0.37% Gravity::get_new_grav_vector() 10 0.8026 0.8142 0.825 0.34% MLCellLinOp::correctionResidual() 189 0.8034 0.8157 0.8223 0.34% FabArray::Saxpy() 182 0.6807 0.7701 0.8077 0.33% MLMG:computeResOfCorrection() 155 0.6581 0.689 0.7062 0.29% VisMF::Write(FabArray) 3 0.5675 0.636 0.6718 0.27% StateDataPhysBCFunct::() 95 0.5974 0.6155 0.6344 0.26% StateData::FillBoundary(geom) 50 0.5929 0.6064 0.6253 0.25% MLMG::computeResWithCrseSolFineCor() 17 0.5768 0.5779 0.5794 0.24% Gravity::update_max_rhs() 10 0.08389 0.2381 0.5533 0.23% MLMG::computeMLResidual() 15 0.4162 0.4475 0.5049 0.21% MultiFab::max() 18 0.02195 0.1788 0.5008 0.20% Castro::init() 1 0.4841 0.4857 0.4871 0.20% AmrLevel::FillCoarsePatch() 5 0.4841 0.4857 0.4871 0.20% amrex::average_down 233 0.2837 0.3869 0.4588 0.19% Castro::check_for_nan() 18 0.3722 0.4201 0.4408 0.18% MultiFab::contains_nan() 18 0.3721 0.42 0.4406 0.18% FillBoundary_nowait() 1553 0.3401 0.3781 0.4322 0.18% FabArray::Xpay() 283 0.3579 0.4043 0.4233 0.17% Gravity::test_composite_phi() 1 0.4169 0.419 0.4218 0.17% MLMG::mgVcycle_down::1 31 0.3137 0.3377 0.4032 0.16% Castro::init(old) 3 0.3677 0.3689 0.3695 0.15% MLCellLinOp::reflux() 30 0.1796 0.2315 0.3659 0.15% MLMG::computeResidual() 31 0.3526 0.3596 0.3649 0.15% Castro::apply_source_to_state() 18 0.2729 0.308 0.3214 0.13% MLPoisson::define() 11 0.2832 0.2957 0.3062 0.12% Castro::react_state() 18 0.2142 0.2483 0.2689 0.11% MLMG::prepareForSolve() 10 0.2215 0.2315 0.2403 0.10% FabArray::setDomainBndry() 92 0.1931 0.2178 0.239 0.10% MLMG::mgVcycle_up::1 31 0.2189 0.2233 0.2268 0.09% Amr::grid_places() 6 0.2142 0.2143 0.2143 0.09% AmrMesh::MakeNewGrids() 6 0.2137 0.2138 0.214 0.09% MLCellLinOp::defineAuxData() 11 0.1773 0.1978 0.2123 0.09% Castro::fill_hybrid_hydro_source() 27 0.1668 0.1878 0.1975 0.08% MLMG::addInterpCorrection() 155 0.1311 0.1693 0.1876 0.08% MLMG::mgVcycle_bottom 65 0.1549 0.1629 0.1851 0.08% MLMG::getGradSolution() 10 0.1723 0.177 0.1796 0.07% MLCellLinOp::compGrad() 18 0.1722 0.1769 0.1796 0.07% Castro::avgDown() 4 0.1227 0.1448 0.1672 0.07% Castro::avgDown(state_indx) 20 0.1227 0.1448 0.1671 0.07% amrex::average_down_w_geom 20 0.1226 0.1447 0.1671 0.07% MLMG::miniCycle() 34 0.1549 0.1555 0.1566 0.06% MLPoisson::FFlux() 150 0.004566 0.04225 0.1553 0.06% FabArray::mult() 56 0.1155 0.1311 0.1365 0.06% Castro::construct_new_hybrid_source() 9 0.1109 0.1248 0.1329 0.05% Castro::errorEst() 6 0.1194 0.1214 0.1242 0.05% AmrLevel::AmrLevel(dm) 5 0.1063 0.1166 0.1233 0.05% StateData::define() 25 0.1061 0.1165 0.1231 0.05% MLMG::ResNormInf() 54 0.07403 0.08765 0.1143 0.05% Castro::swap_state_time_levels() 9 0.08296 0.09895 0.108 0.04% MLCellLinOp::updateSolBC() 25 0.07737 0.08573 0.09875 0.04% MLCellLinOp::defineBC() 11 0.08353 0.08711 0.09047 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08589 0.08745 0.08909 0.04% MLMG::mgVcycle_up::3 31 0.02679 0.06868 0.08303 0.03% MLMG::mgVcycle_down::2 31 0.06892 0.07246 0.07996 0.03% MLMG::MLResNormInf() 15 0.04395 0.06729 0.07667 0.03% Castro::construct_old_hybrid_source() 9 0.056 0.06306 0.06805 0.03% FabArray::norminf(mask) 21 0.0531 0.06061 0.06371 0.03% BndryData::define() 27 0.05664 0.05981 0.0629 0.03% MLMG::interpCorrection_1 17 0.02608 0.03691 0.06191 0.03% Castro::apply_tagging_restrictions() 6 0.05427 0.05523 0.05779 0.02% MLMG::mgVcycle_up::2 31 0.05281 0.05317 0.05343 0.02% MLCellLinOp::updateCorBC() 17 0.05042 0.05072 0.0513 0.02% Castro::derive() 6 0.04771 0.04846 0.05011 0.02% AmrLevel::derive() 6 0.04769 0.04844 0.05009 0.02% Castro::buildMetrics() 5 0.03799 0.0422 0.04659 0.02% Gravity::swapTimeLevels() 9 0.03173 0.03807 0.04319 0.02% FabArray::norminf() 147 0.03751 0.04009 0.04288 0.02% MLMG::mgVcycle_down::3 31 0.03141 0.03253 0.03518 0.01% TagBoxArray::mapPRD 6 0.02181 0.02852 0.03386 0.01% MLMG::actualBottomSolve() 31 4.183e-05 0.007504 0.02989 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01868 0.02296 0.02981 0.01% MLCGSolver::bicgstab 7 0 0.007283 0.02913 0.01% MLMG::MLRhsNormInf() 10 0.02454 0.02769 0.02877 0.01% FabArrayBase::getCPC() 1440 0.02272 0.02472 0.02852 0.01% MultiFab::Multiply() 8 0.02208 0.02533 0.02667 0.01% Castro::FluxRegFineAdd() 9 0.0252 0.02539 0.02575 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.0208 0.02144 0.02302 0.01% MLLinOp::define() 11 0.002486 0.01064 0.02229 0.01% MLLinOp::defineGrids() 11 0.002423 0.01058 0.02223 0.01% MLMG::compResidual() 1 0.01895 0.02035 0.02129 0.01% Castro::build_fine_mask() 44 0.01717 0.01868 0.02121 0.01% MLLinOp::makeSubCommunicator() 11 0.0006443 0.008807 0.0204 0.01% MLMG::mgVcycle_down::4 31 0.005151 0.008784 0.01904 0.01% FabArrayBase::CPC::define() 416 0.01289 0.01481 0.01811 0.01% FabArrayBase::TheFPinfo() 25 0.0007939 0.00802 0.01773 0.01% FPinfo::FPinfo() 3 0.0006061 0.007833 0.01753 0.01% DistributionMapping::KnapSackDoIt() 3 0.0004931 0.007714 0.01741 0.01% DistributionMapping::LeastUsedCPUs() 5 0.0004476 0.007669 0.01737 0.01% Castro::linear_to_hybrid_momentum() 1 0.01436 0.01607 0.01669 0.01% MLMG::mgVcycle_up::4 31 0.004221 0.006854 0.0143 0.01% MLPoisson::prepareForSolve() 11 0.01149 0.01198 0.01248 0.01% MLCellLinOp::prepareForSolve() 11 0.01144 0.01193 0.01244 0.01% Castro::enforce_consistent_e() 1 0.009318 0.01057 0.011 0.00% AMRErrorTag::operator() 6 0.009839 0.01018 0.01087 0.00% average_down_faces 48 0.008968 0.009964 0.01086 0.00% Castro::apply_problem_tags() 6 0.007311 0.007444 0.007669 0.00% FabArrayBase::getFB() 1553 0.005679 0.006075 0.006533 0.00% MLCellLinOp::setLevelBC() 19 0.003604 0.004803 0.005402 0.00% amrex::Dot() 115 0 0.0009655 0.003862 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007891 0.003157 0.00% FabArrayBase::FB::FB() 19 0.00252 0.002828 0.003096 0.00% TagBoxArray::collate() 6 0.0005991 0.0009151 0.00118 0.00% OwnerMask() 6 0.0007374 0.0008466 0.0008929 0.00% Castro::finalize_advance() 9 0.0003533 0.000477 0.0005677 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003713 0.0003849 0.0004123 0.00% makeSFC 33 0.0002646 0.0002756 0.0003006 0.00% Castro::enforce_speed_limit() 87 0.0002523 0.0002702 0.0002873 0.00% AmrMesh-cluster 1 0 3.479e-05 0.0001391 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001137 0.0001197 0.0001252 0.00% FabArray::RecvLayoutMask() 8 9.134e-05 9.934e-05 0.0001164 0.00% DistributionMapping::Distribute() 35 0.0001075 0.0001114 0.0001133 0.00% Amr::writeSmallPlotFile() 1 8.976e-05 9.379e-05 9.672e-05 0.00% ClusterList::intersect() 1 0 2.162e-05 8.648e-05 0.00% BoxList::parallelComplementIn() 12 4.549e-05 5.537e-05 7.56e-05 0.00% knapsack() 3 6.824e-05 7.1e-05 7.352e-05 0.00% Castro::create_source_corrector() 9 6.036e-05 6.102e-05 6.166e-05 0.00% Amr::initSubcycle() 1 4.991e-05 5.056e-05 5.094e-05 0.00% Gravity::set_mass_offset() 6 3.607e-05 3.802e-05 4.248e-05 0.00% Castro::construct_new_rotation_source() 9 2.062e-05 2.355e-05 2.701e-05 0.00% Castro::retry_advance_ctu() 5 2.183e-05 2.393e-05 2.595e-05 0.00% Castro::construct_old_rotation_source() 9 2.23e-05 2.379e-05 2.592e-05 0.00% BoxList::complementIn 12 2.13e-05 2.177e-05 2.223e-05 0.00% ClusterList::chop() 1 0 3.328e-06 1.331e-05 0.00% knapsack()swap 1 3.407e-06 3.617e-06 3.969e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-35-gb892132f6144) finalized