Initializing AMReX (24.04-3-gd78e241d1ad4)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (24.04-3-gd78e241d1ad4) initialized Starting run at 07:31:34 UTC on 2024-04-03. Successfully read inputs file ... Castro git describe: 24.04-1-g6930369b2 AMReX git describe: 24.04-3-gd78e241d1 Microphysics git describe: 24.04 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.031928671 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031072734 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.378652176 Gravity::multilevel_solve_for_new_phi() time = 1.780771343 Gravity::fill_multipole_BCs() time = 0.370599832 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.568902591 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 1.149812514 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.405161583 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.31033342 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.734633839 Gravity::multilevel_solve_for_new_phi() time = 1.742629063 Castro::construct_new_gravity() time = 1.857978752 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730196e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.445262923 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02417303901 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.84332642 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032222447 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.410877913 Gravity::multilevel_solve_for_new_phi() time = 1.930163005 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.843826527 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.048277952 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.40331042 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.048215816 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.573814507e+22 ymom added: 1.475739526e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.202684709 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.800473834 on level 1 Castro::reflux() at level 0 : time = 2.246257279 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.408797997 Gravity::multilevel_solve_for_new_phi() time = 1.336983655 Castro::construct_new_gravity() time = 1.454171152 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.974068379 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029947841 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764863347e+18 ymom added: -1.361438167e+19 zmom added: -8.219114317e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.143998974 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02280327198 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.80987928 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.032276622 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.406700407 Gravity::multilevel_solve_for_new_phi() time = 0.534728772 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.763003366 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.043044756 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.20943885 on level 0 Castro::construct_old_gravity() time = 0.031143537 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.443638388e+22 ymom added: 5.017514388e+21 zmom added: 7.10314939e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.193747181 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.746546596 on level 1 Castro::reflux() at level 0 : time = 2.228954894 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.428346515 Gravity::multilevel_solve_for_new_phi() time = 0.96873152 Castro::construct_new_gravity() time = 1.07789064 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156837e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.985359318 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029485574 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.816822627e+17 ymom added: 2.352230085e+19 zmom added: 9.539392989e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.142450108 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02333806208 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 46.88055202 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.03241146 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.409547041 Gravity::multilevel_solve_for_new_phi() time = 0.537697607 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.74819348 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.021476913 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.04469797 on level 0 Castro::construct_old_gravity() time = 0.029495407 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.199411423e+22 ymom added: 1.180591621e+22 zmom added: 8.023901326e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.194375494 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.76442061 on level 1 Castro::reflux() at level 0 : time = 2.232097492 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.429121862 Gravity::multilevel_solve_for_new_phi() time = 0.970369705 Castro::construct_new_gravity() time = 1.080239708 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.969693343 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029333249 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598115999e+18 ymom added: -2.568853227e+19 zmom added: -1.336957898e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.174824496 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02345265146 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 46.64368163 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.03741658 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.412132064 Gravity::multilevel_solve_for_new_phi() time = 0.540892196 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.816333583 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.023150097 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.16778543 on level 0 Castro::construct_old_gravity() time = 0.032743548 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013261987e+23 ymom added: 1.829917012e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.200655888 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.746273189 on level 1 Castro::reflux() at level 0 : time = 2.222793552 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.42707677 Gravity::multilevel_solve_for_new_phi() time = 0.967014411 Castro::construct_new_gravity() time = 1.077062565 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.980694176 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029803289 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727107814e+18 ymom added: -2.54093091e+19 zmom added: -1.252339841e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.143253841 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02337679909 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 46.91271663 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.560863216 seconds Ending run at 07:35:35 UTC on 2024-04-03. Run time = 241.2757406 Run time without initialization = 229.6530048 Average number of zones advanced per microsecond: 0.022 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 241.3 ... 241.3 ... 241.3 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 91.2653 102.1276 106.8891 44.30% Castro::computeTemp() 88 14.61 16.5 17.24 7.15% check_for_negative_density() 9 0.246 5.025 15.96 6.62% amrex::Copy() 679 13.23 14.91 15.63 6.48% MultiFab::AddProduct() 4205 10.64 11.93 12.46 5.17% Castro::normalize_species() 87 9.932 11.24 11.8 4.89% FabArray::ParallelCopy_finish() 830 1.401 4.368 11.55 4.79% Castro::reset_internal_energy(Fab) 1476 8.256 9.259 9.681 4.01% Castro::estTimeStep() 26 4.021 5.525 9.286 3.85% FabArray::setVal() 5070 6.993 7.888 8.271 3.43% MultiFab::Subtract() 175 6.908 7.788 8.155 3.38% Gravity::fill_multipole_BCs() 11 4.293 4.461 4.742 1.97% Castro::reset_internal_energy(MultiFab) 88 3.224 4.001 4.702 1.95% FabArray::sum() 4205 3.336 3.738 3.908 1.62% FluxRegister::Reflux() 24 2.929 3.287 3.429 1.42% FabArray::ParallelCopy_nowait() 830 2.017 2.235 2.636 1.09% MLPoisson::Fsmooth() 1376 2.081 2.348 2.448 1.01% Castro::enforce_min_density() 87 1.49 1.91 2.429 1.01% Castro::wd_update() 6 0.5291 0.9425 1.862 0.77% Castro::reflux() 4 1.48 1.665 1.727 0.72% Castro::FluxRegCrseInit 9 1.446 1.648 1.726 0.72% FillPatchInterp(MF) 30 1.218 1.326 1.436 0.60% Amr::InitAmr() 1 1.233 1.233 1.233 0.51% amrex::Add() 136 0.966 1.084 1.127 0.47% Castro::initMFs() 5 1.065 1.105 1.126 0.47% Castro::construct_old_gravity_source() 18 0.8963 1.013 1.063 0.44% MLPoisson::Fapply() 318 0.8899 0.9978 1.039 0.43% Castro::construct_new_gravity_source() 9 0.7433 0.859 0.9193 0.38% FillBoundary_finish() 1553 0.2738 0.4419 0.8945 0.37% Castro::problem_post_timestep() 10 0.7792 0.8223 0.87 0.36% FillPatchIterator::Initialize 62 0.7257 0.8245 0.8675 0.36% FabArray::Saxpy() 182 0.7012 0.7881 0.8279 0.34% Castro::volInBoundary() 42 0.6598 0.7571 0.8078 0.33% VisMF::Write(FabArray) 3 0.6198 0.6768 0.7077 0.29% StateData::FillBoundary(geom) 50 0.5909 0.6006 0.6127 0.25% Castro::construct_new_source() 63 0.4693 0.528 0.5535 0.23% MultiFab::contains_nan() 18 0.3874 0.4381 0.4609 0.19% MLCellLinOp::applyBC() 1742 0.3555 0.4264 0.4589 0.19% MultiFab::max() 18 0.02161 0.1419 0.4571 0.19% FabArray::Xpay() 283 0.349 0.3911 0.4068 0.17% FillBoundary_nowait() 1553 0.3303 0.3734 0.4061 0.17% Castro::initialize_do_advance() 9 0.2294 0.2643 0.2784 0.12% Amr::writePlotFile() 2 0.1214 0.1604 0.2404 0.10% FabArray::setDomainBndry() 92 0.1923 0.2182 0.2335 0.10% Castro::construct_old_source() 63 0.1756 0.1987 0.2104 0.09% amrex::average_down 233 0.1944 0.2045 0.2087 0.09% Castro::fill_hybrid_hydro_source() 27 0.1686 0.189 0.1967 0.08% MLCellLinOp::defineAuxData() 11 0.1516 0.1669 0.1741 0.07% MLPoisson::FFlux() 150 0.0046 0.04219 0.1549 0.06% Castro::initData() 1 0.121 0.1432 0.1539 0.06% MLCellLinOp::compGrad() 18 0.1198 0.1356 0.1414 0.06% FabArray::mult() 56 0.1138 0.1284 0.134 0.06% Castro::swap_state_time_levels() 9 0.0893 0.105 0.1325 0.05% MLMG::addInterpCorrection() 155 0.1085 0.1205 0.1257 0.05% StateData::define() 25 0.1071 0.1172 0.1217 0.05% Castro::evaluate_source_change() 193 0.09906 0.1094 0.1141 0.05% Gravity::get_new_grav_vector() 10 0.08768 0.0942 0.09721 0.04% Castro::initialize_advance() 9 0.0705 0.0822 0.09389 0.04% Gravity::get_old_grav_vector() 9 0.07503 0.08577 0.09344 0.04% Amr::coarseTimeStep() 5 0.0006424 0.05477 0.07282 0.03% MLMG::prepareForSolve() 10 0.05971 0.06699 0.07033 0.03% MLCellLinOp::updateSolBC() 25 0.06724 0.06785 0.06813 0.03% MLMG::solve() 10 0.0007505 0.04255 0.068 0.03% FabArray::norminf(mask) 21 0.05357 0.06047 0.06315 0.03% AmrLevel::FillPatch() 62 0.05035 0.05747 0.0628 0.03% Castro::apply_tagging_restrictions() 6 0.04946 0.05426 0.05701 0.02% AmrMesh::MakeNewGrids() 6 0.04612 0.0484 0.05408 0.02% MLCellLinOp::updateCorBC() 17 0.04548 0.04602 0.04643 0.02% MLMG::ResNormInf() 54 0.003108 0.01361 0.04395 0.02% BndryData::define() 27 0.04204 0.0431 0.04347 0.02% FabArray::norminf() 147 0.03729 0.03982 0.04255 0.02% Castro::do_new_sources() 9 0.0005816 0.01023 0.03912 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.02965 0.03337 0.03574 0.01% MLMG::MLResNormInf() 15 0.0001198 0.0246 0.03574 0.01% MLCellLinOp::reflux() 30 0.004127 0.01173 0.03431 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01888 0.02267 0.02903 0.01% Castro::FluxRegFineAdd() 9 0.02558 0.02613 0.02703 0.01% MLCellLinOp::defineBC() 11 0.02586 0.02596 0.02606 0.01% MultiFab::Multiply() 8 0.02169 0.02407 0.02504 0.01% StateDataPhysBCFunct::() 95 0.002033 0.008412 0.02365 0.01% MLMG::interpCorrection_1 17 0.01841 0.01883 0.01977 0.01% Castro::build_fine_mask() 44 0.01661 0.01778 0.01914 0.01% MLLinOp::makeSubCommunicator() 11 0.0006089 0.008226 0.01912 0.01% DistributionMapping::LeastUsedCPUs() 5 0.001919 0.007657 0.01897 0.01% FabArrayBase::CPC::define() 416 0.01315 0.01471 0.01805 0.01% Castro::Castro() 5 0.01456 0.01627 0.017 0.01% Castro::buildMetrics() 5 0.01484 0.01633 0.01689 0.01% Castro::linear_to_hybrid_momentum() 1 0.01363 0.01578 0.01651 0.01% Castro::do_old_sources() 9 0.0006104 0.005909 0.01633 0.01% FillPatchTwoLevels 31 0.0126 0.01314 0.01364 0.01% MLCellLinOp::prepareForSolve() 11 0.01135 0.01163 0.01194 0.00% Castro::enforce_consistent_e() 1 0.009176 0.01056 0.01105 0.00% Gravity::actual_multilevel_solve() 10 0.007635 0.009521 0.01086 0.00% FabArrayBase::getCPC() 1440 0.009643 0.01014 0.01064 0.00% Gravity::actual_solve_with_mlmg() 11 0.009087 0.009631 0.01035 0.00% AMRErrorTag::operator() 6 0.008975 0.009761 0.01031 0.00% Castro::construct_old_gravity() 9 0.0004035 0.005003 0.01013 0.00% Castro::finalize_do_advance() 9 0.008377 0.008846 0.00945 0.00% Gravity::update_max_rhs() 10 0.007347 0.008252 0.009018 0.00% Castro::apply_problem_tags() 6 0.007141 0.007806 0.008202 0.00% average_down_faces 48 0.005677 0.005722 0.005767 0.00% MLCellLinOp::setLevelBC() 19 0.00368 0.004605 0.005395 0.00% FabArray::FillBoundary() 1553 0.004203 0.00441 0.004981 0.00% AmrLevel::derive() 6 0.003694 0.004214 0.004433 0.00% MLCellLinOp::smooth() 688 0.003646 0.00381 0.004105 0.00% MLCGSolver::bicgstab 7 0 0.0009913 0.003965 0.00% FabArrayBase::getFB() 1553 0.00302 0.003207 0.003648 0.00% amrex::Dot() 115 0 0.0009034 0.003614 0.00% Gravity::test_composite_phi() 1 0.002127 0.002839 0.003426 0.00% FabArrayBase::FB::FB() 19 0.0025 0.002814 0.003112 0.00% FabArray::ParallelCopy() 830 0.002858 0.002959 0.003064 0.00% main() 1 0.001504 0.001624 0.001808 0.00% MLCellLinOp::apply() 318 0.0008516 0.001031 0.001524 0.00% MLLinOp::defineGrids() 11 0.001249 0.001377 0.001509 0.00% Amr::regrid() 5 0.0009114 0.001095 0.001457 0.00% Castro::clean_state() 87 0.001111 0.001215 0.001312 0.00% AmrLevel::FillCoarsePatch() 5 0.0008961 0.0009829 0.001177 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002883 0.001153 0.00% TagBoxArray::collate() 6 0.0003722 0.0008581 0.001127 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008884 0.0009084 0.0009328 0.00% OwnerMask() 6 0.000705 0.0008362 0.0008912 0.00% Castro::construct_new_gravity() 9 0.0004379 0.0005784 0.0008689 0.00% MLCellLinOp::correctionResidual() 189 0.0006796 0.0007043 0.0007371 0.00% MLMG::mgVcycle() 65 0.0005617 0.0005974 0.0006272 0.00% Castro::do_advance_ctu() 5 0.0004815 0.0005218 0.0005561 0.00% FillPatchSingleLevel 92 0.0005358 0.0005487 0.0005539 0.00% Amr::grid_places() 6 0.0003672 0.0004435 0.0005067 0.00% Castro::finalize_advance() 9 0.0003057 0.0004083 0.0004935 0.00% TagBoxArray::mapPRD 6 0.0004311 0.0004546 0.00048 0.00% Castro::subcycle_advance_ctu() 5 7.485e-05 0.0001935 0.0004333 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003756 0.000404 0.0004282 0.00% MLMG:computeResOfCorrection() 155 0.0003794 0.0003962 0.0004183 0.00% MLMG::actualBottomSolve() 31 3.884e-05 0.0001269 0.0003787 0.00% MLCellLinOp::solutionResidual() 72 0.0003465 0.0003556 0.0003624 0.00% MLMG::oneIter() 31 0.0003344 0.0003467 0.0003594 0.00% Castro::computeNewDt() 8 9.816e-05 0.000163 0.0003292 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002965 0.0003107 0.0003198 0.00% Amr::timeStep() 9 0.000179 0.0002364 0.0003137 0.00% MLMG::mgVcycle_down::0 31 0.0002677 0.0002819 0.0002979 0.00% Castro::enforce_speed_limit() 87 0.000252 0.0002637 0.0002845 0.00% MLMG::mgVcycle_down::1 31 0.000225 0.0002401 0.0002551 0.00% Amr::defBaseLevel() 1 0.0002379 0.0002447 0.0002503 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002151 0.0002345 0.0002462 0.00% MLMG::mgVcycle_bottom 65 0.0002148 0.0002194 0.0002238 0.00% Castro::post_regrid() 10 0.0001785 0.0001975 0.0002107 0.00% Amr::FinalizeInit() 1 3.01e-05 7.672e-05 0.0002102 0.00% MLMG::mgVcycle_down::2 31 0.0001871 0.0001942 0.0002002 0.00% MLMG::mgVcycle_down::3 31 0.0001831 0.0001895 0.0001972 0.00% MLMG::mgVcycle_up::0 31 0.0001763 0.0001837 0.0001937 0.00% AmrLevel::AmrLevel(dm) 5 0.0001824 0.0001857 0.0001917 0.00% FabArrayBase::TheFPinfo() 25 0.0001712 0.0001766 0.0001886 0.00% MLMG::mgVcycle_down::4 31 0.0001609 0.0001677 0.0001854 0.00% makeSFC 33 0.0001498 0.0001658 0.0001813 0.00% Castro::react_state() 18 0.0001197 0.0001441 0.0001706 0.00% Castro::check_for_nan() 18 0.000124 0.000132 0.0001459 0.00% MLMG::mgVcycle_up::1 31 0.0001323 0.0001387 0.0001452 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001235 0.000134 0.0001434 0.00% MLMG::mgVcycle_up::4 31 0.0001111 0.0001226 0.0001403 0.00% MLMG::mgVcycle_up::2 31 0.0001255 0.000129 0.0001335 0.00% FPinfo::FPinfo() 3 0.0001147 0.0001217 0.0001331 0.00% MLMG::mgVcycle_up::3 31 0.0001207 0.0001252 0.0001322 0.00% Castro::expand_state() 13 0.00011 0.0001138 0.000123 0.00% Castro::apply_source_to_state() 18 0.0001009 0.0001091 0.000121 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001034 0.0001104 0.000116 0.00% DistributionMapping::Distribute() 35 0.0001044 0.000109 0.0001149 0.00% FabArray::RecvLayoutMask() 8 9.075e-05 9.873e-05 0.0001145 0.00% Amr::writeSmallPlotFile() 1 9.147e-05 0.0001024 0.0001132 0.00% MLPoisson::define() 11 9.811e-05 0.0001054 0.0001126 0.00% Castro::errorEst() 6 0.0001006 0.0001058 0.0001107 0.00% Gravity::swapTimeLevels() 9 8.517e-05 9.238e-05 0.0001037 0.00% Castro::advance() 5 7.571e-05 8.582e-05 0.0001011 0.00% Castro::post_timestep() 9 7.559e-05 8.225e-05 9.393e-05 0.00% MLMG::computeResidual() 31 8.177e-05 8.485e-05 9.166e-05 0.00% ClusterList::intersect() 1 0 2.257e-05 9.028e-05 0.00% MLMG::computeMLResidual() 15 7.798e-05 8.087e-05 8.649e-05 0.00% Castro::avgDown(state_indx) 20 6.81e-05 7.319e-05 8.228e-05 0.00% MLMG::miniCycle() 34 6.547e-05 7.001e-05 7.538e-05 0.00% MLLinOp::define() 11 6.001e-05 6.569e-05 7.349e-05 0.00% knapsack() 3 6.511e-05 6.79e-05 7.317e-05 0.00% Castro::create_source_corrector() 9 5.04e-05 5.85e-05 6.846e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.016e-05 6.387e-05 6.716e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.361e-05 5.042e-05 6.639e-05 0.00% MLMG::getGradSolution() 10 5.649e-05 6.038e-05 6.507e-05 0.00% amrex::average_down_w_geom 20 4.895e-05 5.466e-05 5.987e-05 0.00% Castro::construct_new_hybrid_source() 9 4.542e-05 4.813e-05 5.268e-05 0.00% Amr::initSubcycle() 1 5.015e-05 5.056e-05 5.164e-05 0.00% Gravity::set_mass_offset() 6 2.819e-05 3.865e-05 5.06e-05 0.00% MLPoisson::prepareForSolve() 11 4.444e-05 4.736e-05 4.969e-05 0.00% Castro::init(old) 3 4.385e-05 4.484e-05 4.615e-05 0.00% AmrMesh-cluster 1 0 1.01e-05 4.039e-05 0.00% Castro::construct_old_hybrid_source() 9 3.295e-05 3.528e-05 3.902e-05 0.00% BoxList::complementIn 12 2.129e-05 2.607e-05 3.834e-05 0.00% MLMG::MLRhsNormInf() 10 3.125e-05 3.366e-05 3.794e-05 0.00% Castro::avgDown() 4 2.929e-05 3.316e-05 3.766e-05 0.00% Castro::retry_advance_ctu() 5 2.401e-05 2.803e-05 3.387e-05 0.00% DistributionMapping::KnapSackDoIt() 3 3.037e-05 3.13e-05 3.247e-05 0.00% Castro::computeInitialDt() 2 1.519e-05 2.081e-05 3.202e-05 0.00% Castro::construct_new_rotation_source() 9 2.186e-05 2.429e-05 3.058e-05 0.00% Castro::post_init() 1 2.14e-05 2.556e-05 2.981e-05 0.00% MLMG::compResidual() 1 2.664e-05 2.769e-05 2.979e-05 0.00% Amr::bldFineLevels() 1 1.4e-05 2.048e-05 2.939e-05 0.00% BoxList::parallelComplementIn() 12 2.477e-05 2.607e-05 2.733e-05 0.00% Amr::InitializeInit() 1 8.226e-06 1.373e-05 2.707e-05 0.00% Castro::construct_old_rotation_source() 9 2.091e-05 2.253e-05 2.329e-05 0.00% Castro::derive() 6 1.912e-05 2.054e-05 2.193e-05 0.00% Castro::init() 1 1.624e-05 1.642e-05 1.672e-05 0.00% ClusterList::chop() 1 0 3.072e-06 1.229e-05 0.00% Amr::init() 1 6.364e-06 7.954e-06 1.175e-05 0.00% Amr::initialInit() 1 4.411e-06 6.964e-06 1.113e-05 0.00% knapsack()swap 1 3.715e-06 4.053e-06 4.524e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 241.3 241.3 241.3 100.00% Amr::coarseTimeStep() 5 229.1 229.1 229.1 94.95% Amr::timeStep() 9 227.6 227.7 227.7 94.38% Castro::advance() 5 200.2 200.2 200.2 82.96% Castro::subcycle_advance_ctu() 5 191.5 191.9 192.8 79.92% Castro::do_advance_ctu() 5 191.5 191.9 192.8 79.92% Castro::construct_ctu_hydro_source() 9 116.8 117 117.1 48.53% Castro::clean_state() 87 71.13 80.73 84.68 35.10% Castro::computeTemp() 88 45.16 51.24 53.81 22.30% Castro::reset_internal_energy(MultiFab) 88 30.55 34.74 36.57 15.16% Castro::evaluate_source_change() 193 17.69 19.83 20.71 8.58% Castro::enforce_min_density() 87 16.84 19.15 20.2 8.37% Castro::post_regrid() 10 15.83 17.44 18.07 7.49% Amr::regrid() 5 15.71 17.18 17.75 7.36% check_for_negative_density() 9 0.246 5.025 15.96 6.62% amrex::Copy() 679 13.23 14.91 15.63 6.48% Castro::initialize_do_advance() 9 14.92 15.21 15.35 6.36% AmrLevel::FillPatch() 62 7.904 9.879 14.75 6.11% FabArray::ParallelCopy() 830 3.905 6.67 13.71 5.68% FillPatchIterator::Initialize 62 6.124 8.168 13.23 5.48% MultiFab::AddProduct() 4205 10.64 11.93 12.46 5.17% Castro::normalize_species() 87 9.932 11.24 11.8 4.89% Gravity::solve_phi_with_mlmg() 11 11.17 11.34 11.61 4.81% FabArray::ParallelCopy_finish() 830 1.401 4.368 11.55 4.79% Castro::do_old_sources() 9 11.23 11.27 11.32 4.69% Gravity::multilevel_solve_for_new_phi() 10 10.88 11.04 11.3 4.68% Gravity::actual_multilevel_solve() 10 10.84 11.01 11.27 4.67% FillPatchSingleLevel 92 3.904 6.006 11.24 4.66% Castro::do_new_sources() 9 9.193 10.43 10.95 4.54% FillPatchIterator::FillFromLevel0() 31 3.515 5.629 10.82 4.48% Amr::init() 1 10.39 10.39 10.39 4.31% Castro::post_timestep() 9 10.11 10.13 10.18 4.22% Amr::initialInit() 1 9.819 9.819 9.82 4.07% Castro::reset_internal_energy(Fab) 1476 8.256 9.259 9.681 4.01% Castro::estTimeStep() 26 4.021 5.525 9.286 3.85% Castro::reflux() 4 8.869 8.909 8.93 3.70% Castro::initialize_advance() 9 7.326 8.286 8.68 3.60% FabArray::setVal() 5070 6.993 7.888 8.271 3.43% MultiFab::Subtract() 175 6.908 7.788 8.155 3.38% Gravity::actual_solve_with_mlmg() 11 6.772 6.776 6.781 2.81% Castro::expand_state() 13 5.145 5.615 6.691 2.77% Castro::construct_new_gravity() 9 6.254 6.405 6.661 2.76% MLMG::solve() 10 6.264 6.278 6.299 2.61% Amr::FinalizeInit() 1 5.732 5.879 6.261 2.60% MLMG::oneIter() 31 5.074 5.095 5.105 2.12% Gravity::fill_multipole_BCs() 11 4.375 4.54 4.814 2.00% Castro::construct_old_gravity() 9 0.7309 1.813 4.437 1.84% Gravity::get_old_grav_vector() 9 0.7219 1.808 4.436 1.84% FluxRegister::Reflux() 24 4.365 4.368 4.372 1.81% MLMG::mgVcycle() 65 4.253 4.285 4.298 1.78% Castro::finalize_do_advance() 9 1.311 2.164 4.241 1.76% Amr::InitializeInit() 1 3.558 3.941 4.087 1.69% Amr::defBaseLevel() 1 3.558 3.941 4.087 1.69% Castro::problem_post_timestep() 10 2.875 3.205 4.009 1.66% Castro::computeNewDt() 8 2.167 2.676 3.978 1.65% FabArray::sum() 4205 3.336 3.738 3.908 1.62% MLCellLinOp::smooth() 688 3.23 3.247 3.269 1.35% Castro::post_init() 1 2.577 2.696 3.004 1.25% FabArray::ParallelCopy_nowait() 830 2.078 2.298 2.704 1.12% Castro::initData() 1 2.186 2.527 2.696 1.12% Castro::FluxRegCrseInit 9 1.772 2.017 2.654 1.10% Castro::wd_update() 6 1.315 1.7 2.522 1.05% MLPoisson::Fsmooth() 1376 2.081 2.348 2.448 1.01% Castro::construct_new_source() 63 1.751 1.992 2.101 0.87% MLMG::mgVcycle_down::0 31 1.88 1.949 1.973 0.82% MLCellLinOp::applyBC() 1742 1.105 1.252 1.666 0.69% FillPatchIterator::FillFromTwoLevels() 31 1.477 1.496 1.519 0.63% FillPatchTwoLevels 31 1.477 1.496 1.518 0.63% Amr::bldFineLevels() 1 1.268 1.439 1.503 0.62% FillPatchInterp(MF) 30 1.218 1.326 1.436 0.60% Castro::Castro() 5 1.338 1.371 1.418 0.59% MLMG::mgVcycle_up::0 31 1.361 1.39 1.401 0.58% FabArray::FillBoundary() 1553 0.6461 0.8258 1.311 0.54% MLCellLinOp::apply() 318 1.266 1.29 1.308 0.54% Amr::InitAmr() 1 1.233 1.233 1.233 0.51% Castro::initMFs() 5 1.07 1.111 1.131 0.47% Amr::writePlotFile() 2 1.13 1.13 1.131 0.47% amrex::Add() 136 0.966 1.084 1.127 0.47% Castro::construct_old_source() 63 0.8998 1.026 1.081 0.45% Castro::computeInitialDt() 2 0.5472 0.6937 1.077 0.45% Castro::construct_old_gravity_source() 18 0.8963 1.013 1.063 0.44% MLPoisson::Fapply() 318 0.8899 0.9978 1.039 0.43% MLCellLinOp::solutionResidual() 72 0.9605 1.011 1.032 0.43% Castro::construct_new_gravity_source() 9 0.7433 0.859 0.9193 0.38% FillBoundary_finish() 1553 0.2738 0.4419 0.8945 0.37% FabArray::Saxpy() 182 0.7012 0.7881 0.8279 0.34% MLCellLinOp::correctionResidual() 189 0.7973 0.8038 0.8099 0.34% Castro::volInBoundary() 42 0.6599 0.7571 0.8078 0.33% Gravity::get_new_grav_vector() 10 0.7697 0.7814 0.7936 0.33% VisMF::Write(FabArray) 3 0.6198 0.6768 0.7077 0.29% MLMG:computeResOfCorrection() 155 0.6476 0.6793 0.6949 0.29% StateDataPhysBCFunct::() 95 0.5929 0.609 0.6185 0.26% StateData::FillBoundary(geom) 50 0.5909 0.6006 0.6127 0.25% MLMG::computeResWithCrseSolFineCor() 17 0.5707 0.5714 0.5729 0.24% Gravity::update_max_rhs() 10 0.08239 0.201 0.51 0.21% MLMG::computeMLResidual() 15 0.4087 0.4314 0.4873 0.20% Castro::init() 1 0.4807 0.4819 0.4839 0.20% AmrLevel::FillCoarsePatch() 5 0.4806 0.4818 0.4839 0.20% Castro::check_for_nan() 18 0.3875 0.4382 0.4611 0.19% MultiFab::contains_nan() 18 0.3874 0.4381 0.4609 0.19% MultiFab::max() 18 0.02161 0.1419 0.4571 0.19% Gravity::test_composite_phi() 1 0.4105 0.4124 0.4156 0.17% FillBoundary_nowait() 1553 0.336 0.3794 0.4123 0.17% FabArray::Xpay() 283 0.349 0.3911 0.4068 0.17% amrex::average_down 233 0.2758 0.3596 0.4029 0.17% MLMG::mgVcycle_down::1 31 0.3015 0.3246 0.3876 0.16% Castro::init(old) 3 0.3608 0.3627 0.364 0.15% MLCellLinOp::reflux() 30 0.1745 0.2263 0.3611 0.15% MLMG::computeResidual() 31 0.3464 0.3528 0.3565 0.15% Castro::apply_source_to_state() 18 0.2802 0.3153 0.3341 0.14% MLPoisson::define() 11 0.2841 0.2934 0.3014 0.12% Castro::react_state() 18 0.2055 0.237 0.2572 0.11% FabArray::setDomainBndry() 92 0.1923 0.2182 0.2335 0.10% MLMG::prepareForSolve() 10 0.2145 0.2208 0.2238 0.09% MLMG::mgVcycle_up::1 31 0.2182 0.2212 0.2231 0.09% MLCellLinOp::defineAuxData() 11 0.178 0.1966 0.2089 0.09% Amr::grid_places() 6 0.1972 0.1974 0.1974 0.08% AmrMesh::MakeNewGrids() 6 0.1968 0.1969 0.197 0.08% Castro::fill_hybrid_hydro_source() 27 0.1686 0.189 0.1967 0.08% MLMG::addInterpCorrection() 155 0.1312 0.168 0.1852 0.08% MLMG::mgVcycle_bottom 65 0.1515 0.159 0.1807 0.07% MLMG::getGradSolution() 10 0.1658 0.1703 0.1724 0.07% MLCellLinOp::compGrad() 18 0.1658 0.1703 0.1724 0.07% MLPoisson::FFlux() 150 0.0046 0.04219 0.1549 0.06% MLMG::miniCycle() 34 0.1516 0.1518 0.1523 0.06% FabArray::mult() 56 0.1138 0.1284 0.134 0.06% Castro::swap_state_time_levels() 9 0.0893 0.105 0.1325 0.05% Castro::construct_new_hybrid_source() 9 0.1113 0.125 0.1299 0.05% Castro::errorEst() 6 0.1096 0.1195 0.1252 0.05% Castro::avgDown() 4 0.12 0.1229 0.1251 0.05% Castro::avgDown(state_indx) 20 0.12 0.1228 0.1251 0.05% amrex::average_down_w_geom 20 0.1199 0.1228 0.125 0.05% AmrLevel::AmrLevel(dm) 5 0.1073 0.1174 0.1219 0.05% StateData::define() 25 0.1071 0.1172 0.1217 0.05% MLMG::ResNormInf() 54 0.07506 0.08306 0.1059 0.04% MLCellLinOp::updateSolBC() 25 0.07778 0.08547 0.09878 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08446 0.08676 0.08929 0.04% MLCellLinOp::defineBC() 11 0.08495 0.08666 0.08925 0.04% MLMG::mgVcycle_up::3 31 0.02894 0.0698 0.08349 0.03% MLMG::mgVcycle_down::2 31 0.06751 0.07107 0.07731 0.03% MLMG::MLResNormInf() 15 0.044 0.06692 0.07519 0.03% Castro::construct_old_hybrid_source() 9 0.0573 0.06401 0.06711 0.03% FabArray::norminf(mask) 21 0.05357 0.06047 0.06315 0.03% BndryData::define() 27 0.05783 0.0595 0.06197 0.03% MLMG::interpCorrection_1 17 0.0265 0.03685 0.06014 0.02% Castro::apply_tagging_restrictions() 6 0.04946 0.05426 0.05701 0.02% MLMG::mgVcycle_up::2 31 0.05166 0.05216 0.05259 0.02% MLCellLinOp::updateCorBC() 17 0.04924 0.05032 0.05211 0.02% Castro::derive() 6 0.04395 0.04753 0.04957 0.02% AmrLevel::derive() 6 0.04393 0.04751 0.04955 0.02% Castro::buildMetrics() 5 0.03674 0.0406 0.04385 0.02% FabArray::norminf() 147 0.03729 0.03982 0.04255 0.02% Gravity::swapTimeLevels() 9 0.03111 0.03495 0.03843 0.02% MLMG::mgVcycle_down::3 31 0.03116 0.03217 0.03454 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01888 0.02267 0.02903 0.01% MLMG::actualBottomSolve() 31 3.884e-05 0.007253 0.02888 0.01% MLMG::MLRhsNormInf() 10 0.02474 0.02772 0.0288 0.01% FabArrayBase::getCPC() 1440 0.02328 0.02484 0.02869 0.01% TagBoxArray::mapPRD 6 0.004166 0.01585 0.02842 0.01% MLCGSolver::bicgstab 7 0 0.007037 0.02815 0.01% Castro::FluxRegFineAdd() 9 0.02558 0.02613 0.02703 0.01% MultiFab::Multiply() 8 0.02169 0.02407 0.02504 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01955 0.02024 0.02189 0.01% MLMG::compResidual() 1 0.01916 0.02037 0.02104 0.01% MLLinOp::define() 11 0.002352 0.01004 0.021 0.01% MLLinOp::defineGrids() 11 0.002283 0.009978 0.02094 0.01% Castro::build_fine_mask() 44 0.01661 0.01778 0.01914 0.01% MLLinOp::makeSubCommunicator() 11 0.0006089 0.008226 0.01912 0.01% FabArrayBase::TheFPinfo() 25 0.002022 0.007769 0.01909 0.01% DistributionMapping::LeastUsedCPUs() 5 0.001919 0.007657 0.01897 0.01% MLMG::mgVcycle_down::4 31 0.005373 0.008793 0.01893 0.01% FPinfo::FPinfo() 3 0.00185 0.007593 0.01891 0.01% DistributionMapping::KnapSackDoIt() 3 0.001716 0.007471 0.01878 0.01% FabArrayBase::CPC::define() 416 0.01315 0.01471 0.01805 0.01% Castro::linear_to_hybrid_momentum() 1 0.01363 0.01578 0.01651 0.01% MLMG::mgVcycle_up::4 31 0.004424 0.006923 0.01428 0.01% MLPoisson::prepareForSolve() 11 0.0114 0.01168 0.01199 0.00% MLCellLinOp::prepareForSolve() 11 0.01135 0.01163 0.01194 0.00% Castro::enforce_consistent_e() 1 0.009176 0.01056 0.01105 0.00% AMRErrorTag::operator() 6 0.008975 0.009761 0.01031 0.00% average_down_faces 48 0.009004 0.009366 0.009716 0.00% Castro::apply_problem_tags() 6 0.007141 0.007806 0.008202 0.00% FabArrayBase::getFB() 1553 0.005631 0.006021 0.006628 0.00% MLCellLinOp::setLevelBC() 19 0.00368 0.004605 0.005395 0.00% amrex::Dot() 115 0 0.0009034 0.003614 0.00% FabArrayBase::FB::FB() 19 0.0025 0.002814 0.003112 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007672 0.003069 0.00% TagBoxArray::collate() 6 0.0003722 0.0008581 0.001127 0.00% OwnerMask() 6 0.000705 0.0008362 0.0008912 0.00% Castro::finalize_advance() 9 0.0003057 0.0004083 0.0004935 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003472 0.0003749 0.000402 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0003254 0.0003501 0.0003652 0.00% makeSFC 33 0.0002439 0.0002645 0.0002861 0.00% Castro::enforce_speed_limit() 87 0.000252 0.0002637 0.0002845 0.00% AmrMesh-cluster 1 0 3.574e-05 0.000143 0.00% DistributionMapping::Distribute() 35 0.0001044 0.000109 0.0001149 0.00% FabArray::RecvLayoutMask() 8 9.075e-05 9.873e-05 0.0001145 0.00% Amr::writeSmallPlotFile() 1 9.147e-05 0.0001024 0.0001132 0.00% ClusterList::intersect() 1 0 2.257e-05 9.028e-05 0.00% knapsack() 3 6.903e-05 7.195e-05 7.689e-05 0.00% Castro::create_source_corrector() 9 5.04e-05 5.85e-05 6.846e-05 0.00% BoxList::parallelComplementIn() 12 4.606e-05 5.214e-05 6.374e-05 0.00% Amr::initSubcycle() 1 5.015e-05 5.056e-05 5.164e-05 0.00% Gravity::set_mass_offset() 6 2.819e-05 3.865e-05 5.06e-05 0.00% BoxList::complementIn 12 2.129e-05 2.607e-05 3.834e-05 0.00% Castro::retry_advance_ctu() 5 2.401e-05 2.803e-05 3.387e-05 0.00% Castro::construct_new_rotation_source() 9 2.186e-05 2.429e-05 3.058e-05 0.00% Castro::construct_old_rotation_source() 9 2.091e-05 2.253e-05 2.329e-05 0.00% ClusterList::chop() 1 0 3.072e-06 1.229e-05 0.00% knapsack()swap 1 3.715e-06 4.053e-06 4.524e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (24.04-3-gd78e241d1ad4) finalized