Initializing AMReX (24.04-15-g7813bb7d6bc6)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (24.04-15-g7813bb7d6bc6) initialized Starting run at 06:57:24 UTC on 2024-04-15. Successfully read inputs file ... Castro git describe: 24.04-14-g3d76cd5a0 AMReX git describe: 24.04-15-g7813bb7d6 Microphysics git describe: 24.04-7-gd4312a6c reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.035447832 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.034471162 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.358172456 Gravity::multilevel_solve_for_new_phi() time = 1.707775519 Gravity::fill_multipole_BCs() time = 0.361343226 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.523731417 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.568522634 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.449362872e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.730069556 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.48569421 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.744679746 Gravity::multilevel_solve_for_new_phi() time = 1.796591228 Castro::construct_new_gravity() time = 1.91560194 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730185e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.396231907 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02419263439 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.80584809 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.031516791 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402028914 Gravity::multilevel_solve_for_new_phi() time = 1.951384666 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.527402872 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.398938711 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 22.19525657 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Castro::construct_old_gravity() time = 0.040629545 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.574439944e+22 ymom added: 1.475739526e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.210604997 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.904033902 on level 1 Castro::reflux() at level 0 : time = 2.494368604 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.456035431 Gravity::multilevel_solve_for_new_phi() time = 1.448061224 Castro::construct_new_gravity() time = 1.567484524 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616014e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 2.168843697 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029959228 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764863347e+18 ymom added: -1.360987807e+19 zmom added: -8.219114448e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.142594707 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02201566179 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 51.44452806 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031790299 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.399980141 Gravity::multilevel_solve_for_new_phi() time = 0.538379741 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.420628055 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.412211562 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.91548741 on level 0 Castro::construct_old_gravity() time = 0.035027831 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.44217652e+22 ymom added: 4.722366483e+21 zmom added: 7.103293505e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.202194637 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.873641462 on level 1 Castro::reflux() at level 0 : time = 2.441427304 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.472967315 Gravity::multilevel_solve_for_new_phi() time = 1.05464171 Castro::construct_new_gravity() time = 1.172667873 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.256952045 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.030234114 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.816166528e+17 ymom added: 2.350879005e+19 zmom added: 9.539392308e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.149442548 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02235817963 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 48.98617445 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.037531943 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.404136645 Gravity::multilevel_solve_for_new_phi() time = 0.542850931 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.374032343 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.372728754 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 22.45206209 on level 0 Castro::construct_old_gravity() time = 0.035925125 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.198276178e+22 ymom added: 1.180591621e+22 zmom added: 8.023757211e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.204646254 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.882459243 on level 1 Castro::reflux() at level 0 : time = 2.43822069 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.466119035 Gravity::multilevel_solve_for_new_phi() time = 1.028930177 Castro::construct_new_gravity() time = 1.145627526 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 2.085169129 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.030020951 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598346965e+18 ymom added: -2.567952508e+19 zmom added: -1.336958063e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.14247308 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02222258602 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 49.29340337 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.032622284 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.400385989 Gravity::multilevel_solve_for_new_phi() time = 0.534868099 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.421160481 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.439724741 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 22.08220597 on level 0 Castro::construct_old_gravity() time = 0.033294736 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013261987e+23 ymom added: 1.829917012e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.20401672 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.916212664 on level 1 Castro::reflux() at level 0 : time = 2.4367749 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.455163098 Gravity::multilevel_solve_for_new_phi() time = 1.023745514 Castro::construct_new_gravity() time = 1.140528612 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171514e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 2.129702786 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.030088233 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727203076e+18 ymom added: -2.54093091e+19 zmom added: -1.252338631e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.146459595 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02243372011 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 48.70620697 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.60240499 seconds Ending run at 07:01:33 UTC on 2024-04-15. Run time = 249.5334079 Run time without initialization = 237.8406928 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 249.5 ... 249.5 ... 249.5 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 96.4442 106.4250 110.6636 44.35% Castro::computeTemp() 88 14.74 16.56 17.38 6.97% amrex::Copy() 679 12.77 14.37 15.16 6.07% check_for_negative_density() 9 0.3652 4.625 14.59 5.85% Castro::normalize_species() 87 10.15 11.77 12.65 5.07% MultiFab::AddProduct() 4205 10.44 11.72 12.19 4.89% FabArray::ParallelCopy_finish() 830 1.592 5.173 12.08 4.84% Castro::estTimeStep() 26 4.176 6.242 10.04 4.02% Castro::reset_internal_energy(Fab) 1476 8.494 9.403 9.967 3.99% FabArray::setVal() 5070 7.23 8.129 8.536 3.42% MultiFab::Subtract() 175 7.042 7.889 8.205 3.29% Castro::reset_internal_energy(MultiFab) 88 3.57 4.331 4.923 1.97% Gravity::fill_multipole_BCs() 11 4.226 4.502 4.851 1.94% FabArray::sum() 4205 3.454 3.87 4.027 1.61% FluxRegister::Reflux() 24 3.112 3.509 3.844 1.54% FabArray::ParallelCopy_nowait() 830 2.122 2.326 2.762 1.11% Castro::enforce_min_density() 87 1.684 2.03 2.511 1.01% MLPoisson::Fsmooth() 1376 2.115 2.372 2.469 0.99% Castro::wd_update() 6 0.5377 1.023 2.063 0.83% Castro::reflux() 4 1.604 1.776 1.923 0.77% Castro::FluxRegCrseInit 9 1.585 1.771 1.867 0.75% FillPatchInterp(MF) 30 1.255 1.348 1.461 0.59% Castro::construct_old_gravity_source() 18 1.09 1.218 1.328 0.53% Castro::construct_new_gravity_source() 9 1.018 1.149 1.274 0.51% amrex::Add() 136 1.039 1.172 1.235 0.50% Amr::InitAmr() 1 1.182 1.182 1.182 0.47% Castro::initMFs() 5 1.124 1.151 1.174 0.47% Castro::problem_post_timestep() 10 0.8721 0.9745 1.095 0.44% MLPoisson::Fapply() 318 0.8909 0.9969 1.048 0.42% Castro::volInBoundary() 42 0.6578 0.8748 1.023 0.41% FillPatchIterator::Initialize 62 0.855 0.9208 0.987 0.40% FillBoundary_finish() 1553 0.2919 0.4761 0.9215 0.37% FabArray::Saxpy() 182 0.7116 0.7998 0.8462 0.34% VisMF::Write(FabArray) 3 0.6336 0.7003 0.7558 0.30% StateData::FillBoundary(geom) 50 0.5831 0.6101 0.6347 0.25% Castro::construct_new_source() 63 0.4913 0.5499 0.576 0.23% MultiFab::max() 18 0.02417 0.1913 0.5537 0.22% MLCellLinOp::applyBC() 1742 0.3672 0.4447 0.4977 0.20% MultiFab::contains_nan() 18 0.3906 0.4301 0.4465 0.18% FillBoundary_nowait() 1553 0.3759 0.4163 0.4445 0.18% FabArray::Xpay() 283 0.3478 0.3893 0.4075 0.16% Castro::initialize_do_advance() 9 0.258 0.2863 0.3112 0.12% FabArray::setDomainBndry() 92 0.2217 0.2427 0.2673 0.11% Amr::writePlotFile() 2 0.08841 0.1534 0.252 0.10% Castro::construct_old_source() 63 0.2122 0.2332 0.2497 0.10% amrex::average_down 233 0.1973 0.2045 0.208 0.08% Castro::fill_hybrid_hydro_source() 27 0.1648 0.1841 0.1928 0.08% MLCellLinOp::defineAuxData() 11 0.1529 0.169 0.1778 0.07% Castro::initData() 1 0.1185 0.1437 0.1684 0.07% MLPoisson::FFlux() 150 0.004811 0.04177 0.1524 0.06% MLCellLinOp::compGrad() 18 0.1254 0.1426 0.1521 0.06% StateData::define() 25 0.1085 0.124 0.1456 0.06% FabArray::mult() 56 0.1176 0.1328 0.1413 0.06% Castro::swap_state_time_levels() 9 0.08429 0.1033 0.1374 0.06% MLMG::addInterpCorrection() 155 0.1094 0.1223 0.1274 0.05% Castro::evaluate_source_change() 193 0.108 0.1162 0.1214 0.05% Castro::initialize_advance() 9 0.07348 0.08637 0.1103 0.04% Gravity::get_new_grav_vector() 10 0.08679 0.09417 0.09913 0.04% Gravity::get_old_grav_vector() 9 0.07557 0.08573 0.09072 0.04% MLMG::prepareForSolve() 10 0.06629 0.07415 0.08422 0.03% MLMG::solve() 10 0.0007293 0.04341 0.06905 0.03% FabArray::norminf(mask) 21 0.05558 0.06338 0.06861 0.03% MLCellLinOp::updateSolBC() 25 0.06751 0.06788 0.06821 0.03% Castro::apply_tagging_restrictions() 6 0.04812 0.0543 0.05728 0.02% AmrLevel::FillPatch() 62 0.0509 0.0531 0.0571 0.02% AmrMesh::MakeNewGrids() 6 0.04651 0.049 0.05392 0.02% Castro::do_old_sources() 9 0.0006447 0.01247 0.04693 0.02% MLCellLinOp::updateCorBC() 17 0.04579 0.04603 0.04636 0.02% BndryData::define() 27 0.0427 0.04461 0.04579 0.02% FabArray::norminf() 147 0.03755 0.03983 0.04293 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.03004 0.03507 0.04272 0.02% MLMG::ResNormInf() 54 0.001888 0.0125 0.04107 0.02% MLMG::MLResNormInf() 15 0.0001133 0.02607 0.03775 0.02% MLCellLinOp::reflux() 30 0.004513 0.0123 0.0352 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.0209 0.02397 0.03106 0.01% Castro::FluxRegFineAdd() 9 0.02619 0.02802 0.02893 0.01% MLCellLinOp::defineBC() 11 0.02642 0.02663 0.02685 0.01% Castro::enforce_consistent_e() 1 0.009265 0.01714 0.02562 0.01% MultiFab::Multiply() 8 0.02167 0.02432 0.02531 0.01% Castro::build_fine_mask() 44 0.01731 0.01882 0.02177 0.01% Castro::Castro() 5 0.01428 0.01915 0.02175 0.01% StateDataPhysBCFunct::() 95 0.002307 0.008294 0.02147 0.01% MLLinOp::makeSubCommunicator() 11 0.001009 0.01132 0.02099 0.01% Castro::buildMetrics() 5 0.01447 0.01707 0.02019 0.01% DistributionMapping::LeastUsedCPUs() 5 0.00498 0.01125 0.01924 0.01% MLMG::interpCorrection_1 17 0.01885 0.01893 0.01906 0.01% FabArrayBase::CPC::define() 416 0.0132 0.01504 0.01827 0.01% FillPatchTwoLevels 31 0.01252 0.01427 0.01786 0.01% Castro::linear_to_hybrid_momentum() 1 0.01412 0.01566 0.01624 0.01% MLCellLinOp::prepareForSolve() 11 0.0118 0.0121 0.01258 0.01% Gravity::actual_solve_with_mlmg() 11 0.01002 0.01089 0.01148 0.00% FabArrayBase::getCPC() 1440 0.009802 0.01084 0.01141 0.00% AMRErrorTag::operator() 6 0.008907 0.01015 0.01089 0.00% Gravity::actual_multilevel_solve() 10 0.008336 0.009653 0.01085 0.00% Gravity::update_max_rhs() 10 0.008269 0.009164 0.01024 0.00% Castro::apply_problem_tags() 6 0.007074 0.007987 0.008589 0.00% Castro::finalize_do_advance() 9 0.006911 0.007739 0.008257 0.00% average_down_faces 48 0.005812 0.006038 0.006196 0.00% MLCellLinOp::setLevelBC() 19 0.004048 0.005058 0.005949 0.00% AmrLevel::derive() 6 0.003895 0.004848 0.005572 0.00% FabArray::FillBoundary() 1553 0.004356 0.004628 0.004965 0.00% MLCellLinOp::smooth() 688 0.003929 0.004125 0.004236 0.00% amrex::Dot() 115 0 0.001025 0.004099 0.00% MLCGSolver::bicgstab 7 0 0.0009878 0.003951 0.00% FabArrayBase::getFB() 1553 0.003014 0.003212 0.003563 0.00% FabArray::ParallelCopy() 830 0.003047 0.003166 0.003281 0.00% FabArrayBase::FB::FB() 19 0.002496 0.002856 0.003154 0.00% Castro::construct_old_gravity() 9 0.0003809 0.001065 0.002796 0.00% Gravity::test_composite_phi() 1 0.002116 0.002462 0.00265 0.00% main() 1 0.001496 0.00169 0.001855 0.00% Amr::regrid() 5 0.001036 0.001323 0.001846 0.00% MLLinOp::defineGrids() 11 0.001445 0.00151 0.001615 0.00% MLCellLinOp::apply() 318 0.00091 0.001112 0.001574 0.00% AmrLevel::FillCoarsePatch() 5 0.0009134 0.00112 0.001441 0.00% TagBoxArray::collate() 6 0.0005707 0.001056 0.001427 0.00% Castro::clean_state() 87 0.001301 0.001363 0.001396 0.00% Castro::do_new_sources() 9 0.000603 0.0008776 0.001346 0.00% Castro::construct_new_gravity() 9 0.0004548 0.0007192 0.001311 0.00% Amr::coarseTimeStep() 5 0.0006351 0.001131 0.001302 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002901 0.00116 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009299 0.0009697 0.001011 0.00% OwnerMask() 6 0.0007522 0.0008636 0.0009394 0.00% MLCellLinOp::correctionResidual() 189 0.0007072 0.0007304 0.0007636 0.00% MLMG::mgVcycle() 65 0.0006017 0.0006475 0.0006947 0.00% FillPatchSingleLevel 92 0.0005617 0.0006 0.0006233 0.00% Amr::grid_places() 6 0.0004793 0.0005293 0.0005927 0.00% Castro::do_advance_ctu() 5 0.0005188 0.0005489 0.0005647 0.00% Castro::finalize_advance() 9 0.0003826 0.0004552 0.0005299 0.00% TagBoxArray::mapPRD 6 0.0004631 0.0004882 0.0005084 0.00% FillPatchIterator::FillFromLevel0() 31 0.0004338 0.0004471 0.0004577 0.00% MLMG:computeResOfCorrection() 155 0.000398 0.0004116 0.0004227 0.00% Castro::subcycle_advance_ctu() 5 0.0001035 0.0001929 0.0004203 0.00% MLMG::oneIter() 31 0.0003647 0.0003814 0.0003917 0.00% MLCellLinOp::solutionResidual() 72 0.0003525 0.0003695 0.0003899 0.00% MLMG::actualBottomSolve() 31 4.343e-05 0.0001265 0.0003716 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0003111 0.0003244 0.0003412 0.00% MLMG::mgVcycle_down::0 31 0.0002978 0.0003025 0.0003094 0.00% Castro::enforce_speed_limit() 87 0.0002363 0.0002656 0.0002902 0.00% Castro::computeNewDt() 8 0.0001313 0.0001744 0.0002866 0.00% Gravity::solve_phi_with_mlmg() 11 0.00024 0.0002579 0.0002747 0.00% Amr::timeStep() 9 0.0002008 0.0002319 0.000257 0.00% MLMG::mgVcycle_down::1 31 0.0002505 0.0002541 0.0002565 0.00% Amr::defBaseLevel() 1 0.0002278 0.0002418 0.0002556 0.00% Castro::post_regrid() 10 0.0002273 0.0002341 0.0002487 0.00% MLMG::mgVcycle_bottom 65 0.0002196 0.0002276 0.0002428 0.00% MLMG::mgVcycle_down::3 31 0.0001921 0.0002023 0.0002165 0.00% MLMG::mgVcycle_down::2 31 0.000202 0.0002065 0.0002137 0.00% AmrLevel::AmrLevel(dm) 5 0.000184 0.0001976 0.0002055 0.00% Amr::FinalizeInit() 1 3.074e-05 7.663e-05 0.0002036 0.00% MLMG::mgVcycle_up::0 31 0.0001845 0.0001932 0.0002006 0.00% FabArrayBase::TheFPinfo() 25 0.0001732 0.0001804 0.000198 0.00% makeSFC 33 0.0001648 0.0001776 0.000196 0.00% MLMG::mgVcycle_down::4 31 0.0001697 0.0001789 0.0001871 0.00% Castro::check_for_nan() 18 0.0001407 0.0001538 0.0001616 0.00% MLMG::mgVcycle_up::1 31 0.0001428 0.0001459 0.0001475 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001423 0.0001445 0.0001474 0.00% MLMG::mgVcycle_up::2 31 0.0001343 0.0001369 0.0001411 0.00% FabArray::RecvLayoutMask() 8 0.0001034 0.000116 0.0001404 0.00% Castro::expand_state() 13 0.000112 0.0001262 0.0001397 0.00% Castro::errorEst() 6 0.0001116 0.0001225 0.0001365 0.00% MLMG::mgVcycle_up::3 31 0.0001231 0.0001296 0.0001357 0.00% FPinfo::FPinfo() 3 0.0001075 0.0001204 0.0001357 0.00% Castro::react_state() 18 0.0001094 0.0001202 0.0001294 0.00% MLMG::mgVcycle_up::4 31 0.0001121 0.0001204 0.0001279 0.00% Amr::writeSmallPlotFile() 1 9.335e-05 0.0001107 0.0001276 0.00% Castro::advance() 5 0.000103 0.0001134 0.0001262 0.00% Castro::apply_source_to_state() 18 9.88e-05 0.0001149 0.000122 0.00% DistributionMapping::Distribute() 35 0.00011 0.0001153 0.0001193 0.00% MLPoisson::define() 11 0.0001113 0.0001134 0.0001173 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001046 0.0001093 0.0001154 0.00% Gravity::swapTimeLevels() 9 8.175e-05 9.629e-05 0.00011 0.00% MLMG::computeResidual() 31 8.665e-05 9.317e-05 9.814e-05 0.00% Castro::post_timestep() 9 8.369e-05 9.032e-05 9.745e-05 0.00% ClusterList::intersect() 1 0 2.276e-05 9.105e-05 0.00% MLMG::computeMLResidual() 15 7.967e-05 8.609e-05 8.937e-05 0.00% Castro::avgDown(state_indx) 20 6.851e-05 7.633e-05 8.442e-05 0.00% MLLinOp::define() 11 6.644e-05 7.39e-05 7.807e-05 0.00% MLMG::miniCycle() 34 6.802e-05 7.305e-05 7.778e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.571e-05 6.955e-05 7.35e-05 0.00% MLMG::getGradSolution() 10 6.505e-05 6.742e-05 7.337e-05 0.00% knapsack() 3 6.41e-05 6.851e-05 7.213e-05 0.00% Castro::create_source_corrector() 9 5.57e-05 6.387e-05 7.104e-05 0.00% amrex::average_down_w_geom 20 4.865e-05 5.864e-05 6.466e-05 0.00% Gravity::set_mass_offset() 6 4.356e-05 5.22e-05 5.863e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 3.968e-05 4.638e-05 5.795e-05 0.00% Castro::construct_new_hybrid_source() 9 4.784e-05 5.171e-05 5.499e-05 0.00% Amr::initSubcycle() 1 5.18e-05 5.205e-05 5.237e-05 0.00% Castro::init(old) 3 4.291e-05 4.868e-05 5.224e-05 0.00% MLPoisson::prepareForSolve() 11 4.919e-05 5.057e-05 5.213e-05 0.00% BoxList::parallelComplementIn() 12 3.647e-05 4.332e-05 4.891e-05 0.00% AmrMesh-cluster 1 0 1.156e-05 4.626e-05 0.00% Castro::construct_old_hybrid_source() 9 3.816e-05 3.995e-05 4.097e-05 0.00% Castro::avgDown() 4 3.109e-05 3.509e-05 3.773e-05 0.00% MLMG::MLRhsNormInf() 10 3.047e-05 3.368e-05 3.667e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.837e-05 3.266e-05 3.626e-05 0.00% Castro::post_init() 1 2.522e-05 2.909e-05 3.483e-05 0.00% MLMG::compResidual() 1 2.695e-05 2.98e-05 3.351e-05 0.00% Amr::InitializeInit() 1 7.48e-06 1.476e-05 3.112e-05 0.00% Castro::retry_advance_ctu() 5 2.225e-05 2.685e-05 2.874e-05 0.00% Castro::derive() 6 2.24e-05 2.377e-05 2.704e-05 0.00% Castro::construct_new_rotation_source() 9 2.293e-05 2.486e-05 2.689e-05 0.00% Castro::construct_old_rotation_source() 9 2.455e-05 2.58e-05 2.675e-05 0.00% Castro::computeInitialDt() 2 2.089e-05 2.293e-05 2.641e-05 0.00% BoxList::complementIn 12 2.351e-05 2.476e-05 2.585e-05 0.00% Amr::bldFineLevels() 1 2.201e-05 2.315e-05 2.475e-05 0.00% ClusterList::chop() 1 0 5.458e-06 2.183e-05 0.00% Castro::init() 1 1.491e-05 1.62e-05 1.719e-05 0.00% Amr::initialInit() 1 4.652e-06 8.509e-06 1.682e-05 0.00% Amr::init() 1 6.416e-06 7.721e-06 1.051e-05 0.00% knapsack()swap 1 3.507e-06 3.878e-06 4.502e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 249.5 249.5 249.5 100.00% Amr::coarseTimeStep() 5 237.2 237.2 237.2 95.07% Amr::timeStep() 9 235.8 235.8 235.8 94.51% Castro::advance() 5 207.4 207.4 207.4 83.10% Castro::subcycle_advance_ctu() 5 198.4 198.9 199.9 80.11% Castro::do_advance_ctu() 5 198.4 198.9 199.9 80.11% Castro::construct_ctu_hydro_source() 9 121.2 121.5 121.7 48.75% Castro::clean_state() 87 72.12 81.66 86.45 34.65% Castro::computeTemp() 88 45.77 51.55 54.55 21.86% Castro::reset_internal_energy(MultiFab) 88 31.04 34.98 37.17 14.89% Castro::evaluate_source_change() 193 17.71 19.86 20.66 8.28% Castro::enforce_min_density() 87 16.96 19.24 20.42 8.18% Castro::post_regrid() 10 15.88 17.56 18.3 7.33% Amr::regrid() 5 15.84 17.36 17.99 7.21% Castro::initialize_do_advance() 9 15.37 15.53 15.65 6.27% amrex::Copy() 679 12.77 14.37 15.16 6.07% AmrLevel::FillPatch() 62 8.381 10.36 14.97 6.00% check_for_negative_density() 9 0.3652 4.625 14.59 5.85% FabArray::ParallelCopy() 830 4.423 7.565 14.37 5.76% FillPatchIterator::Initialize 62 6.684 8.733 13.51 5.41% Castro::do_old_sources() 9 11.38 11.89 13.17 5.28% Castro::normalize_species() 87 10.15 11.77 12.65 5.07% MultiFab::AddProduct() 4205 10.44 11.72 12.19 4.89% FabArray::ParallelCopy_finish() 830 1.592 5.173 12.08 4.84% Gravity::solve_phi_with_mlmg() 11 11.3 11.58 11.92 4.78% Gravity::multilevel_solve_for_new_phi() 10 11.03 11.29 11.62 4.66% Castro::do_new_sources() 9 9.675 10.84 11.62 4.66% Gravity::actual_multilevel_solve() 10 10.99 11.26 11.59 4.65% FillPatchSingleLevel 92 4.319 6.439 11.33 4.54% FillPatchIterator::FillFromLevel0() 31 3.923 5.997 10.83 4.34% Castro::post_timestep() 9 10.82 10.82 10.82 4.34% Amr::init() 1 10.51 10.51 10.51 4.21% Castro::estTimeStep() 26 4.176 6.242 10.04 4.02% Amr::initialInit() 1 9.986 9.986 9.987 4.00% Castro::reset_internal_energy(Fab) 1476 8.494 9.403 9.967 3.99% Castro::reflux() 4 9.605 9.715 9.811 3.93% Castro::initialize_advance() 9 7.453 8.445 8.984 3.60% FabArray::setVal() 5070 7.23 8.129 8.536 3.42% MultiFab::Subtract() 175 7.042 7.889 8.205 3.29% Castro::expand_state() 13 5.544 5.985 7.089 2.84% Castro::construct_new_gravity() 9 6.493 6.739 7.062 2.83% Gravity::actual_solve_with_mlmg() 11 6.969 6.976 6.982 2.80% MLMG::solve() 10 6.437 6.455 6.479 2.60% Amr::FinalizeInit() 1 5.654 5.928 6.409 2.57% MLMG::oneIter() 31 5.209 5.228 5.235 2.10% Gravity::fill_multipole_BCs() 11 4.31 4.58 4.921 1.97% FluxRegister::Reflux() 24 4.718 4.778 4.88 1.96% Castro::finalize_do_advance() 9 1.388 2.607 4.599 1.84% Castro::problem_post_timestep() 10 3.305 3.772 4.594 1.84% MLMG::mgVcycle() 65 4.356 4.387 4.398 1.76% Amr::InitializeInit() 1 3.578 4.058 4.333 1.74% Amr::defBaseLevel() 1 3.577 4.058 4.333 1.74% Castro::computeNewDt() 8 2.284 2.855 4.176 1.67% FabArray::sum() 4205 3.454 3.87 4.027 1.61% MLCellLinOp::smooth() 688 3.325 3.334 3.35 1.34% Castro::FluxRegCrseInit 9 1.943 2.355 3.141 1.26% Castro::post_init() 1 2.453 2.619 2.978 1.19% Castro::initData() 1 2.195 2.613 2.887 1.16% FabArray::ParallelCopy_nowait() 830 2.187 2.388 2.828 1.13% Castro::wd_update() 6 1.43 1.898 2.721 1.09% Castro::construct_new_source() 63 2.105 2.314 2.499 1.00% MLPoisson::Fsmooth() 1376 2.115 2.372 2.469 0.99% Castro::construct_old_gravity() 9 0.6903 1.309 2.454 0.98% Gravity::get_old_grav_vector() 9 0.6897 1.308 2.454 0.98% MLMG::mgVcycle_down::0 31 1.924 1.988 2.014 0.81% MLCellLinOp::applyBC() 1742 1.207 1.348 1.744 0.70% FillPatchIterator::FillFromTwoLevels() 31 1.512 1.573 1.616 0.65% FillPatchTwoLevels 31 1.511 1.573 1.615 0.65% Amr::bldFineLevels() 1 1.252 1.447 1.524 0.61% Castro::Castro() 5 1.354 1.403 1.462 0.59% FillPatchInterp(MF) 30 1.255 1.348 1.461 0.59% MLMG::mgVcycle_up::0 31 1.397 1.424 1.433 0.57% FabArray::FillBoundary() 1553 0.717 0.9032 1.377 0.55% MLCellLinOp::apply() 318 1.285 1.312 1.339 0.54% Castro::construct_old_gravity_source() 18 1.09 1.218 1.328 0.53% Castro::construct_new_gravity_source() 9 1.018 1.149 1.274 0.51% Castro::computeInitialDt() 2 0.5129 0.7877 1.268 0.51% amrex::Add() 136 1.039 1.172 1.235 0.50% Amr::InitAmr() 1 1.182 1.182 1.182 0.47% Castro::initMFs() 5 1.13 1.157 1.18 0.47% Amr::writePlotFile() 2 1.127 1.127 1.127 0.45% Castro::construct_old_source() 63 0.9743 1.056 1.089 0.44% MLCellLinOp::solutionResidual() 72 0.9759 1.029 1.049 0.42% MLPoisson::Fapply() 318 0.8909 0.9969 1.048 0.42% Castro::volInBoundary() 42 0.6579 0.8749 1.023 0.41% FillBoundary_finish() 1553 0.2919 0.4761 0.9215 0.37% FabArray::Saxpy() 182 0.7116 0.7998 0.8462 0.34% MLCellLinOp::correctionResidual() 189 0.8113 0.8166 0.8249 0.33% Gravity::get_new_grav_vector() 10 0.7956 0.8052 0.8195 0.33% VisMF::Write(FabArray) 3 0.6336 0.7003 0.7558 0.30% MLMG:computeResOfCorrection() 155 0.6581 0.6883 0.7085 0.28% StateDataPhysBCFunct::() 95 0.6018 0.6184 0.6413 0.26% StateData::FillBoundary(geom) 50 0.5831 0.6101 0.6347 0.25% Gravity::update_max_rhs() 10 0.08528 0.2501 0.6073 0.24% MLMG::computeResWithCrseSolFineCor() 17 0.5885 0.5895 0.5914 0.24% MultiFab::max() 18 0.02417 0.1913 0.5537 0.22% MLMG::computeMLResidual() 15 0.4288 0.4503 0.4995 0.20% Castro::init() 1 0.4797 0.4842 0.4877 0.20% AmrLevel::FillCoarsePatch() 5 0.4797 0.4842 0.4877 0.20% FillBoundary_nowait() 1553 0.3814 0.4224 0.4507 0.18% Castro::check_for_nan() 18 0.3908 0.4302 0.4467 0.18% MultiFab::contains_nan() 18 0.3906 0.4301 0.4465 0.18% amrex::average_down 233 0.2751 0.3791 0.4392 0.18% FabArray::Xpay() 283 0.3478 0.3893 0.4075 0.16% Gravity::test_composite_phi() 1 0.4023 0.403 0.4048 0.16% Castro::init(old) 3 0.3781 0.3907 0.3957 0.16% MLMG::mgVcycle_down::1 31 0.3137 0.3368 0.3955 0.16% MLCellLinOp::reflux() 30 0.1865 0.2387 0.3753 0.15% MLMG::computeResidual() 31 0.3473 0.3547 0.3583 0.14% Castro::apply_source_to_state() 18 0.2849 0.3201 0.3388 0.14% MLPoisson::define() 11 0.2892 0.3021 0.316 0.13% Castro::react_state() 18 0.2274 0.2491 0.2713 0.11% FabArray::setDomainBndry() 92 0.2217 0.2427 0.2673 0.11% MLMG::prepareForSolve() 10 0.2355 0.2403 0.2458 0.10% MLMG::mgVcycle_up::1 31 0.2212 0.2245 0.2262 0.09% MLCellLinOp::defineAuxData() 11 0.1792 0.1992 0.2139 0.09% Amr::grid_places() 6 0.2033 0.2034 0.2035 0.08% AmrMesh::MakeNewGrids() 6 0.2028 0.2029 0.203 0.08% Castro::fill_hybrid_hydro_source() 27 0.1648 0.1841 0.1928 0.08% MLMG::mgVcycle_bottom 65 0.159 0.1677 0.1913 0.08% MLMG::addInterpCorrection() 155 0.133 0.1705 0.1886 0.08% MLMG::getGradSolution() 10 0.1798 0.1836 0.1857 0.07% MLCellLinOp::compGrad() 18 0.1798 0.1836 0.1856 0.07% MLMG::miniCycle() 34 0.159 0.1601 0.1608 0.06% MLPoisson::FFlux() 150 0.004811 0.04177 0.1524 0.06% AmrLevel::AmrLevel(dm) 5 0.1087 0.1242 0.1458 0.06% StateData::define() 25 0.1085 0.124 0.1456 0.06% FabArray::mult() 56 0.1176 0.1328 0.1413 0.06% Castro::swap_state_time_levels() 9 0.08429 0.1033 0.1374 0.06% Castro::avgDown() 4 0.1248 0.128 0.1349 0.05% Castro::avgDown(state_indx) 20 0.1248 0.128 0.1348 0.05% amrex::average_down_w_geom 20 0.1247 0.1279 0.1347 0.05% Castro::construct_new_hybrid_source() 9 0.1106 0.1232 0.1298 0.05% Castro::errorEst() 6 0.1053 0.1198 0.1267 0.05% MLCellLinOp::updateSolBC() 25 0.07994 0.09501 0.1098 0.04% MLMG::ResNormInf() 54 0.07464 0.08387 0.1042 0.04% MLCellLinOp::defineBC() 11 0.08675 0.08953 0.09328 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08589 0.08727 0.08976 0.04% MLMG::mgVcycle_up::3 31 0.02627 0.06775 0.08235 0.03% MLMG::mgVcycle_down::2 31 0.07032 0.0751 0.08092 0.03% MLMG::MLResNormInf() 15 0.04908 0.07057 0.07835 0.03% FabArray::norminf(mask) 21 0.05558 0.06338 0.06861 0.03% MLMG::interpCorrection_1 17 0.02766 0.04083 0.06625 0.03% BndryData::define() 27 0.05877 0.0616 0.06508 0.03% Castro::construct_old_hybrid_source() 9 0.05432 0.06097 0.0637 0.03% Castro::apply_tagging_restrictions() 6 0.04812 0.0543 0.05728 0.02% MLMG::mgVcycle_up::2 31 0.05285 0.05341 0.05391 0.02% MLCellLinOp::updateCorBC() 17 0.0499 0.05077 0.05284 0.02% Castro::derive() 6 0.04105 0.04728 0.0498 0.02% AmrLevel::derive() 6 0.04103 0.04726 0.04978 0.02% Castro::buildMetrics() 5 0.03336 0.0411 0.04855 0.02% FabArray::norminf() 147 0.03755 0.03983 0.04293 0.02% TagBoxArray::mapPRD 6 0.01353 0.02157 0.03948 0.02% Gravity::swapTimeLevels() 9 0.03306 0.03576 0.03926 0.02% MLMG::mgVcycle_down::3 31 0.03179 0.0328 0.03538 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.0209 0.02397 0.03106 0.01% MLMG::actualBottomSolve() 31 4.343e-05 0.007737 0.03081 0.01% MLCGSolver::bicgstab 7 0 0.007509 0.03003 0.01% MLMG::MLRhsNormInf() 10 0.0254 0.02852 0.02985 0.01% FabArrayBase::getCPC() 1440 0.023 0.02588 0.02968 0.01% Castro::FluxRegFineAdd() 9 0.02619 0.02802 0.02893 0.01% Castro::enforce_consistent_e() 1 0.009265 0.01714 0.02562 0.01% MultiFab::Multiply() 8 0.02167 0.02432 0.02531 0.01% MLLinOp::define() 11 0.002993 0.0133 0.0231 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.0209 0.02171 0.02307 0.01% MLLinOp::defineGrids() 11 0.002918 0.01322 0.02302 0.01% Castro::build_fine_mask() 44 0.01731 0.01882 0.02177 0.01% MLLinOp::makeSubCommunicator() 11 0.001009 0.01132 0.02099 0.01% MLMG::compResidual() 1 0.01767 0.01955 0.02035 0.01% FabArrayBase::TheFPinfo() 25 0.005328 0.01157 0.01957 0.01% FPinfo::FPinfo() 3 0.005154 0.01139 0.01939 0.01% DistributionMapping::KnapSackDoIt() 3 0.005039 0.01127 0.01927 0.01% DistributionMapping::LeastUsedCPUs() 5 0.00498 0.01125 0.01924 0.01% MLMG::mgVcycle_down::4 31 0.005856 0.009434 0.01923 0.01% FabArrayBase::CPC::define() 416 0.0132 0.01504 0.01827 0.01% Castro::linear_to_hybrid_momentum() 1 0.01412 0.01566 0.01624 0.01% MLMG::mgVcycle_up::4 31 0.004824 0.007304 0.01419 0.01% MLPoisson::prepareForSolve() 11 0.01185 0.01215 0.01263 0.01% MLCellLinOp::prepareForSolve() 11 0.0118 0.0121 0.01258 0.01% AMRErrorTag::operator() 6 0.008907 0.01015 0.01089 0.00% average_down_faces 48 0.009712 0.01012 0.01039 0.00% Castro::apply_problem_tags() 6 0.007074 0.007987 0.008589 0.00% FabArrayBase::getFB() 1553 0.00551 0.006068 0.006602 0.00% MLCellLinOp::setLevelBC() 19 0.004048 0.005058 0.005949 0.00% amrex::Dot() 115 0 0.001025 0.004099 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0008408 0.003363 0.00% FabArrayBase::FB::FB() 19 0.002496 0.002856 0.003154 0.00% TagBoxArray::collate() 6 0.0005707 0.001056 0.001427 0.00% OwnerMask() 6 0.0007522 0.0008636 0.0009394 0.00% Castro::finalize_advance() 9 0.0003826 0.0004552 0.0005299 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003701 0.0003914 0.000419 0.00% makeSFC 33 0.0002655 0.0002821 0.0003035 0.00% Castro::enforce_speed_limit() 87 0.0002363 0.0002656 0.0002902 0.00% AmrMesh-cluster 1 0 3.978e-05 0.0001591 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001095 0.0001375 0.0001518 0.00% FabArray::RecvLayoutMask() 8 0.0001034 0.000116 0.0001404 0.00% Amr::writeSmallPlotFile() 1 9.335e-05 0.0001107 0.0001276 0.00% DistributionMapping::Distribute() 35 0.00011 0.0001153 0.0001193 0.00% ClusterList::intersect() 1 0 2.276e-05 9.105e-05 0.00% knapsack() 3 6.762e-05 7.239e-05 7.612e-05 0.00% BoxList::parallelComplementIn() 12 6.232e-05 6.807e-05 7.242e-05 0.00% Castro::create_source_corrector() 9 5.57e-05 6.387e-05 7.104e-05 0.00% Gravity::set_mass_offset() 6 4.356e-05 5.22e-05 5.863e-05 0.00% Amr::initSubcycle() 1 5.18e-05 5.205e-05 5.237e-05 0.00% Castro::retry_advance_ctu() 5 2.225e-05 2.685e-05 2.874e-05 0.00% Castro::construct_new_rotation_source() 9 2.293e-05 2.486e-05 2.689e-05 0.00% Castro::construct_old_rotation_source() 9 2.455e-05 2.58e-05 2.675e-05 0.00% BoxList::complementIn 12 2.351e-05 2.476e-05 2.585e-05 0.00% ClusterList::chop() 1 0 5.458e-06 2.183e-05 0.00% knapsack()swap 1 3.507e-06 3.878e-06 4.502e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (24.04-15-g7813bb7d6bc6) finalized