Initializing AMReX (24.04-15-g7813bb7d6bc6)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (24.04-15-g7813bb7d6bc6) initialized Starting run at 07:02:17 UTC on 2024-04-12. Successfully read inputs file ... Castro git describe: 24.04-14-g3d76cd5a0 AMReX git describe: 24.04-15-g7813bb7d6 Microphysics git describe: 24.04-7-gd4312a6c reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.034616579 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031486804 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.370091135 Gravity::multilevel_solve_for_new_phi() time = 1.767548849 Gravity::fill_multipole_BCs() time = 0.369437988 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.55364992 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.554459828 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.660996462 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.01428756 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.618815499 Gravity::multilevel_solve_for_new_phi() time = 1.658085157 Castro::construct_new_gravity() time = 1.774019893 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207933e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.347745519 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02458627434 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.13016676 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032133032 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.40585577 Gravity::multilevel_solve_for_new_phi() time = 1.936341334 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.495036863 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.327992609 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.60398935 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Castro::construct_old_gravity() time = 0.031864724 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.574439944e+22 ymom added: 1.475739526e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.195909984 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.854942126 on level 1 Castro::reflux() at level 0 : time = 2.35289147 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.46470533 Gravity::multilevel_solve_for_new_phi() time = 1.391103974 Castro::construct_new_gravity() time = 1.502839927 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.989025436 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.031581194 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764827561e+18 ymom added: -1.361438167e+19 zmom added: -8.219114448e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.176241339 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02264178102 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 50.11103604 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.032270184 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.404404216 Gravity::multilevel_solve_for_new_phi() time = 0.534808219 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.507238671 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.417661694 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.97704574 on level 0 Castro::construct_old_gravity() time = 0.033340283 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.443767416e+22 ymom added: 4.427218578e+21 zmom added: 7.103293505e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.204468551 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.834599633 on level 1 Castro::reflux() at level 0 : time = 2.325507816 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.453567094 Gravity::multilevel_solve_for_new_phi() time = 1.012365253 Castro::construct_new_gravity() time = 1.123673434 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 2.010693733 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029418698 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.823055827e+17 ymom added: 2.352230085e+19 zmom added: 9.53939202e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.145486667 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02274467863 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.98955856 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032280808 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.405139808 Gravity::multilevel_solve_for_new_phi() time = 0.535494011 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.508200718 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.334745053 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.31551424 on level 0 Castro::construct_old_gravity() time = 0.031759414 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.200546555e+22 ymom added: 1.239621202e+22 zmom added: 8.024045442e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.195057358 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.790099537 on level 1 Castro::reflux() at level 0 : time = 2.305497369 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.453374404 Gravity::multilevel_solve_for_new_phi() time = 0.996264081 Castro::construct_new_gravity() time = 1.111117603 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.992821035 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029130415 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.597986102e+18 ymom added: -2.568853227e+19 zmom added: -1.336957486e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.145640437 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02321829061 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.09852998 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.031364647 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.40015736 Gravity::multilevel_solve_for_new_phi() time = 0.528061529 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.417305858 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.33748825 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.13991876 on level 0 Castro::construct_old_gravity() time = 0.034282701 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013464896e+23 ymom added: 1.888946593e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.197002614 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.791720406 on level 1 Castro::reflux() at level 0 : time = 2.291377958 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.465477184 Gravity::multilevel_solve_for_new_phi() time = 1.004895248 Castro::construct_new_gravity() time = 1.113838839 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.972109739 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029212368 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727060174e+18 ymom added: -2.54093091e+19 zmom added: -1.252338494e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.146572275 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02339218647 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 46.71947919 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.525643263 seconds Ending run at 07:06:19 UTC on 2024-04-12. Run time = 242.2167782 Run time without initialization = 230.5761464 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 242.2 ... 242.2 ... 242.2 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 94.1519 103.4027 107.4515 44.36% Castro::computeTemp() 88 14.74 16.57 17.4 7.18% amrex::Copy() 679 12.75 14.16 14.75 6.09% check_for_negative_density() 9 0.9318 4.982 14.23 5.87% Castro::normalize_species() 87 10.14 11.5 12.42 5.13% MultiFab::AddProduct() 4205 10.51 11.67 12.12 5.00% FabArray::ParallelCopy_finish() 830 1.627 4.33 10.72 4.43% Castro::reset_internal_energy(Fab) 1476 8.275 9.221 9.773 4.03% Castro::estTimeStep() 26 4.068 5.646 9.085 3.75% FabArray::setVal() 5070 7.33 8.092 8.394 3.47% MultiFab::Subtract() 175 6.985 7.777 8.087 3.34% Gravity::fill_multipole_BCs() 11 4.243 4.442 4.735 1.95% Castro::reset_internal_energy(MultiFab) 88 3.46 3.991 4.483 1.85% FabArray::sum() 4205 3.479 3.863 4.01 1.66% FluxRegister::Reflux() 24 2.962 3.37 3.658 1.51% FabArray::ParallelCopy_nowait() 830 2.112 2.255 2.616 1.08% MLPoisson::Fsmooth() 1376 2.122 2.351 2.443 1.01% Castro::enforce_min_density() 87 1.549 1.834 2.09 0.86% Castro::reflux() 4 1.476 1.682 1.823 0.75% Castro::FluxRegCrseInit 9 1.472 1.662 1.737 0.72% Castro::wd_update() 6 0.529 0.8566 1.571 0.65% FillPatchInterp(MF) 30 1.26 1.345 1.431 0.59% Castro::construct_old_gravity_source() 18 0.9388 1.056 1.207 0.50% amrex::Add() 136 1.029 1.143 1.19 0.49% Castro::initMFs() 5 1.07 1.118 1.172 0.48% Amr::InitAmr() 1 1.16 1.16 1.16 0.48% MLPoisson::Fapply() 318 0.8986 1.001 1.042 0.43% Castro::construct_new_gravity_source() 9 0.808 0.9046 1.018 0.42% Castro::problem_post_timestep() 10 0.7209 0.8626 0.9512 0.39% FillPatchIterator::Initialize 62 0.7435 0.8308 0.8715 0.36% FabArray::Saxpy() 182 0.7103 0.7905 0.8239 0.34% FillBoundary_finish() 1553 0.2845 0.457 0.8186 0.34% Castro::volInBoundary() 42 0.6654 0.7208 0.7689 0.32% VisMF::Write(FabArray) 3 0.5775 0.6575 0.7007 0.29% StateData::FillBoundary(geom) 50 0.5738 0.5909 0.6042 0.25% Castro::construct_new_source() 63 0.4864 0.5388 0.5633 0.23% MultiFab::max() 18 0.02252 0.1829 0.5362 0.22% MLCellLinOp::applyBC() 1742 0.3657 0.4285 0.4751 0.20% FillBoundary_nowait() 1553 0.3166 0.3809 0.451 0.19% MultiFab::contains_nan() 18 0.3831 0.4269 0.4475 0.18% FabArray::Xpay() 283 0.3502 0.3894 0.4053 0.17% Castro::initialize_do_advance() 9 0.2409 0.2664 0.277 0.11% Amr::writePlotFile() 2 0.09483 0.1449 0.2592 0.11% Castro::construct_old_source() 63 0.2035 0.2269 0.2446 0.10% FabArray::setDomainBndry() 92 0.2028 0.2229 0.2347 0.10% amrex::average_down 233 0.1941 0.2026 0.2066 0.09% Castro::fill_hybrid_hydro_source() 27 0.1685 0.1853 0.1926 0.08% MLCellLinOp::defineAuxData() 11 0.1544 0.1666 0.1723 0.07% Castro::initData() 1 0.1253 0.1403 0.1577 0.07% MLPoisson::FFlux() 150 0.00465 0.04156 0.152 0.06% MLCellLinOp::compGrad() 18 0.1279 0.1431 0.1508 0.06% FabArray::mult() 56 0.1173 0.1312 0.1372 0.06% MLMG::addInterpCorrection() 155 0.1103 0.1232 0.1287 0.05% StateData::define() 25 0.1073 0.1167 0.122 0.05% Castro::evaluate_source_change() 193 0.1063 0.1109 0.1151 0.05% Castro::swap_state_time_levels() 9 0.08886 0.09298 0.0968 0.04% Gravity::get_new_grav_vector() 10 0.08796 0.09271 0.09467 0.04% Gravity::get_old_grav_vector() 9 0.07582 0.08593 0.09408 0.04% Castro::initialize_advance() 9 0.06717 0.07593 0.07981 0.03% MLMG::prepareForSolve() 10 0.06465 0.06895 0.07223 0.03% MLMG::solve() 10 0.002732 0.04394 0.0695 0.03% MLCellLinOp::updateSolBC() 25 0.06716 0.0678 0.06858 0.03% FabArray::norminf(mask) 21 0.05609 0.06338 0.06773 0.03% AmrLevel::FillPatch() 62 0.047 0.05418 0.0621 0.03% Castro::apply_tagging_restrictions() 6 0.04761 0.05329 0.05566 0.02% AmrMesh::MakeNewGrids() 6 0.04608 0.04809 0.05236 0.02% MLCellLinOp::updateCorBC() 17 0.04568 0.04615 0.04681 0.02% MLMG::ResNormInf() 54 0.005051 0.01695 0.04573 0.02% BndryData::define() 27 0.04297 0.04325 0.04375 0.02% FabArray::norminf() 147 0.03817 0.04096 0.04324 0.02% MLMG::MLResNormInf() 15 0.0001028 0.0234 0.03491 0.01% Gravity::multilevel_solve_for_new_phi() 10 0.03164 0.03329 0.03443 0.01% MLCellLinOp::reflux() 30 0.004211 0.01187 0.03431 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01932 0.02262 0.02848 0.01% MLCellLinOp::defineBC() 11 0.02601 0.02631 0.02658 0.01% Castro::FluxRegFineAdd() 9 0.02547 0.02586 0.0265 0.01% MultiFab::Multiply() 8 0.02196 0.02442 0.02532 0.01% Castro::do_old_sources() 9 0.000605 0.006474 0.02203 0.01% Castro::Castro() 5 0.01464 0.01718 0.02093 0.01% Castro::build_fine_mask() 44 0.01768 0.01839 0.01942 0.01% MLMG::interpCorrection_1 17 0.01847 0.01868 0.0189 0.01% MLLinOp::makeSubCommunicator() 11 0.0006074 0.008453 0.01846 0.01% FabArrayBase::CPC::define() 416 0.01296 0.01495 0.01814 0.01% Castro::linear_to_hybrid_momentum() 1 0.0139 0.01592 0.01721 0.01% FillPatchTwoLevels 31 0.0126 0.01407 0.01698 0.01% Castro::buildMetrics() 5 0.01479 0.0162 0.0168 0.01% StateDataPhysBCFunct::() 95 0.002171 0.007046 0.01591 0.01% MLCellLinOp::prepareForSolve() 11 0.01145 0.01169 0.01196 0.00% Castro::enforce_consistent_e() 1 0.008861 0.01018 0.01073 0.00% FabArrayBase::getCPC() 1440 0.009809 0.01015 0.01055 0.00% AMRErrorTag::operator() 6 0.008855 0.009948 0.01048 0.00% Gravity::actual_solve_with_mlmg() 11 0.009233 0.009742 0.01008 0.00% Gravity::actual_multilevel_solve() 10 0.00881 0.009306 0.01005 0.00% Castro::finalize_do_advance() 9 0.007558 0.008098 0.009159 0.00% Gravity::update_max_rhs() 10 0.008067 0.008266 0.008529 0.00% Castro::apply_problem_tags() 6 0.007006 0.007843 0.008163 0.00% Castro::do_new_sources() 9 0.0005237 0.003185 0.008034 0.00% average_down_faces 48 0.005714 0.005797 0.005867 0.00% MLCellLinOp::setLevelBC() 19 0.003847 0.004606 0.005614 0.00% AmrLevel::derive() 6 0.003746 0.004343 0.005007 0.00% FabArray::FillBoundary() 1553 0.004244 0.004464 0.004726 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0007941 0.002341 0.004211 0.00% MLCellLinOp::smooth() 688 0.003828 0.003942 0.004096 0.00% MLCGSolver::bicgstab 7 0 0.0009739 0.003896 0.00% FabArrayBase::getFB() 1553 0.003061 0.003232 0.00354 0.00% amrex::Dot() 115 0 0.0008471 0.003388 0.00% FabArrayBase::FB::FB() 19 0.002679 0.00302 0.003296 0.00% FabArray::ParallelCopy() 830 0.002986 0.003089 0.003205 0.00% Gravity::test_composite_phi() 1 0.002421 0.002523 0.002762 0.00% Castro::construct_new_gravity() 9 0.0003986 0.001059 0.001877 0.00% main() 1 0.001551 0.001639 0.001804 0.00% MLLinOp::defineGrids() 11 0.001342 0.001439 0.00164 0.00% MLCellLinOp::apply() 318 0.0008936 0.001064 0.001473 0.00% Amr::regrid() 5 0.0009839 0.001098 0.001373 0.00% Castro::clean_state() 87 0.001178 0.001267 0.001363 0.00% TagBoxArray::collate() 6 0.000566 0.0009544 0.001262 0.00% AmrLevel::FillCoarsePatch() 5 0.000973 0.001122 0.00122 0.00% MLCGSolver::ParallelAllReduce 183 0 0.000286 0.001144 0.00% Amr::coarseTimeStep() 5 0.000434 0.0009429 0.001123 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009006 0.0009233 0.0009362 0.00% OwnerMask() 6 0.0007637 0.0008474 0.000928 0.00% MLCellLinOp::correctionResidual() 189 0.0006765 0.0006891 0.00071 0.00% MLMG::mgVcycle() 65 0.0005898 0.0006119 0.0006318 0.00% Castro::do_advance_ctu() 5 0.0004807 0.0005329 0.000591 0.00% Amr::grid_places() 6 0.0004368 0.0004983 0.0005703 0.00% FillPatchSingleLevel 92 0.0005432 0.0005529 0.000569 0.00% Castro::construct_old_gravity() 9 0.0003961 0.0004373 0.0005444 0.00% Castro::finalize_advance() 9 0.0002926 0.0004091 0.0005126 0.00% TagBoxArray::mapPRD 6 0.0004415 0.0004532 0.0004652 0.00% FillPatchIterator::FillFromLevel0() 31 0.0004023 0.0004216 0.0004649 0.00% MLMG:computeResOfCorrection() 155 0.0003856 0.0004007 0.0004252 0.00% MLMG::oneIter() 31 0.0003449 0.0003596 0.0003773 0.00% MLCellLinOp::solutionResidual() 72 0.0003467 0.0003595 0.000376 0.00% MLMG::actualBottomSolve() 31 3.996e-05 0.0001251 0.0003744 0.00% Castro::subcycle_advance_ctu() 5 7.369e-05 0.000161 0.0003641 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002916 0.0003066 0.0003214 0.00% MLMG::mgVcycle_down::0 31 0.0002862 0.000293 0.0003044 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002386 0.0002488 0.0002654 0.00% Amr::timeStep() 9 0.0001827 0.0002128 0.0002614 0.00% Castro::enforce_speed_limit() 87 0.0002422 0.0002476 0.0002586 0.00% MLMG::mgVcycle_down::1 31 0.0002416 0.0002518 0.0002583 0.00% Amr::defBaseLevel() 1 0.0002224 0.0002344 0.0002436 0.00% Castro::computeNewDt() 8 0.0001154 0.0001489 0.0002293 0.00% Castro::post_regrid() 10 0.0001979 0.000208 0.0002223 0.00% MLMG::mgVcycle_bottom 65 0.0002089 0.0002144 0.0002175 0.00% AmrLevel::AmrLevel(dm) 5 0.0001824 0.0001975 0.0002166 0.00% Amr::FinalizeInit() 1 2.713e-05 7.517e-05 0.0002134 0.00% MLMG::mgVcycle_down::3 31 0.0001912 0.000201 0.0002116 0.00% MLMG::mgVcycle_down::2 31 0.0001895 0.0001978 0.0002115 0.00% MLMG::mgVcycle_up::0 31 0.0001858 0.0001891 0.0001933 0.00% makeSFC 33 0.0001615 0.0001721 0.0001898 0.00% FabArrayBase::TheFPinfo() 25 0.0001682 0.0001745 0.0001856 0.00% MLMG::mgVcycle_down::4 31 0.0001631 0.0001756 0.000183 0.00% Castro::check_for_nan() 18 0.0001287 0.0001399 0.0001509 0.00% MLMG::mgVcycle_up::1 31 0.000136 0.000141 0.0001493 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001272 0.0001357 0.0001437 0.00% MLMG::mgVcycle_up::2 31 0.0001289 0.0001339 0.0001417 0.00% Castro::expand_state() 13 0.0001031 0.0001154 0.0001317 0.00% MLMG::mgVcycle_up::3 31 0.0001192 0.0001264 0.0001309 0.00% Amr::writeSmallPlotFile() 1 0.0001143 0.0001224 0.0001287 0.00% MLMG::mgVcycle_up::4 31 0.0001095 0.0001181 0.0001265 0.00% Castro::react_state() 18 0.0001071 0.0001138 0.0001242 0.00% FPinfo::FPinfo() 3 0.0001128 0.0001178 0.0001231 0.00% FabArray::RecvLayoutMask() 8 9.209e-05 0.0001046 0.0001198 0.00% MLPoisson::define() 11 0.0001046 0.0001093 0.0001149 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001079 0.0001099 0.0001147 0.00% Castro::apply_source_to_state() 18 9.857e-05 0.000107 0.0001141 0.00% DistributionMapping::Distribute() 35 0.0001049 0.0001076 0.0001126 0.00% Castro::errorEst() 6 0.0001038 0.0001072 0.0001117 0.00% Gravity::swapTimeLevels() 9 8.533e-05 9.286e-05 0.0001003 0.00% Castro::advance() 5 8.609e-05 9.05e-05 9.499e-05 0.00% ClusterList::intersect() 1 0 2.284e-05 9.135e-05 0.00% MLMG::computeResidual() 31 8.549e-05 8.774e-05 8.889e-05 0.00% MLMG::computeMLResidual() 15 7.489e-05 8.02e-05 8.756e-05 0.00% MLMG::getGradSolution() 10 6.029e-05 6.672e-05 8.207e-05 0.00% Castro::post_timestep() 9 7.574e-05 7.888e-05 8.158e-05 0.00% Castro::avgDown(state_indx) 20 6.758e-05 7.169e-05 7.702e-05 0.00% MLLinOp::define() 11 5.901e-05 6.379e-05 7.225e-05 0.00% MLMG::miniCycle() 34 6.646e-05 6.859e-05 7.064e-05 0.00% knapsack() 3 6.415e-05 6.617e-05 6.975e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.067e-05 6.484e-05 6.832e-05 0.00% Castro::create_source_corrector() 9 5.045e-05 5.92e-05 6.428e-05 0.00% Castro::init(old) 3 4.415e-05 4.822e-05 5.782e-05 0.00% Castro::construct_new_hybrid_source() 9 4.487e-05 5.047e-05 5.75e-05 0.00% MLPoisson::prepareForSolve() 11 4.789e-05 5.074e-05 5.295e-05 0.00% amrex::average_down_w_geom 20 4.878e-05 5.086e-05 5.263e-05 0.00% Amr::initSubcycle() 1 5.052e-05 5.168e-05 5.229e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 3.929e-05 4.169e-05 4.692e-05 0.00% AmrMesh-cluster 1 0 1.062e-05 4.247e-05 0.00% Gravity::set_mass_offset() 6 3.137e-05 3.641e-05 4.115e-05 0.00% Castro::avgDown() 4 3.074e-05 3.373e-05 3.853e-05 0.00% Castro::construct_old_hybrid_source() 9 3.488e-05 3.644e-05 3.837e-05 0.00% MLMG::MLRhsNormInf() 10 3.021e-05 3.312e-05 3.652e-05 0.00% MLMG::compResidual() 1 2.572e-05 2.91e-05 3.411e-05 0.00% DistributionMapping::KnapSackDoIt() 3 3.044e-05 3.158e-05 3.275e-05 0.00% BoxList::parallelComplementIn() 12 2.436e-05 2.579e-05 2.828e-05 0.00% Castro::post_init() 1 2.086e-05 2.407e-05 2.607e-05 0.00% Castro::construct_old_rotation_source() 9 2.07e-05 2.303e-05 2.585e-05 0.00% Amr::bldFineLevels() 1 1.854e-05 2.118e-05 2.539e-05 0.00% Castro::retry_advance_ctu() 5 2.276e-05 2.419e-05 2.5e-05 0.00% BoxList::complementIn 12 2.185e-05 2.238e-05 2.3e-05 0.00% Castro::construct_new_rotation_source() 9 2.118e-05 2.199e-05 2.286e-05 0.00% Castro::init() 1 1.558e-05 1.809e-05 2.195e-05 0.00% Castro::computeInitialDt() 2 1.334e-05 1.614e-05 2.118e-05 0.00% Castro::derive() 6 1.904e-05 1.99e-05 2.041e-05 0.00% Amr::InitializeInit() 1 6.916e-06 8.098e-06 1.081e-05 0.00% ClusterList::chop() 1 0 2.679e-06 1.072e-05 0.00% Amr::init() 1 6.523e-06 6.739e-06 7.073e-06 0.00% Amr::initialInit() 1 4.207e-06 5.427e-06 6.982e-06 0.00% knapsack()swap 1 3.675e-06 4.252e-06 5.143e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 242.2 242.2 242.2 100.00% Amr::coarseTimeStep() 5 230 230 230 94.98% Amr::timeStep() 9 228.7 228.7 228.7 94.41% Castro::advance() 5 201.3 201.3 201.3 83.10% Castro::subcycle_advance_ctu() 5 192.7 193 193.7 79.97% Castro::do_advance_ctu() 5 192.7 193 193.7 79.97% Castro::construct_ctu_hydro_source() 9 118 118.2 118.3 48.83% Castro::clean_state() 87 71.66 80.29 84.1 34.72% Castro::computeTemp() 88 45.33 50.79 53.52 22.09% Castro::reset_internal_energy(MultiFab) 88 30.59 34.23 36.12 14.91% Castro::evaluate_source_change() 193 17.83 19.79 20.54 8.48% Castro::enforce_min_density() 87 16.96 18.88 19.83 8.19% Castro::post_regrid() 10 15.9 17.37 18.06 7.45% Amr::regrid() 5 15.77 17.08 17.69 7.30% Castro::initialize_do_advance() 9 14.47 15.04 15.38 6.35% amrex::Copy() 679 12.75 14.16 14.75 6.09% check_for_negative_density() 9 0.9318 4.982 14.23 5.87% AmrLevel::FillPatch() 62 8.081 9.636 13.86 5.72% FabArray::ParallelCopy() 830 4.309 6.648 12.91 5.33% Castro::do_old_sources() 9 11.11 11.59 12.86 5.31% Castro::normalize_species() 87 10.14 11.5 12.42 5.13% FillPatchIterator::Initialize 62 6.393 8.019 12.41 5.13% MultiFab::AddProduct() 4205 10.51 11.67 12.12 5.00% Gravity::solve_phi_with_mlmg() 11 11.17 11.37 11.66 4.82% Gravity::multilevel_solve_for_new_phi() 10 10.87 11.07 11.36 4.69% Gravity::actual_multilevel_solve() 10 10.84 11.04 11.33 4.68% Castro::do_new_sources() 9 9.309 10.41 10.93 4.51% FabArray::ParallelCopy_finish() 830 1.627 4.33 10.72 4.43% Amr::init() 1 10.48 10.48 10.48 4.33% FillPatchSingleLevel 92 4.201 5.86 10.4 4.29% Castro::post_timestep() 9 10.09 10.09 10.1 4.17% FillPatchIterator::FillFromLevel0() 31 3.834 5.485 9.988 4.12% Amr::initialInit() 1 9.925 9.925 9.926 4.10% Castro::reset_internal_energy(Fab) 1476 8.275 9.221 9.773 4.03% Castro::reflux() 4 9.045 9.173 9.275 3.83% Castro::estTimeStep() 26 4.068 5.646 9.085 3.75% Castro::initialize_advance() 9 7.586 8.301 8.621 3.56% FabArray::setVal() 5070 7.33 8.092 8.394 3.47% MultiFab::Subtract() 175 6.985 7.777 8.087 3.34% Gravity::actual_solve_with_mlmg() 11 6.826 6.833 6.84 2.82% Castro::construct_new_gravity() 9 6.271 6.461 6.744 2.78% Amr::FinalizeInit() 1 5.79 5.978 6.381 2.63% MLMG::solve() 10 6.311 6.322 6.338 2.62% Castro::expand_state() 13 5.17 5.478 6.099 2.52% MLMG::oneIter() 31 5.109 5.126 5.133 2.12% Gravity::fill_multipole_BCs() 11 4.324 4.519 4.807 1.98% FluxRegister::Reflux() 24 4.484 4.554 4.661 1.92% MLMG::mgVcycle() 65 4.281 4.308 4.318 1.78% Castro::finalize_do_advance() 9 1.358 2.21 4.137 1.71% Amr::InitializeInit() 1 3.545 3.947 4.136 1.71% Amr::defBaseLevel() 1 3.545 3.947 4.136 1.71% FabArray::sum() 4205 3.479 3.863 4.01 1.66% Castro::problem_post_timestep() 10 2.801 3.183 3.899 1.61% Castro::computeNewDt() 8 2.203 2.741 3.85 1.59% MLCellLinOp::smooth() 688 3.246 3.266 3.296 1.36% Castro::post_init() 1 2.562 2.722 3.075 1.27% Castro::initData() 1 2.171 2.517 2.749 1.13% FabArray::ParallelCopy_nowait() 830 2.171 2.315 2.679 1.11% Castro::FluxRegCrseInit 9 1.784 2.069 2.673 1.10% Castro::construct_old_gravity() 9 0.6743 1.323 2.602 1.07% Gravity::get_old_grav_vector() 9 0.6738 1.322 2.602 1.07% MLPoisson::Fsmooth() 1376 2.122 2.351 2.443 1.01% Castro::wd_update() 6 1.194 1.577 2.293 0.95% Castro::construct_new_source() 63 1.846 2.052 2.197 0.91% MLMG::mgVcycle_down::0 31 1.896 1.96 1.982 0.82% MLCellLinOp::applyBC() 1742 1.141 1.277 1.646 0.68% Amr::bldFineLevels() 1 1.252 1.446 1.528 0.63% FillPatchIterator::FillFromTwoLevels() 31 1.464 1.481 1.509 0.62% FillPatchTwoLevels 31 1.464 1.481 1.509 0.62% Castro::Castro() 5 1.346 1.389 1.449 0.60% FillPatchInterp(MF) 30 1.26 1.345 1.431 0.59% MLMG::mgVcycle_up::0 31 1.37 1.394 1.405 0.58% MLCellLinOp::apply() 318 1.265 1.298 1.315 0.54% FabArray::FillBoundary() 1553 0.666 0.8485 1.28 0.53% Castro::construct_old_gravity_source() 18 0.9388 1.056 1.207 0.50% amrex::Add() 136 1.029 1.143 1.19 0.49% Castro::initMFs() 5 1.075 1.124 1.177 0.49% Amr::InitAmr() 1 1.16 1.16 1.161 0.48% Castro::computeInitialDt() 2 0.5145 0.7033 1.106 0.46% Castro::construct_old_source() 63 0.9372 1.038 1.09 0.45% Amr::writePlotFile() 2 1.08 1.08 1.08 0.45% MLPoisson::Fapply() 318 0.8986 1.001 1.042 0.43% MLCellLinOp::solutionResidual() 72 0.9627 1.014 1.035 0.43% Castro::construct_new_gravity_source() 9 0.808 0.9046 1.018 0.42% FabArray::Saxpy() 182 0.7103 0.7905 0.8239 0.34% FillBoundary_finish() 1553 0.2845 0.457 0.8186 0.34% MLCellLinOp::correctionResidual() 189 0.7978 0.8051 0.8132 0.34% Gravity::get_new_grav_vector() 10 0.7715 0.7805 0.791 0.33% Castro::volInBoundary() 42 0.6654 0.7209 0.7689 0.32% VisMF::Write(FabArray) 3 0.5775 0.6575 0.7007 0.29% MLMG:computeResOfCorrection() 155 0.6471 0.6819 0.6989 0.29% StateDataPhysBCFunct::() 95 0.576 0.598 0.6111 0.25% StateData::FillBoundary(geom) 50 0.5738 0.5909 0.6042 0.25% Gravity::update_max_rhs() 10 0.08105 0.2406 0.5901 0.24% MLMG::computeResWithCrseSolFineCor() 17 0.5705 0.5713 0.5731 0.24% MultiFab::max() 18 0.02252 0.1829 0.5362 0.22% MLMG::computeMLResidual() 15 0.4132 0.4337 0.489 0.20% Castro::init() 1 0.4835 0.486 0.4879 0.20% AmrLevel::FillCoarsePatch() 5 0.4835 0.486 0.4879 0.20% FillBoundary_nowait() 1553 0.3229 0.3871 0.4572 0.19% Castro::check_for_nan() 18 0.3832 0.427 0.4477 0.18% MultiFab::contains_nan() 18 0.3831 0.4269 0.4475 0.18% Gravity::test_composite_phi() 1 0.4135 0.4141 0.4149 0.17% amrex::average_down 233 0.2774 0.3632 0.4083 0.17% FabArray::Xpay() 283 0.3502 0.3894 0.4053 0.17% MLMG::mgVcycle_down::1 31 0.3128 0.3341 0.3926 0.16% MLCellLinOp::reflux() 30 0.1816 0.2309 0.3637 0.15% MLMG::computeResidual() 31 0.3472 0.3556 0.3614 0.15% Castro::init(old) 3 0.3528 0.3551 0.356 0.15% Castro::apply_source_to_state() 18 0.2842 0.3164 0.3305 0.14% MLPoisson::define() 11 0.2887 0.2932 0.2977 0.12% Castro::react_state() 18 0.221 0.2447 0.2626 0.11% FabArray::setDomainBndry() 92 0.2028 0.2229 0.2347 0.10% MLMG::prepareForSolve() 10 0.2181 0.2224 0.2249 0.09% MLMG::mgVcycle_up::1 31 0.2162 0.2201 0.223 0.09% MLCellLinOp::defineAuxData() 11 0.1809 0.1955 0.2061 0.09% Amr::grid_places() 6 0.1941 0.1942 0.1943 0.08% AmrMesh::MakeNewGrids() 6 0.1936 0.1937 0.1938 0.08% Castro::fill_hybrid_hydro_source() 27 0.1685 0.1853 0.1926 0.08% MLMG::addInterpCorrection() 155 0.1325 0.1685 0.188 0.08% MLMG::getGradSolution() 10 0.1813 0.1834 0.1849 0.08% MLCellLinOp::compGrad() 18 0.1812 0.1833 0.1849 0.08% MLMG::mgVcycle_bottom 65 0.1506 0.1578 0.1785 0.07% MLPoisson::FFlux() 150 0.00465 0.04156 0.152 0.06% MLMG::miniCycle() 34 0.1506 0.151 0.1511 0.06% Castro::avgDown() 4 0.1184 0.1314 0.1423 0.06% Castro::avgDown(state_indx) 20 0.1184 0.1313 0.1422 0.06% amrex::average_down_w_geom 20 0.1183 0.1312 0.1422 0.06% FabArray::mult() 56 0.1173 0.1312 0.1372 0.06% Castro::construct_new_hybrid_source() 9 0.1138 0.1241 0.1288 0.05% Castro::errorEst() 6 0.1045 0.1167 0.1225 0.05% AmrLevel::AmrLevel(dm) 5 0.1075 0.1169 0.1222 0.05% StateData::define() 25 0.1073 0.1167 0.122 0.05% MLMG::ResNormInf() 54 0.08032 0.08889 0.1092 0.05% Castro::swap_state_time_levels() 9 0.08886 0.09298 0.0968 0.04% MLCellLinOp::updateSolBC() 25 0.07812 0.08352 0.09365 0.04% MLCellLinOp::defineBC() 11 0.08624 0.0872 0.08916 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08578 0.08669 0.08894 0.04% MLMG::mgVcycle_down::2 31 0.06879 0.07252 0.07974 0.03% MLMG::mgVcycle_up::3 31 0.02577 0.06455 0.07829 0.03% MLMG::MLResNormInf() 15 0.04519 0.06793 0.07693 0.03% FabArray::norminf(mask) 21 0.05609 0.06338 0.06773 0.03% Castro::construct_old_hybrid_source() 9 0.05483 0.06123 0.06389 0.03% BndryData::define() 27 0.05885 0.05964 0.06142 0.03% MLMG::interpCorrection_1 17 0.02656 0.03913 0.06012 0.02% MLMG::mgVcycle_up::2 31 0.05376 0.05493 0.05612 0.02% Castro::apply_tagging_restrictions() 6 0.04761 0.05329 0.05566 0.02% MLCellLinOp::updateCorBC() 17 0.04955 0.05052 0.05243 0.02% Castro::derive() 6 0.04091 0.04548 0.04869 0.02% AmrLevel::derive() 6 0.04089 0.04546 0.04867 0.02% Castro::buildMetrics() 5 0.03745 0.04426 0.04728 0.02% FabArray::norminf() 147 0.03817 0.04096 0.04324 0.02% Gravity::swapTimeLevels() 9 0.03438 0.03639 0.03889 0.02% MLMG::mgVcycle_down::3 31 0.03097 0.03227 0.0346 0.01% TagBoxArray::mapPRD 6 0.008347 0.01685 0.0311 0.01% MLMG::MLRhsNormInf() 10 0.02569 0.02863 0.02969 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01932 0.02262 0.02848 0.01% FabArrayBase::getCPC() 1440 0.02277 0.0251 0.02813 0.01% MLMG::actualBottomSolve() 31 3.996e-05 0.006912 0.02752 0.01% MLCGSolver::bicgstab 7 0 0.006698 0.02679 0.01% Castro::FluxRegFineAdd() 9 0.02547 0.02586 0.0265 0.01% MultiFab::Multiply() 8 0.02196 0.02442 0.02532 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.02008 0.02056 0.02166 0.01% MLMG::compResidual() 1 0.01816 0.02011 0.02145 0.01% MLLinOp::define() 11 0.002383 0.01034 0.02058 0.01% MLLinOp::defineGrids() 11 0.002324 0.01027 0.0205 0.01% Castro::build_fine_mask() 44 0.01768 0.01839 0.01942 0.01% MLMG::mgVcycle_down::4 31 0.005427 0.008956 0.01853 0.01% MLLinOp::makeSubCommunicator() 11 0.0006074 0.008453 0.01846 0.01% FabArrayBase::CPC::define() 416 0.01296 0.01495 0.01814 0.01% Castro::linear_to_hybrid_momentum() 1 0.0139 0.01592 0.01721 0.01% MLMG::mgVcycle_up::4 31 0.004473 0.007042 0.01401 0.01% MLPoisson::prepareForSolve() 11 0.0115 0.01174 0.012 0.00% MLCellLinOp::prepareForSolve() 11 0.01145 0.01169 0.01196 0.00% Castro::enforce_consistent_e() 1 0.008861 0.01018 0.01073 0.00% AMRErrorTag::operator() 6 0.008855 0.009948 0.01048 0.00% average_down_faces 48 0.009106 0.009798 0.01018 0.00% Castro::apply_problem_tags() 6 0.007006 0.007843 0.008163 0.00% FabArrayBase::getFB() 1553 0.005856 0.006252 0.006539 0.00% MLCellLinOp::setLevelBC() 19 0.003847 0.004606 0.005614 0.00% FabArrayBase::TheFPinfo() 25 0.001124 0.002667 0.00453 0.00% FPinfo::FPinfo() 3 0.0009385 0.002492 0.004361 0.00% DistributionMapping::KnapSackDoIt() 3 0.0008154 0.002375 0.004243 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0007941 0.002341 0.004211 0.00% amrex::Dot() 115 0 0.0008471 0.003388 0.00% FabArrayBase::FB::FB() 19 0.002679 0.00302 0.003296 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007364 0.002946 0.00% TagBoxArray::collate() 6 0.000566 0.0009544 0.001262 0.00% OwnerMask() 6 0.0007637 0.0008474 0.000928 0.00% Castro::finalize_advance() 9 0.0002926 0.0004091 0.0005126 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003648 0.0003793 0.0004065 0.00% makeSFC 33 0.0002563 0.0002693 0.0002917 0.00% Castro::enforce_speed_limit() 87 0.0002422 0.0002476 0.0002586 0.00% AmrMesh-cluster 1 0 3.614e-05 0.0001445 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001013 0.0001202 0.0001366 0.00% Amr::writeSmallPlotFile() 1 0.0001143 0.0001224 0.0001287 0.00% FabArray::RecvLayoutMask() 8 9.209e-05 0.0001046 0.0001198 0.00% DistributionMapping::Distribute() 35 0.0001049 0.0001076 0.0001126 0.00% ClusterList::intersect() 1 0 2.284e-05 9.135e-05 0.00% knapsack() 3 6.782e-05 7.042e-05 7.402e-05 0.00% Castro::create_source_corrector() 9 5.045e-05 5.92e-05 6.428e-05 0.00% Amr::initSubcycle() 1 5.052e-05 5.168e-05 5.229e-05 0.00% BoxList::parallelComplementIn() 12 4.692e-05 4.817e-05 5.033e-05 0.00% Gravity::set_mass_offset() 6 3.137e-05 3.641e-05 4.115e-05 0.00% Castro::construct_old_rotation_source() 9 2.07e-05 2.303e-05 2.585e-05 0.00% Castro::retry_advance_ctu() 5 2.276e-05 2.419e-05 2.5e-05 0.00% BoxList::complementIn 12 2.185e-05 2.238e-05 2.3e-05 0.00% Castro::construct_new_rotation_source() 9 2.118e-05 2.199e-05 2.286e-05 0.00% ClusterList::chop() 1 0 2.679e-06 1.072e-05 0.00% knapsack()swap 1 3.675e-06 4.252e-06 5.143e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (24.04-15-g7813bb7d6bc6) finalized