Initializing AMReX (24.04-2-gf2536be82d98)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (24.04-2-gf2536be82d98) initialized Starting run at 07:49:52 UTC on 2024-04-02. Successfully read inputs file ... Castro git describe: 24.04 AMReX git describe: 24.04-2-gf2536be82 Microphysics git describe: 24.04 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.031694996 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031040394 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.36776098 Gravity::multilevel_solve_for_new_phi() time = 1.784335495 Gravity::fill_multipole_BCs() time = 0.378438 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.533194863 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.866560981 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793356e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.290031385 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 20.58620149 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.623725645 Gravity::multilevel_solve_for_new_phi() time = 1.663883146 Castro::construct_new_gravity() time = 1.784512207 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730196e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.290800184 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02497730945 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 37.50143804 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.033196525 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.443725661 Gravity::multilevel_solve_for_new_phi() time = 1.997889958 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.74532375 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.032457902 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.4663502 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.030719808 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.571938195e+22 ymom added: 1.475739526e+22 zmom added: 6.441948907e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.193577288 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.79914311 on level 1 Castro::reflux() at level 0 : time = 2.270829366 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.426280278 Gravity::multilevel_solve_for_new_phi() time = 1.366573065 Castro::construct_new_gravity() time = 1.480115208 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.984659611 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.029348215 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.764881231e+18 ymom added: -1.360987807e+19 zmom added: -8.219114393e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.142853455 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02277588917 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.88222699 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.033008655 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402452252 Gravity::multilevel_solve_for_new_phi() time = 0.533839522 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.734541684 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.037360426 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.28972808 on level 0 Castro::construct_old_gravity() time = 0.032805093 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.444240407e+22 ymom added: 4.427218578e+21 zmom added: 7.10286116e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.196072459 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.790574149 on level 1 Castro::reflux() at level 0 : time = 2.264885596 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.427551165 Gravity::multilevel_solve_for_new_phi() time = 0.989164361 Castro::construct_new_gravity() time = 1.100617407 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156837e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.990929113 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029588187 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.823580672e+17 ymom added: 2.351779725e+19 zmom added: 9.539392927e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.141103632 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02324800937 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 46.98571572 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032917501 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.410361486 Gravity::multilevel_solve_for_new_phi() time = 0.544309997 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.839907259 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.020742998 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.31638086 on level 0 Castro::construct_old_gravity() time = 0.031707781 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.200688419e+22 ymom added: 1.239621202e+22 zmom added: 8.023757211e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.192768772 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.787961061 on level 1 Castro::reflux() at level 0 : time = 2.224177722 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.425247072 Gravity::multilevel_solve_for_new_phi() time = 0.978181536 Castro::construct_new_gravity() time = 1.091243738 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.965786163 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.034883251 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598289206e+18 ymom added: -2.568853227e+19 zmom added: -1.336957843e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.146377519 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02325330402 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.01971196 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.032577015 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402820262 Gravity::multilevel_solve_for_new_phi() time = 0.534885759 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.857162099 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.045961627 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.41119603 on level 0 Castro::construct_old_gravity() time = 0.035625897 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013402454e+23 ymom added: 1.770887431e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.193988283 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.794102039 on level 1 Castro::reflux() at level 0 : time = 2.271019457 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.454814049 Gravity::multilevel_solve_for_new_phi() time = 1.002025698 Castro::construct_new_gravity() time = 1.111693751 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.949294676 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029722095 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.727139562e+18 ymom added: -2.54093091e+19 zmom added: -1.252338466e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.142254582 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02317405675 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.18058803 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.535464958 seconds Ending run at 07:53:53 UTC on 2024-04-02. Run time = 240.7439807 Run time without initialization = 229.1068453 Average number of zones advanced per microsecond: 0.022 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 240.7 ... 240.7 ... 240.7 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 93.4413 102.5696 106.8127 44.37% Castro::computeTemp() 88 14.87 16.6 17.39 7.22% amrex::Copy() 679 13.07 14.58 15.26 6.34% check_for_negative_density() 9 0.285 4.622 13.83 5.75% MultiFab::AddProduct() 4205 10.64 11.79 12.29 5.10% Castro::normalize_species() 87 10.08 11.26 11.75 4.88% FabArray::ParallelCopy_finish() 830 1.389 4.399 10.61 4.41% Castro::reset_internal_energy(Fab) 1476 8.437 9.272 9.646 4.01% Castro::estTimeStep() 26 3.963 5.447 8.811 3.66% FabArray::setVal() 5070 7.172 7.99 8.37 3.48% MultiFab::Subtract() 175 6.813 7.583 7.923 3.29% Gravity::fill_multipole_BCs() 11 4.32 4.459 4.687 1.95% Castro::reset_internal_energy(MultiFab) 88 3.251 3.847 4.143 1.72% FabArray::sum() 4205 3.496 3.867 4.032 1.67% FluxRegister::Reflux() 24 2.945 3.304 3.456 1.44% FabArray::ParallelCopy_nowait() 830 2.096 2.285 2.656 1.10% MLPoisson::Fsmooth() 1376 2.149 2.387 2.484 1.03% Castro::enforce_min_density() 87 1.533 1.82 2.048 0.85% Castro::FluxRegCrseInit 9 1.496 1.697 1.776 0.74% Castro::reflux() 4 1.502 1.687 1.754 0.73% Castro::wd_update() 6 0.523 0.9114 1.678 0.70% FillPatchInterp(MF) 30 1.258 1.322 1.401 0.58% Amr::InitAmr() 1 1.207 1.207 1.207 0.50% amrex::Add() 136 1.008 1.109 1.154 0.48% Castro::initMFs() 5 1.085 1.116 1.13 0.47% MLPoisson::Fapply() 318 0.9108 1.016 1.072 0.45% Castro::construct_old_gravity_source() 18 0.8924 0.999 1.045 0.43% Castro::construct_new_gravity_source() 9 0.7374 0.8429 0.8853 0.37% FillBoundary_finish() 1553 0.3293 0.479 0.8719 0.36% FillPatchIterator::Initialize 62 0.7432 0.8285 0.8697 0.36% Castro::problem_post_timestep() 10 0.7343 0.7662 0.835 0.35% FabArray::Saxpy() 182 0.6883 0.7677 0.8051 0.33% Castro::volInBoundary() 42 0.6945 0.7346 0.7708 0.32% VisMF::Write(FabArray) 3 0.5837 0.6414 0.6825 0.28% StateData::FillBoundary(geom) 50 0.5901 0.6007 0.6143 0.26% Castro::construct_new_source() 63 0.476 0.5296 0.5515 0.23% MultiFab::max() 18 0.02538 0.1904 0.526 0.22% MultiFab::contains_nan() 18 0.4022 0.4469 0.4666 0.19% MLCellLinOp::applyBC() 1742 0.3626 0.4221 0.4539 0.19% FillBoundary_nowait() 1553 0.3576 0.3917 0.4452 0.18% FabArray::Xpay() 283 0.3508 0.392 0.4159 0.17% Castro::initialize_do_advance() 9 0.2308 0.2607 0.2747 0.11% FabArray::setDomainBndry() 92 0.194 0.2164 0.2288 0.10% Amr::writePlotFile() 2 0.08021 0.1397 0.2215 0.09% Castro::fill_hybrid_hydro_source() 27 0.1706 0.1936 0.2142 0.09% Castro::construct_old_source() 63 0.1781 0.1997 0.2125 0.09% amrex::average_down 233 0.1952 0.203 0.2073 0.09% MLCellLinOp::compGrad() 18 0.1263 0.1492 0.1813 0.08% MLCellLinOp::defineAuxData() 11 0.1541 0.1679 0.1758 0.07% Castro::initData() 1 0.1277 0.1465 0.1648 0.07% MLPoisson::FFlux() 150 0.004617 0.04311 0.1583 0.07% FabArray::mult() 56 0.1172 0.1314 0.1386 0.06% MLMG::addInterpCorrection() 155 0.1078 0.121 0.1273 0.05% StateData::define() 25 0.1078 0.1186 0.1234 0.05% Castro::evaluate_source_change() 193 0.1037 0.1116 0.1157 0.05% Gravity::get_new_grav_vector() 10 0.08647 0.09479 0.09949 0.04% Castro::swap_state_time_levels() 9 0.08609 0.09097 0.09457 0.04% Gravity::get_old_grav_vector() 9 0.07883 0.08657 0.09202 0.04% Castro::initialize_advance() 9 0.06908 0.07684 0.08088 0.03% MLMG::prepareForSolve() 10 0.06264 0.06796 0.07105 0.03% MLCellLinOp::updateSolBC() 25 0.06742 0.06799 0.0691 0.03% MLMG::solve() 10 0.0005931 0.04369 0.06812 0.03% AmrLevel::FillPatch() 62 0.04758 0.05613 0.06611 0.03% FabArray::norminf(mask) 21 0.05343 0.05982 0.06258 0.03% Castro::apply_tagging_restrictions() 6 0.04844 0.05379 0.05646 0.02% MLMG::ResNormInf() 54 0.001617 0.02202 0.05341 0.02% AmrMesh::MakeNewGrids() 6 0.04669 0.05 0.05319 0.02% MLCellLinOp::updateCorBC() 17 0.04573 0.04618 0.04696 0.02% FabArray::norminf() 147 0.04069 0.04314 0.04604 0.02% BndryData::define() 27 0.04252 0.04287 0.04355 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.03139 0.03478 0.0433 0.02% MLMG::MLResNormInf() 15 0.0001042 0.02472 0.0357 0.01% MLCellLinOp::reflux() 30 0.004218 0.01158 0.03343 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01888 0.02289 0.02903 0.01% MultiFab::Multiply() 8 0.02281 0.02482 0.02673 0.01% MLCellLinOp::defineBC() 11 0.02615 0.02631 0.02658 0.01% Castro::FluxRegFineAdd() 9 0.02512 0.02574 0.02613 0.01% StateDataPhysBCFunct::() 95 0.00197 0.006775 0.02049 0.01% Castro::build_fine_mask() 44 0.01716 0.01787 0.01912 0.01% MLMG::interpCorrection_1 17 0.01864 0.01886 0.01901 0.01% FabArrayBase::CPC::define() 416 0.01324 0.01485 0.01786 0.01% Castro::buildMetrics() 5 0.01447 0.01637 0.01712 0.01% Castro::Castro() 5 0.01525 0.01646 0.01701 0.01% Castro::linear_to_hybrid_momentum() 1 0.0141 0.01589 0.01652 0.01% MLLinOp::makeSubCommunicator() 11 0.0005786 0.009296 0.01586 0.01% FillPatchTwoLevels 31 0.01295 0.01357 0.01441 0.01% MLCellLinOp::prepareForSolve() 11 0.01154 0.01168 0.0119 0.00% Castro::enforce_consistent_e() 1 0.009441 0.01064 0.01109 0.00% Gravity::actual_multilevel_solve() 10 0.008428 0.00931 0.01054 0.00% AMRErrorTag::operator() 6 0.008896 0.00997 0.01054 0.00% FabArrayBase::getCPC() 1440 0.009948 0.01009 0.01047 0.00% Gravity::actual_solve_with_mlmg() 11 0.009285 0.009675 0.01008 0.00% Castro::finalize_do_advance() 9 0.007663 0.008713 0.009629 0.00% Gravity::update_max_rhs() 10 0.007638 0.008182 0.008786 0.00% Castro::apply_problem_tags() 6 0.006802 0.007594 0.007953 0.00% Castro::construct_old_gravity() 9 0.0004108 0.003784 0.007799 0.00% MLCellLinOp::setLevelBC() 19 0.004331 0.005437 0.006618 0.00% average_down_faces 48 0.00567 0.005702 0.005776 0.00% FabArray::FillBoundary() 1553 0.004326 0.004497 0.004771 0.00% Castro::do_new_sources() 9 0.0005459 0.002042 0.004548 0.00% AmrLevel::derive() 6 0.003774 0.0042 0.004395 0.00% MLCellLinOp::smooth() 688 0.003888 0.003968 0.004116 0.00% MLCGSolver::bicgstab 7 0 0.0009632 0.003853 0.00% FabArrayBase::getFB() 1553 0.003152 0.003317 0.003652 0.00% Castro::do_old_sources() 9 0.0005896 0.001648 0.00365 0.00% DistributionMapping::LeastUsedCPUs() 5 0.001332 0.002041 0.003546 0.00% Gravity::test_composite_phi() 1 0.002455 0.003092 0.003441 0.00% amrex::Dot() 115 0 0.000831 0.003324 0.00% FabArray::ParallelCopy() 830 0.002997 0.003066 0.003147 0.00% FabArrayBase::FB::FB() 19 0.00257 0.002851 0.003136 0.00% Castro::construct_new_gravity() 9 0.0005074 0.001146 0.0027 0.00% main() 1 0.001511 0.001577 0.001621 0.00% MLLinOp::defineGrids() 11 0.001304 0.001425 0.001574 0.00% MLCellLinOp::apply() 318 0.0008825 0.001065 0.001495 0.00% Amr::regrid() 5 0.0009816 0.001085 0.001344 0.00% Castro::clean_state() 87 0.001192 0.001263 0.001338 0.00% TagBoxArray::collate() 6 0.0004284 0.0009048 0.00127 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002834 0.001134 0.00% Amr::coarseTimeStep() 5 0.0004808 0.0009146 0.001062 0.00% AmrLevel::FillCoarsePatch() 5 0.0009143 0.0009278 0.0009359 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0009066 0.0009152 0.0009294 0.00% OwnerMask() 6 0.0007501 0.0008516 0.0009249 0.00% MLCellLinOp::correctionResidual() 189 0.0006934 0.0007084 0.0007322 0.00% MLMG::mgVcycle() 65 0.0005879 0.0005956 0.0006104 0.00% FillPatchSingleLevel 92 0.000502 0.0005569 0.000577 0.00% Amr::grid_places() 6 0.0004915 0.00052 0.0005734 0.00% Castro::do_advance_ctu() 5 0.0004787 0.000532 0.0005733 0.00% TagBoxArray::mapPRD 6 0.0004345 0.0004514 0.0004605 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003718 0.0004073 0.0004501 0.00% Castro::finalize_advance() 9 0.0002903 0.0003776 0.0004455 0.00% MLMG:computeResOfCorrection() 155 0.0003884 0.0003972 0.0004104 0.00% Castro::subcycle_advance_ctu() 5 6.487e-05 0.000148 0.0003901 0.00% MLCellLinOp::solutionResidual() 72 0.0003449 0.0003654 0.000387 0.00% MLMG::oneIter() 31 0.0003344 0.0003564 0.0003719 0.00% MLMG::actualBottomSolve() 31 4.071e-05 0.0001203 0.000357 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.000273 0.0003036 0.0003182 0.00% MLMG::mgVcycle_down::0 31 0.0002758 0.0002857 0.000299 0.00% Castro::enforce_speed_limit() 87 0.0002541 0.0002707 0.0002846 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002261 0.000245 0.0002634 0.00% Amr::defBaseLevel() 1 0.0001939 0.0002338 0.0002504 0.00% MLMG::mgVcycle_down::1 31 0.0002309 0.0002418 0.0002498 0.00% Castro::computeNewDt() 8 0.0001106 0.0001446 0.0002416 0.00% Amr::timeStep() 9 0.0001975 0.0002141 0.0002318 0.00% MLMG::mgVcycle_bottom 65 0.0002147 0.0002195 0.0002246 0.00% Castro::post_regrid() 10 0.0001996 0.0002088 0.0002232 0.00% MLMG::mgVcycle_up::0 31 0.0001858 0.0001932 0.0002017 0.00% AmrLevel::AmrLevel(dm) 5 0.0001741 0.0001871 0.0002008 0.00% MLMG::mgVcycle_down::2 31 0.0001962 0.0001987 0.0002002 0.00% MLMG::mgVcycle_down::3 31 0.0001911 0.0001952 0.0001999 0.00% FabArrayBase::TheFPinfo() 25 0.0001624 0.0001803 0.0001929 0.00% makeSFC 33 0.0001592 0.0001752 0.0001918 0.00% MLMG::mgVcycle_down::4 31 0.0001667 0.0001722 0.000185 0.00% Amr::FinalizeInit() 1 3.528e-05 7.641e-05 0.0001781 0.00% Castro::react_state() 18 0.0001222 0.0001383 0.0001546 0.00% Castro::check_for_nan() 18 0.0001272 0.0001378 0.0001514 0.00% MLMG::mgVcycle_up::1 31 0.0001347 0.0001398 0.0001471 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001311 0.0001363 0.0001458 0.00% FPinfo::FPinfo() 3 0.0001069 0.0001216 0.000142 0.00% MLMG::mgVcycle_up::4 31 0.0001139 0.000123 0.0001418 0.00% MLMG::mgVcycle_up::2 31 0.0001292 0.0001321 0.000135 0.00% MLMG::mgVcycle_up::3 31 0.000124 0.0001273 0.0001329 0.00% Castro::apply_source_to_state() 18 9.583e-05 0.0001078 0.0001255 0.00% DistributionMapping::Distribute() 35 0.0001052 0.000114 0.0001225 0.00% Castro::expand_state() 13 0.000112 0.0001169 0.0001218 0.00% MLPoisson::define() 11 0.000103 0.0001092 0.0001182 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001058 0.0001122 0.0001167 0.00% Castro::errorEst() 6 0.0001047 0.0001075 0.0001098 0.00% Amr::writeSmallPlotFile() 1 8.945e-05 9.973e-05 0.0001098 0.00% FabArray::RecvLayoutMask() 8 9.359e-05 9.935e-05 0.0001091 0.00% Castro::advance() 5 7.486e-05 8.217e-05 9.656e-05 0.00% Gravity::swapTimeLevels() 9 8.629e-05 9.086e-05 9.379e-05 0.00% Castro::avgDown(state_indx) 20 6.683e-05 7.456e-05 8.907e-05 0.00% MLMG::computeResidual() 31 8.489e-05 8.592e-05 8.68e-05 0.00% MLMG::computeMLResidual() 15 7.579e-05 7.971e-05 8.476e-05 0.00% Castro::post_timestep() 9 7.861e-05 8.07e-05 8.382e-05 0.00% ClusterList::intersect() 1 0 2.068e-05 8.273e-05 0.00% Castro::create_source_corrector() 9 5.151e-05 6.254e-05 8.186e-05 0.00% MLLinOp::define() 11 6.476e-05 7.4e-05 7.947e-05 0.00% knapsack() 3 6.302e-05 6.797e-05 7.519e-05 0.00% MLMG::getGradSolution() 10 6.237e-05 6.712e-05 7.203e-05 0.00% MLMG::miniCycle() 34 6.597e-05 6.822e-05 7.025e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.175e-05 6.421e-05 6.593e-05 0.00% amrex::average_down_w_geom 20 4.999e-05 5.402e-05 5.764e-05 0.00% Castro::construct_new_hybrid_source() 9 4.41e-05 4.86e-05 5.29e-05 0.00% Amr::initSubcycle() 1 4.995e-05 5.182e-05 5.259e-05 0.00% MLPoisson::prepareForSolve() 11 4.564e-05 4.786e-05 5e-05 0.00% Castro::init(old) 3 4.393e-05 4.664e-05 4.92e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 3.882e-05 4.386e-05 4.832e-05 0.00% Gravity::set_mass_offset() 6 3.665e-05 4.137e-05 4.446e-05 0.00% BoxList::parallelComplementIn() 12 2.853e-05 3.67e-05 4.355e-05 0.00% Castro::construct_old_rotation_source() 9 2.086e-05 2.728e-05 4.14e-05 0.00% Castro::retry_advance_ctu() 5 2.222e-05 2.857e-05 4.054e-05 0.00% Amr::bldFineLevels() 1 2.195e-05 2.9e-05 3.968e-05 0.00% Castro::computeInitialDt() 2 1.464e-05 2.256e-05 3.857e-05 0.00% Castro::construct_old_hybrid_source() 9 3.318e-05 3.552e-05 3.808e-05 0.00% AmrMesh-cluster 1 0 9.257e-06 3.703e-05 0.00% Castro::post_init() 1 2.375e-05 2.936e-05 3.632e-05 0.00% Castro::avgDown() 4 2.994e-05 3.23e-05 3.561e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.927e-05 3.138e-05 3.487e-05 0.00% MLMG::MLRhsNormInf() 10 3.183e-05 3.294e-05 3.485e-05 0.00% MLMG::compResidual() 1 2.677e-05 2.764e-05 2.826e-05 0.00% Castro::construct_new_rotation_source() 9 1.95e-05 2.185e-05 2.633e-05 0.00% BoxList::complementIn 12 2.371e-05 2.455e-05 2.547e-05 0.00% Castro::derive() 6 1.928e-05 2.122e-05 2.457e-05 0.00% Castro::init() 1 1.567e-05 1.656e-05 1.725e-05 0.00% ClusterList::chop() 1 0 2.687e-06 1.075e-05 0.00% Amr::InitializeInit() 1 8.25e-06 9.137e-06 1.047e-05 0.00% Amr::init() 1 6.494e-06 7.398e-06 9.773e-06 0.00% Amr::initialInit() 1 4.216e-06 5.454e-06 6.374e-06 0.00% knapsack()swap 1 3.729e-06 4.204e-06 4.585e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 240.7 240.7 240.7 100.00% Amr::coarseTimeStep() 5 228.6 228.6 228.6 94.94% Amr::timeStep() 9 227.3 227.3 227.3 94.40% Castro::advance() 5 199.9 199.9 199.9 83.03% Castro::subcycle_advance_ctu() 5 191.2 191.6 192.5 79.94% Castro::do_advance_ctu() 5 191.2 191.6 192.5 79.94% Castro::construct_ctu_hydro_source() 9 117 117.1 117.1 48.66% Castro::clean_state() 87 71.83 80.18 84.03 34.90% Castro::computeTemp() 88 45.59 50.91 53.33 22.15% Castro::reset_internal_energy(MultiFab) 88 30.72 34.31 35.94 14.93% Castro::evaluate_source_change() 193 17.93 19.84 20.69 8.59% Castro::enforce_min_density() 87 16.95 18.9 19.89 8.26% Castro::post_regrid() 10 15.97 17.45 18.13 7.53% Amr::regrid() 5 15.85 17.18 17.78 7.38% Castro::initialize_do_advance() 9 14.82 15.17 15.38 6.39% amrex::Copy() 679 13.07 14.58 15.26 6.34% AmrLevel::FillPatch() 62 8.111 9.846 13.96 5.80% check_for_negative_density() 9 0.285 4.622 13.83 5.75% FabArray::ParallelCopy() 830 4.113 6.748 12.84 5.33% FillPatchIterator::Initialize 62 6.377 8.184 12.48 5.18% MultiFab::AddProduct() 4205 10.64 11.79 12.29 5.10% Castro::normalize_species() 87 10.08 11.26 11.75 4.88% Gravity::solve_phi_with_mlmg() 11 11.34 11.48 11.7 4.86% Gravity::multilevel_solve_for_new_phi() 10 11.05 11.18 11.39 4.73% Gravity::actual_multilevel_solve() 10 11 11.14 11.36 4.72% Castro::do_old_sources() 9 11.08 11.13 11.2 4.65% Castro::do_new_sources() 9 9.188 10.28 10.75 4.47% FabArray::ParallelCopy_finish() 830 1.389 4.399 10.61 4.41% FillPatchSingleLevel 92 4.176 6.045 10.46 4.34% Amr::init() 1 10.43 10.43 10.43 4.33% FillPatchIterator::FillFromLevel0() 31 3.841 5.666 10.06 4.18% Castro::post_timestep() 9 10.03 10.03 10.03 4.17% Amr::initialInit() 1 9.895 9.895 9.896 4.11% Castro::reset_internal_energy(Fab) 1476 8.437 9.272 9.646 4.01% Castro::reflux() 4 8.94 9.001 9.028 3.75% Castro::estTimeStep() 26 3.963 5.447 8.811 3.66% Castro::initialize_advance() 9 7.429 8.247 8.706 3.62% FabArray::setVal() 5070 7.172 7.99 8.37 3.48% MultiFab::Subtract() 175 6.813 7.583 7.923 3.29% Gravity::actual_solve_with_mlmg() 11 6.912 6.922 6.937 2.88% Castro::construct_new_gravity() 9 6.357 6.483 6.691 2.78% Castro::expand_state() 13 5.305 5.638 6.455 2.68% MLMG::solve() 10 6.372 6.387 6.406 2.66% Amr::FinalizeInit() 1 5.774 5.961 6.309 2.62% MLMG::oneIter() 31 5.162 5.181 5.19 2.16% Gravity::fill_multipole_BCs() 11 4.402 4.538 4.761 1.98% FluxRegister::Reflux() 24 4.378 4.398 4.414 1.83% MLMG::mgVcycle() 65 4.328 4.357 4.367 1.81% Castro::construct_old_gravity() 9 0.7014 1.826 4.165 1.73% Gravity::get_old_grav_vector() 9 0.6936 1.822 4.165 1.73% Amr::InitializeInit() 1 3.586 3.934 4.122 1.71% Amr::defBaseLevel() 1 3.586 3.934 4.122 1.71% FabArray::sum() 4205 3.496 3.867 4.032 1.67% Castro::finalize_do_advance() 9 1.349 2.125 4.001 1.66% Castro::problem_post_timestep() 10 2.701 3.089 3.815 1.58% Castro::computeNewDt() 8 2.126 2.645 3.784 1.57% MLCellLinOp::smooth() 688 3.298 3.31 3.336 1.39% Castro::post_init() 1 2.567 2.73 3.063 1.27% FabArray::ParallelCopy_nowait() 830 2.156 2.346 2.72 1.13% Castro::FluxRegCrseInit 9 1.828 2.108 2.676 1.11% Castro::initData() 1 2.216 2.5 2.624 1.09% MLPoisson::Fsmooth() 1376 2.149 2.387 2.484 1.03% Castro::wd_update() 6 1.258 1.646 2.372 0.99% Castro::construct_new_source() 63 1.75 1.974 2.069 0.86% MLMG::mgVcycle_down::0 31 1.915 1.979 2.009 0.83% MLCellLinOp::applyBC() 1742 1.149 1.303 1.691 0.70% Amr::bldFineLevels() 1 1.271 1.457 1.538 0.64% FillPatchIterator::FillFromTwoLevels() 31 1.454 1.474 1.484 0.62% FillPatchTwoLevels 31 1.454 1.473 1.484 0.62% Castro::Castro() 5 1.344 1.393 1.448 0.60% MLMG::mgVcycle_up::0 31 1.395 1.421 1.431 0.59% FillPatchInterp(MF) 30 1.258 1.322 1.401 0.58% MLCellLinOp::apply() 318 1.287 1.316 1.347 0.56% FabArray::FillBoundary() 1553 0.7018 0.8814 1.328 0.55% Amr::InitAmr() 1 1.208 1.208 1.208 0.50% amrex::Add() 136 1.008 1.109 1.154 0.48% Castro::initMFs() 5 1.09 1.122 1.137 0.47% MLPoisson::Fapply() 318 0.9108 1.016 1.072 0.45% Amr::writePlotFile() 2 1.069 1.069 1.069 0.44% Castro::construct_old_source() 63 0.907 1.017 1.062 0.44% MLCellLinOp::solutionResidual() 72 0.9682 1.023 1.053 0.44% Castro::construct_old_gravity_source() 18 0.8924 0.999 1.045 0.43% Castro::computeInitialDt() 2 0.4982 0.6855 1.034 0.43% Castro::construct_new_gravity_source() 9 0.7374 0.8429 0.8853 0.37% FillBoundary_finish() 1553 0.3293 0.479 0.8719 0.36% MLCellLinOp::correctionResidual() 189 0.8121 0.8183 0.8313 0.35% FabArray::Saxpy() 182 0.6883 0.7677 0.8051 0.33% Gravity::get_new_grav_vector() 10 0.7827 0.79 0.7975 0.33% Castro::volInBoundary() 42 0.6946 0.7346 0.7709 0.32% MLMG:computeResOfCorrection() 155 0.659 0.6915 0.7151 0.30% VisMF::Write(FabArray) 3 0.5837 0.6414 0.6825 0.28% StateDataPhysBCFunct::() 95 0.5921 0.6075 0.6169 0.26% StateData::FillBoundary(geom) 50 0.5901 0.6007 0.6143 0.26% MLMG::computeResWithCrseSolFineCor() 17 0.5806 0.5815 0.5833 0.24% Gravity::update_max_rhs() 10 0.08933 0.252 0.5831 0.24% MultiFab::max() 18 0.02538 0.1904 0.526 0.22% MLMG::computeMLResidual() 15 0.4121 0.435 0.494 0.21% Castro::init() 1 0.4777 0.4812 0.4833 0.20% AmrLevel::FillCoarsePatch() 5 0.4777 0.4812 0.4833 0.20% Castro::check_for_nan() 18 0.4023 0.4471 0.4667 0.19% MultiFab::contains_nan() 18 0.4022 0.4469 0.4666 0.19% FillBoundary_nowait() 1553 0.3634 0.3979 0.4515 0.19% Gravity::test_composite_phi() 1 0.4215 0.4237 0.4268 0.18% FabArray::Xpay() 283 0.3508 0.392 0.4159 0.17% amrex::average_down 233 0.2762 0.3622 0.4023 0.17% MLMG::mgVcycle_down::1 31 0.3056 0.3332 0.3914 0.16% MLMG::computeResidual() 31 0.3492 0.3581 0.3669 0.15% MLCellLinOp::reflux() 30 0.1792 0.2304 0.3655 0.15% Castro::init(old) 3 0.3502 0.3523 0.3536 0.15% Castro::apply_source_to_state() 18 0.2762 0.3081 0.3228 0.13% MLPoisson::define() 11 0.2853 0.296 0.3066 0.13% Castro::react_state() 18 0.2196 0.2459 0.263 0.11% FabArray::setDomainBndry() 92 0.194 0.2164 0.2288 0.10% MLMG::prepareForSolve() 10 0.2141 0.2237 0.2285 0.09% MLMG::mgVcycle_up::1 31 0.2212 0.225 0.2268 0.09% Castro::fill_hybrid_hydro_source() 27 0.1706 0.1936 0.2142 0.09% MLMG::getGradSolution() 10 0.1982 0.2032 0.2113 0.09% MLCellLinOp::compGrad() 18 0.1981 0.2031 0.2112 0.09% MLCellLinOp::defineAuxData() 11 0.1809 0.1977 0.2104 0.09% Amr::grid_places() 6 0.1944 0.1945 0.1945 0.08% AmrMesh::MakeNewGrids() 6 0.1939 0.194 0.194 0.08% MLMG::addInterpCorrection() 155 0.1326 0.1665 0.1841 0.08% MLMG::mgVcycle_bottom 65 0.1518 0.159 0.1793 0.07% MLPoisson::FFlux() 150 0.004617 0.04311 0.1583 0.07% MLMG::miniCycle() 34 0.1518 0.1522 0.1527 0.06% Castro::construct_new_hybrid_source() 9 0.1134 0.1286 0.1423 0.06% FabArray::mult() 56 0.1172 0.1314 0.1386 0.06% Castro::avgDown() 4 0.1214 0.1243 0.1265 0.05% Castro::avgDown(state_indx) 20 0.1214 0.1243 0.1265 0.05% amrex::average_down_w_geom 20 0.1213 0.1242 0.1264 0.05% AmrLevel::AmrLevel(dm) 5 0.108 0.1188 0.1236 0.05% StateData::define() 25 0.1078 0.1186 0.1234 0.05% Castro::errorEst() 6 0.106 0.1174 0.1229 0.05% MLMG::ResNormInf() 54 0.07663 0.09305 0.1169 0.05% MLCellLinOp::updateSolBC() 25 0.07886 0.08433 0.09482 0.04% Castro::swap_state_time_levels() 9 0.08609 0.09097 0.09457 0.04% MLCellLinOp::defineBC() 11 0.08479 0.08702 0.08935 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08598 0.08688 0.08925 0.04% MLMG::mgVcycle_down::2 31 0.06965 0.0731 0.07918 0.03% MLMG::mgVcycle_up::3 31 0.02549 0.06447 0.07783 0.03% MLMG::MLResNormInf() 15 0.04393 0.06683 0.07502 0.03% Castro::construct_old_hybrid_source() 9 0.05738 0.06508 0.07205 0.03% FabArray::norminf(mask) 21 0.05343 0.05982 0.06258 0.03% BndryData::define() 27 0.05742 0.05948 0.06155 0.03% MLMG::interpCorrection_1 17 0.02691 0.03792 0.05872 0.02% Castro::apply_tagging_restrictions() 6 0.04844 0.05379 0.05646 0.02% MLMG::mgVcycle_up::2 31 0.05192 0.05293 0.05394 0.02% MLCellLinOp::updateCorBC() 17 0.0497 0.05054 0.05269 0.02% Castro::buildMetrics() 5 0.03589 0.04158 0.05025 0.02% Castro::derive() 6 0.04173 0.04595 0.04781 0.02% AmrLevel::derive() 6 0.04171 0.04593 0.04779 0.02% FabArray::norminf() 147 0.04069 0.04314 0.04604 0.02% Gravity::swapTimeLevels() 9 0.0338 0.03568 0.03904 0.02% MLMG::mgVcycle_down::3 31 0.03116 0.03227 0.03458 0.01% MLMG::MLRhsNormInf() 10 0.02496 0.02774 0.02915 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01888 0.02289 0.02903 0.01% TagBoxArray::mapPRD 6 0.004339 0.01385 0.02873 0.01% FabArrayBase::getCPC() 1440 0.02371 0.02495 0.02781 0.01% MLMG::actualBottomSolve() 31 4.071e-05 0.006894 0.02745 0.01% MLCGSolver::bicgstab 7 0 0.006689 0.02675 0.01% MultiFab::Multiply() 8 0.02281 0.02482 0.02673 0.01% Castro::FluxRegFineAdd() 9 0.02512 0.02574 0.02613 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01935 0.02134 0.02222 0.01% MLMG::compResidual() 1 0.01808 0.02034 0.02221 0.01% Castro::build_fine_mask() 44 0.01716 0.01787 0.01912 0.01% MLMG::mgVcycle_down::4 31 0.005439 0.008851 0.01858 0.01% FabArrayBase::CPC::define() 416 0.01324 0.01485 0.01786 0.01% MLLinOp::define() 11 0.002325 0.01119 0.01784 0.01% MLLinOp::defineGrids() 11 0.00226 0.01111 0.01777 0.01% Castro::linear_to_hybrid_momentum() 1 0.0141 0.01589 0.01652 0.01% MLLinOp::makeSubCommunicator() 11 0.0005786 0.009296 0.01586 0.01% MLMG::mgVcycle_up::4 31 0.004452 0.00692 0.01396 0.01% average_down_faces 48 0.008913 0.01056 0.01212 0.01% MLPoisson::prepareForSolve() 11 0.01159 0.01173 0.01195 0.00% MLCellLinOp::prepareForSolve() 11 0.01154 0.01168 0.0119 0.00% Castro::enforce_consistent_e() 1 0.009441 0.01064 0.01109 0.00% AMRErrorTag::operator() 6 0.008896 0.00997 0.01054 0.00% Castro::apply_problem_tags() 6 0.006802 0.007594 0.007953 0.00% MLCellLinOp::setLevelBC() 19 0.004331 0.005437 0.006618 0.00% FabArrayBase::getFB() 1553 0.005843 0.006168 0.006598 0.00% FabArrayBase::TheFPinfo() 25 0.001666 0.002371 0.003932 0.00% FPinfo::FPinfo() 3 0.001482 0.00219 0.003739 0.00% DistributionMapping::KnapSackDoIt() 3 0.001375 0.002069 0.003597 0.00% DistributionMapping::LeastUsedCPUs() 5 0.001332 0.002041 0.003546 0.00% amrex::Dot() 115 0 0.000831 0.003324 0.00% FabArrayBase::FB::FB() 19 0.00257 0.002851 0.003136 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007215 0.002886 0.00% TagBoxArray::collate() 6 0.0004284 0.0009048 0.00127 0.00% OwnerMask() 6 0.0007501 0.0008516 0.0009249 0.00% Castro::finalize_advance() 9 0.0002903 0.0003776 0.0004455 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003614 0.0003905 0.0004182 0.00% makeSFC 33 0.0002556 0.0002783 0.0003015 0.00% Castro::enforce_speed_limit() 87 0.0002541 0.0002707 0.0002846 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001053 0.0001302 0.0001817 0.00% AmrMesh-cluster 1 0 3.263e-05 0.0001305 0.00% DistributionMapping::Distribute() 35 0.0001052 0.000114 0.0001225 0.00% Amr::writeSmallPlotFile() 1 8.945e-05 9.973e-05 0.0001098 0.00% FabArray::RecvLayoutMask() 8 9.359e-05 9.935e-05 0.0001091 0.00% ClusterList::intersect() 1 0 2.068e-05 8.273e-05 0.00% Castro::create_source_corrector() 9 5.151e-05 6.254e-05 8.186e-05 0.00% knapsack() 3 6.753e-05 7.217e-05 7.977e-05 0.00% BoxList::parallelComplementIn() 12 5.32e-05 6.125e-05 6.901e-05 0.00% Amr::initSubcycle() 1 4.995e-05 5.182e-05 5.259e-05 0.00% Gravity::set_mass_offset() 6 3.665e-05 4.137e-05 4.446e-05 0.00% Castro::construct_old_rotation_source() 9 2.086e-05 2.728e-05 4.14e-05 0.00% Castro::retry_advance_ctu() 5 2.222e-05 2.857e-05 4.054e-05 0.00% Castro::construct_new_rotation_source() 9 1.95e-05 2.185e-05 2.633e-05 0.00% BoxList::complementIn 12 2.371e-05 2.455e-05 2.547e-05 0.00% ClusterList::chop() 1 0 2.687e-06 1.075e-05 0.00% knapsack()swap 1 3.729e-06 4.204e-06 4.585e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (24.04-2-gf2536be82d98) finalized