Initializing AMReX (v2024-42-g7437aa17d36f)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-42-g7437aa17d36f) initialized Starting run at 09:58:39 UTC on 2024-03-28. Successfully read inputs file ... Castro git describe: 24.03-37-g6b90dd1c0 AMReX git describe: v2024-42-g7437aa17d Microphysics git describe: 24.03-23-g3112c13a reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.032173818 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.031467411 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.36111289 Gravity::multilevel_solve_for_new_phi() time = 1.74874813 Gravity::fill_multipole_BCs() time = 0.369287639 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.542460395 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.838496024 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.401884567 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.23940269 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.616773148 Gravity::multilevel_solve_for_new_phi() time = 1.669961787 Castro::construct_new_gravity() time = 1.788050015 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730191e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.308306803 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02444397803 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.38459225 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.032194643 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402947199 Gravity::multilevel_solve_for_new_phi() time = 1.911547322 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.604044608 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.049402151 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 22.21575265 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.040402931 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.570257339e+22 ymom added: 1.475739526e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.198831636 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.764294981 on level 1 Castro::reflux() at level 0 : time = 2.242009487 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.424195902 Gravity::multilevel_solve_for_new_phi() time = 1.358268905 Castro::construct_new_gravity() time = 1.472018383 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.982635458 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.030162457 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.76500649e+18 ymom added: -1.361438167e+19 zmom added: -8.219114503e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.142991575 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02244228337 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 50.34899101 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031777971 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.419020274 Gravity::multilevel_solve_for_new_phi() time = 0.549748683 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.756211243 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.046781109 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.23716637 on level 0 Castro::construct_old_gravity() time = 0.043101006 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.442864444e+22 ymom added: 4.427218578e+21 zmom added: 7.103293505e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.206169334 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.765574568 on level 1 Castro::reflux() at level 0 : time = 2.258359632 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.429894245 Gravity::multilevel_solve_for_new_phi() time = 0.975290022 Castro::construct_new_gravity() time = 1.087491333 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.98895549 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.029482333 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.818790753e+17 ymom added: 2.351329365e+19 zmom added: 9.539391635e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.146877064 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02322650398 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.17210205 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.031752315 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.404263027 Gravity::multilevel_solve_for_new_phi() time = 0.53249884 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.713844045 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 1.963044371 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.19635767 on level 0 Castro::construct_old_gravity() time = 0.030621066 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.20239123e+22 ymom added: 1.180591621e+22 zmom added: 8.023901326e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.194895765 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.768712911 on level 1 Castro::reflux() at level 0 : time = 2.218538076 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.424076199 Gravity::multilevel_solve_for_new_phi() time = 0.968882412 Castro::construct_new_gravity() time = 1.082848839 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.955422058 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.03031617 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598448e+18 ymom added: -2.567952508e+19 zmom added: -1.336957651e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.141121569 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.023459337 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 46.67026443 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.032029814 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.409466998 Gravity::multilevel_solve_for_new_phi() time = 0.538772306 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.784119307 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.014014871 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.41827008 on level 0 Castro::construct_old_gravity() time = 0.032507256 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013626193e+23 ymom added: 1.888946593e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.19712902 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.824039044 on level 1 Castro::reflux() at level 0 : time = 2.273374527 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.429123589 Gravity::multilevel_solve_for_new_phi() time = 0.974474595 Castro::construct_new_gravity() time = 1.088020809 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171514e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.96071492 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029226675 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.726853775e+18 ymom added: -2.54093091e+19 zmom added: -1.252338095e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.136731643 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02321372918 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.09499794 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.538968034 seconds Ending run at 10:02:40 UTC on 2024-03-28. Run time = 241.7281471 Run time without initialization = 230.2122529 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 241.7 ... 241.7 ... 241.7 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 92.8181 103.3602 108.0760 44.71% Castro::computeTemp() 88 14.66 16.45 17.14 7.09% check_for_negative_density() 9 0.2248 4.969 15.58 6.44% amrex::Copy() 679 12.57 14.12 14.84 6.14% MultiFab::AddProduct() 4205 10.48 11.72 12.23 5.06% Castro::normalize_species() 87 10.12 11.22 11.67 4.83% FabArray::ParallelCopy_finish() 830 1.468 4.124 10.81 4.47% Castro::reset_internal_energy(Fab) 1476 8.265 9.178 9.52 3.94% Castro::estTimeStep() 26 4.057 5.458 9.047 3.74% FabArray::setVal() 5070 7.445 8.348 8.771 3.63% MultiFab::Subtract() 175 6.99 7.835 8.208 3.40% Castro::reset_internal_energy(MultiFab) 88 3.287 4.081 4.866 2.01% Gravity::fill_multipole_BCs() 11 4.279 4.401 4.612 1.91% FabArray::sum() 4205 3.499 3.892 4.044 1.67% FluxRegister::Reflux() 24 2.949 3.299 3.444 1.42% Castro::enforce_min_density() 87 1.498 1.978 2.69 1.11% FabArray::ParallelCopy_nowait() 830 2.049 2.268 2.658 1.10% MLPoisson::Fsmooth() 1376 2.116 2.367 2.456 1.02% Castro::wd_update() 6 0.5783 0.9421 1.822 0.75% Castro::FluxRegCrseInit 9 1.495 1.665 1.788 0.74% Castro::reflux() 4 1.475 1.664 1.74 0.72% FillPatchInterp(MF) 30 1.256 1.367 1.473 0.61% Amr::InitAmr() 1 1.153 1.153 1.154 0.48% amrex::Add() 136 0.9804 1.09 1.144 0.47% Castro::initMFs() 5 1.07 1.109 1.13 0.47% Castro::construct_old_gravity_source() 18 0.8973 1.009 1.072 0.44% MLPoisson::Fapply() 318 0.9231 1.025 1.064 0.44% Castro::construct_new_gravity_source() 9 0.7507 0.8464 0.917 0.38% Castro::problem_post_timestep() 10 0.7878 0.8285 0.9054 0.37% Castro::volInBoundary() 42 0.6565 0.7507 0.8734 0.36% FillPatchIterator::Initialize 62 0.7408 0.8257 0.8732 0.36% FillBoundary_finish() 1553 0.3041 0.4415 0.8419 0.35% FabArray::Saxpy() 182 0.7141 0.7977 0.8363 0.35% VisMF::Write(FabArray) 3 0.5733 0.6556 0.707 0.29% StateData::FillBoundary(geom) 50 0.584 0.5966 0.6085 0.25% Castro::construct_new_source() 63 0.4729 0.5258 0.5481 0.23% MultiFab::max() 18 0.03381 0.1804 0.515 0.21% MultiFab::contains_nan() 18 0.398 0.4422 0.4605 0.19% MLCellLinOp::applyBC() 1742 0.3552 0.4167 0.4534 0.19% FillBoundary_nowait() 1553 0.3384 0.3781 0.4318 0.18% FabArray::Xpay() 283 0.3454 0.3855 0.3993 0.17% Castro::initialize_do_advance() 9 0.2335 0.2591 0.2787 0.12% Amr::writePlotFile() 2 0.09266 0.1459 0.2543 0.11% FabArray::setDomainBndry() 92 0.1958 0.2175 0.2296 0.09% amrex::average_down 233 0.1974 0.2071 0.2121 0.09% Castro::construct_old_source() 63 0.1789 0.1958 0.2064 0.09% Castro::fill_hybrid_hydro_source() 27 0.1724 0.1912 0.1985 0.08% MLCellLinOp::defineAuxData() 11 0.1579 0.1718 0.1776 0.07% MLPoisson::FFlux() 150 0.004642 0.0423 0.155 0.06% Castro::initData() 1 0.1288 0.1438 0.1535 0.06% MLCellLinOp::compGrad() 18 0.1232 0.1378 0.144 0.06% FabArray::mult() 56 0.1145 0.1289 0.1382 0.06% MLMG::addInterpCorrection() 155 0.1093 0.1226 0.1299 0.05% StateData::define() 25 0.1067 0.1176 0.1237 0.05% Castro::evaluate_source_change() 193 0.1018 0.1107 0.1158 0.05% Castro::swap_state_time_levels() 9 0.08402 0.09393 0.1035 0.04% Gravity::get_new_grav_vector() 10 0.08931 0.09704 0.101 0.04% Gravity::get_old_grav_vector() 9 0.07934 0.08995 0.09787 0.04% Castro::initialize_advance() 9 0.06611 0.08052 0.09333 0.04% MLMG::prepareForSolve() 10 0.06118 0.06745 0.0744 0.03% MLCellLinOp::updateSolBC() 25 0.06671 0.0676 0.06888 0.03% MLMG::solve() 10 0.0006877 0.0423 0.06654 0.03% FabArray::norminf(mask) 21 0.0574 0.06284 0.06499 0.03% Castro::apply_tagging_restrictions() 6 0.04886 0.05406 0.05721 0.02% AmrLevel::FillPatch() 62 0.05308 0.05388 0.0561 0.02% AmrMesh::MakeNewGrids() 6 0.0472 0.04935 0.05334 0.02% MLCellLinOp::updateCorBC() 17 0.04543 0.04593 0.04683 0.02% MLMG::ResNormInf() 54 0.001788 0.01592 0.04588 0.02% BndryData::define() 27 0.04203 0.04254 0.04322 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.02998 0.03543 0.04203 0.02% FabArray::norminf() 147 0.03458 0.03847 0.04196 0.02% MLMG::MLResNormInf() 15 0.0001112 0.02384 0.03388 0.01% MLCellLinOp::reflux() 30 0.004264 0.01165 0.03338 0.01% StateDataPhysBCFunct::() 95 0.002251 0.009242 0.03007 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01936 0.02359 0.02935 0.01% Castro::FluxRegFineAdd() 9 0.02566 0.02594 0.02645 0.01% MLCellLinOp::defineBC() 11 0.02571 0.02601 0.02622 0.01% MultiFab::Multiply() 8 0.02361 0.02483 0.02531 0.01% MLMG::interpCorrection_1 17 0.01869 0.01881 0.01906 0.01% Castro::build_fine_mask() 44 0.01712 0.01805 0.01891 0.01% MLLinOp::makeSubCommunicator() 11 0.0006009 0.009884 0.01857 0.01% FabArrayBase::CPC::define() 416 0.01291 0.01474 0.01781 0.01% Castro::do_old_sources() 9 0.0006282 0.007795 0.01765 0.01% Castro::do_new_sources() 9 0.0005905 0.005107 0.01754 0.01% Castro::Castro() 5 0.01469 0.01644 0.01718 0.01% Castro::linear_to_hybrid_momentum() 1 0.01411 0.01576 0.01712 0.01% Castro::buildMetrics() 5 0.0145 0.0162 0.01687 0.01% FillPatchTwoLevels 31 0.01345 0.01409 0.0152 0.01% DistributionMapping::LeastUsedCPUs() 5 0.0009033 0.004326 0.01237 0.01% MLCellLinOp::prepareForSolve() 11 0.01149 0.01167 0.01188 0.00% Castro::enforce_consistent_e() 1 0.009694 0.0105 0.01079 0.00% AMRErrorTag::operator() 6 0.0088 0.009915 0.01041 0.00% FabArrayBase::getCPC() 1440 0.00992 0.0101 0.0104 0.00% Gravity::update_max_rhs() 10 0.007541 0.008651 0.01038 0.00% Gravity::actual_multilevel_solve() 10 0.008053 0.00913 0.01012 0.00% Castro::finalize_do_advance() 9 0.007515 0.008638 0.00964 0.00% Gravity::actual_solve_with_mlmg() 11 0.00938 0.009523 0.00962 0.00% Castro::apply_problem_tags() 6 0.006267 0.007059 0.007335 0.00% Castro::construct_old_gravity() 9 0.0004462 0.002122 0.006586 0.00% MLCellLinOp::setLevelBC() 19 0.003654 0.004674 0.005856 0.00% average_down_faces 48 0.00572 0.005788 0.005817 0.00% FabArray::FillBoundary() 1553 0.004216 0.004405 0.004774 0.00% AmrLevel::derive() 6 0.004199 0.004244 0.004304 0.00% MLCellLinOp::smooth() 688 0.003737 0.003866 0.003957 0.00% Gravity::test_composite_phi() 1 0.002963 0.003347 0.003831 0.00% MLCGSolver::bicgstab 7 0 0.0009396 0.003759 0.00% FabArrayBase::getFB() 1553 0.00313 0.00329 0.00365 0.00% amrex::Dot() 115 0 0.000851 0.003404 0.00% FabArrayBase::FB::FB() 19 0.00245 0.002827 0.003163 0.00% FabArray::ParallelCopy() 830 0.002954 0.002997 0.00304 0.00% main() 1 0.001506 0.001868 0.002228 0.00% MLLinOp::defineGrids() 11 0.001302 0.001404 0.001538 0.00% MLCellLinOp::apply() 318 0.0008645 0.001053 0.001491 0.00% Amr::regrid() 5 0.0009406 0.001091 0.001363 0.00% Castro::clean_state() 87 0.001256 0.001304 0.001362 0.00% TagBoxArray::collate() 6 0.0005333 0.0009092 0.001256 0.00% AmrLevel::FillCoarsePatch() 5 0.0009274 0.0009922 0.001178 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002903 0.001161 0.00% Amr::coarseTimeStep() 5 0.0004978 0.0009497 0.001109 0.00% OwnerMask() 6 0.000719 0.0008478 0.0009185 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008889 0.0008925 0.0008968 0.00% Castro::construct_new_gravity() 9 0.0004582 0.0005862 0.0008661 0.00% MLCellLinOp::correctionResidual() 189 0.0006893 0.0007011 0.0007134 0.00% MLMG::mgVcycle() 65 0.0005907 0.0006178 0.0006403 0.00% Castro::do_advance_ctu() 5 0.0004978 0.0005436 0.0006158 0.00% Amr::grid_places() 6 0.0004267 0.0005213 0.0005935 0.00% FillPatchSingleLevel 92 0.000535 0.000542 0.0005453 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003713 0.0004168 0.0004689 0.00% TagBoxArray::mapPRD 6 0.0004228 0.0004372 0.0004495 0.00% Castro::subcycle_advance_ctu() 5 6.807e-05 0.0001654 0.0004318 0.00% Amr::timeStep() 9 0.0001872 0.0002663 0.0004218 0.00% Castro::finalize_advance() 9 0.0003632 0.0003795 0.0004096 0.00% Amr::FinalizeInit() 1 3.276e-05 0.0001248 0.0003964 0.00% MLMG:computeResOfCorrection() 155 0.0003846 0.0003879 0.0003903 0.00% MLCellLinOp::solutionResidual() 72 0.0003548 0.000365 0.0003693 0.00% MLMG::oneIter() 31 0.0003332 0.0003405 0.0003473 0.00% MLMG::actualBottomSolve() 31 4.17e-05 0.000119 0.0003408 0.00% MLMG::mgVcycle_down::0 31 0.0002837 0.0002951 0.00031 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002891 0.0002949 0.0002976 0.00% Castro::computeNewDt() 8 0.0001215 0.0001714 0.0002851 0.00% Castro::enforce_speed_limit() 87 0.0002523 0.000257 0.0002612 0.00% Gravity::solve_phi_with_mlmg() 11 0.000223 0.0002385 0.000261 0.00% Amr::defBaseLevel() 1 0.0002317 0.0002459 0.0002587 0.00% Castro::post_regrid() 10 0.0001947 0.0002139 0.0002578 0.00% MLMG::mgVcycle_down::1 31 0.0002304 0.0002394 0.0002448 0.00% MLMG::mgVcycle_bottom 65 0.0002029 0.0002068 0.0002119 0.00% MLMG::mgVcycle_down::2 31 0.0001863 0.0001925 0.0002031 0.00% makeSFC 33 0.000162 0.000179 0.0002013 0.00% AmrLevel::AmrLevel(dm) 5 0.000193 0.0001976 0.0002004 0.00% MLMG::mgVcycle_up::0 31 0.0001875 0.0001897 0.0001963 0.00% MLMG::mgVcycle_down::3 31 0.0001847 0.0001879 0.000194 0.00% FabArrayBase::TheFPinfo() 25 0.0001681 0.0001718 0.0001786 0.00% MLMG::mgVcycle_down::4 31 0.000162 0.0001684 0.000174 0.00% Castro::react_state() 18 0.0001387 0.0001537 0.0001619 0.00% Castro::check_for_nan() 18 0.0001383 0.000146 0.0001533 0.00% MLMG::mgVcycle_up::1 31 0.0001332 0.0001372 0.0001404 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001297 0.0001323 0.0001372 0.00% MLMG::mgVcycle_up::2 31 0.0001267 0.0001286 0.0001318 0.00% FPinfo::FPinfo() 3 0.0001043 0.0001162 0.0001317 0.00% MLMG::mgVcycle_up::3 31 0.0001191 0.000126 0.0001302 0.00% Amr::writeSmallPlotFile() 1 9.388e-05 0.0001072 0.0001298 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001052 0.0001136 0.0001274 0.00% MLMG::mgVcycle_up::4 31 0.0001115 0.0001155 0.0001243 0.00% Castro::expand_state() 13 0.000105 0.0001155 0.0001243 0.00% Castro::advance() 5 8.326e-05 0.000103 0.0001176 0.00% DistributionMapping::Distribute() 35 0.0001078 0.0001115 0.0001137 0.00% MLPoisson::define() 11 0.0001042 0.0001074 0.0001119 0.00% FabArray::RecvLayoutMask() 8 8.903e-05 9.613e-05 0.0001101 0.00% Castro::apply_source_to_state() 18 0.0001006 0.0001066 0.0001088 0.00% Castro::errorEst() 6 0.000101 0.0001033 0.0001053 0.00% Gravity::swapTimeLevels() 9 8.732e-05 9.499e-05 0.0001026 0.00% Castro::post_timestep() 9 7.982e-05 8.603e-05 8.901e-05 0.00% MLMG::computeResidual() 31 8.237e-05 8.426e-05 8.737e-05 0.00% MLMG::computeMLResidual() 15 7.832e-05 8.112e-05 8.337e-05 0.00% ClusterList::intersect() 1 0 2.037e-05 8.146e-05 0.00% knapsack() 3 6.195e-05 6.726e-05 7.61e-05 0.00% Castro::avgDown(state_indx) 20 6.468e-05 6.859e-05 7.132e-05 0.00% MLMG::miniCycle() 34 6.944e-05 6.999e-05 7.124e-05 0.00% Amr::initialInit() 1 6.055e-06 2.26e-05 7.121e-05 0.00% Castro::create_source_corrector() 9 5.484e-05 6.249e-05 7.12e-05 0.00% MLLinOp::define() 11 5.969e-05 6.339e-05 6.719e-05 0.00% MLMG::getGradSolution() 10 5.715e-05 6.111e-05 6.316e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 5.921e-05 6.169e-05 6.305e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.77e-05 5.422e-05 6.182e-05 0.00% amrex::average_down_w_geom 20 5.05e-05 5.234e-05 5.515e-05 0.00% Castro::construct_new_hybrid_source() 9 4.853e-05 5.097e-05 5.224e-05 0.00% Amr::initSubcycle() 1 5.086e-05 5.138e-05 5.166e-05 0.00% MLPoisson::prepareForSolve() 11 4.552e-05 4.709e-05 4.991e-05 0.00% Gravity::set_mass_offset() 6 3.652e-05 4.285e-05 4.929e-05 0.00% Castro::construct_old_rotation_source() 9 2.309e-05 2.972e-05 4.629e-05 0.00% Castro::init(old) 3 4.331e-05 4.444e-05 4.623e-05 0.00% Castro::retry_advance_ctu() 5 2.297e-05 2.877e-05 4.341e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.935e-05 3.457e-05 3.882e-05 0.00% Castro::construct_old_hybrid_source() 9 3.638e-05 3.709e-05 3.834e-05 0.00% MLMG::compResidual() 1 2.595e-05 3.026e-05 3.736e-05 0.00% AmrMesh-cluster 1 0 9.276e-06 3.71e-05 0.00% Amr::InitializeInit() 1 7.39e-06 1.509e-05 3.512e-05 0.00% MLMG::MLRhsNormInf() 10 3.273e-05 3.372e-05 3.472e-05 0.00% Castro::post_init() 1 2.38e-05 2.882e-05 3.358e-05 0.00% Castro::avgDown() 4 2.905e-05 3.106e-05 3.32e-05 0.00% Castro::construct_new_rotation_source() 9 2.221e-05 2.55e-05 3.046e-05 0.00% BoxList::parallelComplementIn() 12 2.401e-05 2.591e-05 2.909e-05 0.00% Amr::bldFineLevels() 1 1.858e-05 2.146e-05 2.464e-05 0.00% BoxList::complementIn 12 2.2e-05 2.268e-05 2.347e-05 0.00% Castro::derive() 6 1.913e-05 2.064e-05 2.21e-05 0.00% Castro::computeInitialDt() 2 1.588e-05 1.827e-05 2.195e-05 0.00% Castro::init() 1 1.558e-05 1.693e-05 1.776e-05 0.00% Amr::init() 1 6.852e-06 8.045e-06 1.117e-05 0.00% ClusterList::chop() 1 0 2.55e-06 1.02e-05 0.00% knapsack()swap 1 3.591e-06 3.851e-06 4.25e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 241.7 241.7 241.7 100.00% Amr::coarseTimeStep() 5 229.7 229.7 229.7 95.01% Amr::timeStep() 9 228.3 228.3 228.3 94.46% Castro::advance() 5 200.9 200.9 200.9 83.11% Castro::subcycle_advance_ctu() 5 192.2 192.5 193.6 80.08% Castro::do_advance_ctu() 5 192.2 192.5 193.6 80.08% Castro::construct_ctu_hydro_source() 9 118.2 118.3 118.3 48.95% Castro::clean_state() 87 71.25 80.27 84.08 34.78% Castro::computeTemp() 88 45.08 50.86 53.3 22.05% Castro::reset_internal_energy(MultiFab) 88 30.42 34.41 36.17 14.96% Castro::evaluate_source_change() 193 17.92 20 20.85 8.62% Castro::enforce_min_density() 87 16.82 19.11 20.31 8.40% Castro::post_regrid() 10 15.68 17.31 17.97 7.43% Amr::regrid() 5 15.61 17.08 17.65 7.30% check_for_negative_density() 9 0.2248 4.969 15.58 6.44% Castro::initialize_do_advance() 9 15.09 15.12 15.14 6.26% amrex::Copy() 679 12.57 14.12 14.84 6.14% AmrLevel::FillPatch() 62 7.789 9.52 14.02 5.80% FabArray::ParallelCopy() 830 3.757 6.455 13.01 5.38% FillPatchIterator::Initialize 62 6.094 7.904 12.57 5.20% MultiFab::AddProduct() 4205 10.48 11.72 12.23 5.06% Castro::normalize_species() 87 10.12 11.22 11.67 4.83% Gravity::solve_phi_with_mlmg() 11 11.19 11.32 11.53 4.77% Castro::do_old_sources() 9 11.18 11.2 11.26 4.66% Gravity::multilevel_solve_for_new_phi() 10 10.9 11.03 11.22 4.64% Gravity::actual_multilevel_solve() 10 10.87 10.99 11.19 4.63% FabArray::ParallelCopy_finish() 830 1.468 4.124 10.81 4.47% Castro::do_new_sources() 9 9.213 10.32 10.76 4.45% FillPatchSingleLevel 92 3.835 5.71 10.5 4.34% Amr::init() 1 10.36 10.36 10.36 4.29% Castro::post_timestep() 9 10.15 10.15 10.15 4.20% FillPatchIterator::FillFromLevel0() 31 3.451 5.331 10.1 4.18% Amr::initialInit() 1 9.819 9.819 9.819 4.06% Castro::reset_internal_energy(Fab) 1476 8.265 9.178 9.52 3.94% Castro::estTimeStep() 26 4.057 5.458 9.047 3.74% Castro::reflux() 4 8.919 8.964 8.984 3.72% FabArray::setVal() 5070 7.445 8.348 8.771 3.63% Castro::initialize_advance() 9 7.305 8.356 8.734 3.61% MultiFab::Subtract() 175 6.99 7.835 8.208 3.40% Gravity::actual_solve_with_mlmg() 11 6.815 6.825 6.835 2.83% Castro::expand_state() 13 5.107 5.568 6.748 2.79% Castro::construct_new_gravity() 9 6.332 6.449 6.637 2.75% MLMG::solve() 10 6.306 6.318 6.336 2.62% Amr::FinalizeInit() 1 5.712 5.875 6.252 2.59% MLMG::oneIter() 31 5.105 5.123 5.13 2.12% Gravity::fill_multipole_BCs() 11 4.359 4.479 4.684 1.94% FluxRegister::Reflux() 24 4.406 4.421 4.432 1.83% MLMG::mgVcycle() 65 4.286 4.315 4.326 1.79% Amr::InitializeInit() 1 3.567 3.944 4.106 1.70% Amr::defBaseLevel() 1 3.567 3.944 4.106 1.70% FabArray::sum() 4205 3.499 3.892 4.044 1.67% Castro::finalize_do_advance() 9 1.353 2.106 4.015 1.66% Castro::problem_post_timestep() 10 2.81 3.224 4.011 1.66% Castro::computeNewDt() 8 2.179 2.686 3.988 1.65% Castro::construct_old_gravity() 9 0.7052 1.556 3.824 1.58% Gravity::get_old_grav_vector() 9 0.6986 1.554 3.823 1.58% MLCellLinOp::smooth() 688 3.252 3.264 3.289 1.36% Castro::post_init() 1 2.564 2.707 3.035 1.26% Castro::FluxRegCrseInit 9 1.835 2.138 2.798 1.16% Castro::initData() 1 2.187 2.537 2.738 1.13% FabArray::ParallelCopy_nowait() 830 2.108 2.328 2.721 1.13% Castro::wd_update() 6 1.277 1.693 2.479 1.03% MLPoisson::Fsmooth() 1376 2.116 2.367 2.456 1.02% Castro::construct_new_source() 63 1.776 1.989 2.107 0.87% MLMG::mgVcycle_down::0 31 1.911 1.977 1.999 0.83% MLCellLinOp::applyBC() 1742 1.106 1.247 1.64 0.68% FillPatchIterator::FillFromTwoLevels() 31 1.522 1.53 1.541 0.64% FillPatchTwoLevels 31 1.522 1.529 1.54 0.64% Amr::bldFineLevels() 1 1.259 1.423 1.5 0.62% FillPatchInterp(MF) 30 1.256 1.367 1.473 0.61% MLMG::mgVcycle_up::0 31 1.373 1.399 1.409 0.58% Castro::Castro() 5 1.311 1.365 1.403 0.58% MLCellLinOp::apply() 318 1.29 1.314 1.333 0.55% FabArray::FillBoundary() 1553 0.6637 0.8301 1.284 0.53% Amr::InitAmr() 1 1.154 1.154 1.154 0.48% amrex::Add() 136 0.9804 1.09 1.144 0.47% Castro::initMFs() 5 1.075 1.115 1.137 0.47% Amr::writePlotFile() 2 1.082 1.082 1.082 0.45% Castro::construct_old_source() 63 0.9191 1.019 1.079 0.45% Castro::construct_old_gravity_source() 18 0.8973 1.009 1.072 0.44% MLPoisson::Fapply() 318 0.9231 1.025 1.064 0.44% Castro::computeInitialDt() 2 0.5125 0.6747 1.052 0.44% MLCellLinOp::solutionResidual() 72 0.9712 1.022 1.041 0.43% Castro::construct_new_gravity_source() 9 0.7507 0.8464 0.917 0.38% Castro::volInBoundary() 42 0.6566 0.7507 0.8735 0.36% FillBoundary_finish() 1553 0.3041 0.4415 0.8419 0.35% FabArray::Saxpy() 182 0.7141 0.7977 0.8363 0.35% MLCellLinOp::correctionResidual() 189 0.8054 0.8095 0.8174 0.34% Gravity::get_new_grav_vector() 10 0.7896 0.7976 0.8055 0.33% VisMF::Write(FabArray) 3 0.5733 0.6556 0.707 0.29% MLMG:computeResOfCorrection() 155 0.6571 0.6874 0.703 0.29% StateDataPhysBCFunct::() 95 0.5862 0.6059 0.6189 0.26% StateData::FillBoundary(geom) 50 0.584 0.5966 0.6085 0.25% MLMG::computeResWithCrseSolFineCor() 17 0.5707 0.5714 0.5731 0.24% Gravity::update_max_rhs() 10 0.1028 0.2436 0.5709 0.24% MultiFab::max() 18 0.03381 0.1804 0.515 0.21% MLMG::computeMLResidual() 15 0.41 0.4357 0.4939 0.20% Castro::init() 1 0.4889 0.4922 0.4939 0.20% AmrLevel::FillCoarsePatch() 5 0.4889 0.4922 0.4938 0.20% Castro::check_for_nan() 18 0.3981 0.4423 0.4607 0.19% MultiFab::contains_nan() 18 0.398 0.4422 0.4605 0.19% FillBoundary_nowait() 1553 0.344 0.3843 0.4382 0.18% Gravity::test_composite_phi() 1 0.4105 0.4115 0.4142 0.17% FabArray::Xpay() 283 0.3454 0.3855 0.3993 0.17% amrex::average_down 233 0.2793 0.3576 0.3965 0.16% MLMG::mgVcycle_down::1 31 0.3076 0.3291 0.3884 0.16% MLMG::computeResidual() 31 0.3482 0.357 0.3633 0.15% Castro::init(old) 3 0.3623 0.3628 0.3632 0.15% MLCellLinOp::reflux() 30 0.1772 0.2282 0.363 0.15% Castro::apply_source_to_state() 18 0.2866 0.3206 0.3381 0.14% MLPoisson::define() 11 0.2922 0.2997 0.3102 0.13% Castro::react_state() 18 0.2234 0.2566 0.2754 0.11% FabArray::setDomainBndry() 92 0.1958 0.2175 0.2296 0.09% MLMG::prepareForSolve() 10 0.2152 0.2215 0.2255 0.09% MLMG::mgVcycle_up::1 31 0.2144 0.2186 0.2208 0.09% MLCellLinOp::defineAuxData() 11 0.1846 0.201 0.2114 0.09% Castro::fill_hybrid_hydro_source() 27 0.1724 0.1912 0.1985 0.08% Amr::grid_places() 6 0.1914 0.1914 0.1915 0.08% AmrMesh::MakeNewGrids() 6 0.1908 0.1909 0.1911 0.08% MLMG::addInterpCorrection() 155 0.1352 0.1682 0.1844 0.08% MLMG::mgVcycle_bottom 65 0.1477 0.1548 0.1755 0.07% MLMG::getGradSolution() 10 0.169 0.1726 0.1749 0.07% MLCellLinOp::compGrad() 18 0.169 0.1725 0.1748 0.07% MLPoisson::FFlux() 150 0.004642 0.0423 0.155 0.06% MLMG::miniCycle() 34 0.1477 0.148 0.1483 0.06% FabArray::mult() 56 0.1145 0.1289 0.1382 0.06% Castro::construct_new_hybrid_source() 9 0.1139 0.1273 0.1324 0.05% Castro::avgDown() 4 0.1228 0.1249 0.129 0.05% Castro::avgDown(state_indx) 20 0.1227 0.1249 0.129 0.05% amrex::average_down_w_geom 20 0.1227 0.1248 0.1289 0.05% AmrLevel::AmrLevel(dm) 5 0.1069 0.1178 0.1239 0.05% StateData::define() 25 0.1067 0.1176 0.1237 0.05% Castro::errorEst() 6 0.1055 0.1163 0.1211 0.05% MLMG::ResNormInf() 54 0.07414 0.08609 0.1101 0.05% Castro::swap_state_time_levels() 9 0.08402 0.09393 0.1035 0.04% MLCellLinOp::updateSolBC() 25 0.07848 0.08297 0.0935 0.04% MLCellLinOp::defineBC() 11 0.08519 0.0869 0.08857 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08494 0.08592 0.0884 0.04% MLMG::mgVcycle_down::2 31 0.06805 0.07132 0.07912 0.03% MLMG::mgVcycle_up::3 31 0.02557 0.0647 0.07797 0.03% MLMG::MLResNormInf() 15 0.04471 0.0676 0.07577 0.03% Castro::construct_old_hybrid_source() 9 0.05861 0.06397 0.06668 0.03% FabArray::norminf(mask) 21 0.0574 0.06284 0.06499 0.03% BndryData::define() 27 0.05795 0.05964 0.06111 0.03% MLMG::interpCorrection_1 17 0.02666 0.03603 0.0584 0.02% Castro::apply_tagging_restrictions() 6 0.04886 0.05406 0.05721 0.02% MLMG::mgVcycle_up::2 31 0.05133 0.05236 0.05332 0.02% MLCellLinOp::updateCorBC() 17 0.04925 0.05023 0.05252 0.02% Castro::derive() 6 0.04143 0.04514 0.04642 0.02% AmrLevel::derive() 6 0.04141 0.04512 0.0464 0.02% Castro::buildMetrics() 5 0.03964 0.04251 0.04427 0.02% FabArray::norminf() 147 0.03458 0.03847 0.04196 0.02% Gravity::swapTimeLevels() 9 0.03161 0.03681 0.04061 0.02% MLMG::mgVcycle_down::3 31 0.0305 0.03191 0.03448 0.01% MLMG::MLRhsNormInf() 10 0.02562 0.02844 0.02959 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01936 0.02359 0.02935 0.01% FabArrayBase::getCPC() 1440 0.02331 0.02484 0.02784 0.01% MLMG::actualBottomSolve() 31 4.17e-05 0.006943 0.02764 0.01% MLCGSolver::bicgstab 7 0 0.006737 0.02695 0.01% Castro::FluxRegFineAdd() 9 0.02566 0.02594 0.02645 0.01% TagBoxArray::mapPRD 6 0.004964 0.01254 0.02607 0.01% MultiFab::Multiply() 8 0.02361 0.02483 0.02531 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.0194 0.02008 0.0215 0.01% MLMG::compResidual() 1 0.02053 0.02071 0.02085 0.01% MLLinOp::define() 11 0.002363 0.01174 0.0206 0.01% MLLinOp::defineGrids() 11 0.002296 0.01168 0.02054 0.01% Castro::build_fine_mask() 44 0.01712 0.01805 0.01891 0.01% MLLinOp::makeSubCommunicator() 11 0.0006009 0.009884 0.01857 0.01% MLMG::mgVcycle_down::4 31 0.005384 0.008774 0.0185 0.01% FabArrayBase::CPC::define() 416 0.01291 0.01474 0.01781 0.01% Castro::linear_to_hybrid_momentum() 1 0.01411 0.01576 0.01712 0.01% MLMG::mgVcycle_up::4 31 0.004495 0.006886 0.01393 0.01% FabArrayBase::TheFPinfo() 25 0.001218 0.004664 0.01274 0.01% FPinfo::FPinfo() 3 0.001049 0.004492 0.01256 0.01% DistributionMapping::KnapSackDoIt() 3 0.0009448 0.004376 0.01243 0.01% DistributionMapping::LeastUsedCPUs() 5 0.0009033 0.004326 0.01237 0.01% MLPoisson::prepareForSolve() 11 0.01154 0.01171 0.01193 0.00% MLCellLinOp::prepareForSolve() 11 0.01149 0.01167 0.01188 0.00% Castro::enforce_consistent_e() 1 0.009694 0.0105 0.01079 0.00% AMRErrorTag::operator() 6 0.0088 0.009915 0.01041 0.00% average_down_faces 48 0.008976 0.009442 0.00993 0.00% Castro::apply_problem_tags() 6 0.006267 0.007059 0.007335 0.00% FabArrayBase::getFB() 1553 0.005644 0.006117 0.006664 0.00% MLCellLinOp::setLevelBC() 19 0.003654 0.004674 0.005856 0.00% amrex::Dot() 115 0 0.000851 0.003404 0.00% FabArrayBase::FB::FB() 19 0.00245 0.002827 0.003163 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007467 0.002987 0.00% TagBoxArray::collate() 6 0.0005333 0.0009092 0.001256 0.00% OwnerMask() 6 0.000719 0.0008478 0.0009185 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003681 0.0003932 0.0004311 0.00% Castro::finalize_advance() 9 0.0003632 0.0003795 0.0004096 0.00% makeSFC 33 0.0002629 0.0002796 0.0003036 0.00% Castro::enforce_speed_limit() 87 0.0002523 0.000257 0.0002612 0.00% Amr::writeSmallPlotFile() 1 9.388e-05 0.0001072 0.0001298 0.00% AmrMesh-cluster 1 0 3.219e-05 0.0001288 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001114 0.0001206 0.0001263 0.00% DistributionMapping::Distribute() 35 0.0001078 0.0001115 0.0001137 0.00% FabArray::RecvLayoutMask() 8 8.903e-05 9.613e-05 0.0001101 0.00% ClusterList::intersect() 1 0 2.037e-05 8.146e-05 0.00% knapsack() 3 6.562e-05 7.111e-05 8e-05 0.00% Castro::create_source_corrector() 9 5.484e-05 6.249e-05 7.12e-05 0.00% Amr::initSubcycle() 1 5.086e-05 5.138e-05 5.166e-05 0.00% BoxList::parallelComplementIn() 12 4.6e-05 4.859e-05 5.126e-05 0.00% Gravity::set_mass_offset() 6 3.652e-05 4.285e-05 4.929e-05 0.00% Castro::construct_old_rotation_source() 9 2.309e-05 2.972e-05 4.629e-05 0.00% Castro::retry_advance_ctu() 5 2.297e-05 2.877e-05 4.341e-05 0.00% Castro::construct_new_rotation_source() 9 2.221e-05 2.55e-05 3.046e-05 0.00% BoxList::complementIn 12 2.2e-05 2.268e-05 2.347e-05 0.00% ClusterList::chop() 1 0 2.55e-06 1.02e-05 0.00% knapsack()swap 1 3.591e-06 3.851e-06 4.25e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-42-g7437aa17d36f) finalized