Initializing AMReX (23.11-17-g9e35dc19489d)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (23.11-17-g9e35dc19489d) initialized Starting run at 08:01:37 UTC on 2023-11-21. Successfully read inputs file ... Castro git describe: 23.11-6-gb720d68fa AMReX git describe: 23.11-17-g9e35dc194 Microphysics git describe: 23.11-9-ge9e31f40 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.036129914 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.035250779 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.368266994 Gravity::multilevel_solve_for_new_phi() time = 1.767201087 Gravity::fill_multipole_BCs() time = 0.362514669 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.595598834 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 1.004534824 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.486885158 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.23000246 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.722052533 Gravity::multilevel_solve_for_new_phi() time = 1.813863691 Castro::construct_new_gravity() time = 1.931595815 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.075144428e+26 zmom added: -5.613375295e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.538923063 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.184461852e+27 ener added: 2.243834889e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02408491711 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.190352593e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.82385801 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.03430803 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.454009503 Gravity::multilevel_solve_for_new_phi() time = 1.966515017 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849569753e+31 ener added: 2.785937322e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.885557306 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158191e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222073e+26 zmom added: -2.866993882e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.033863514 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.61542694 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.551111111e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.03263496 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253775e+40 ener added: 1.79168512e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.57123462e+22 ymom added: 1.564283897e+22 zmom added: 6.442237137e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.197316792 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.812534198 on level 1 Castro::reflux() at level 0 : time = 2.318000106 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.603051692 Gravity::multilevel_solve_for_new_phi() time = 1.558275262 Castro::construct_new_gravity() time = 1.67027179 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.864072325e+21 ymom added: -6.244635708e+21 zmom added: -1.246148803e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 1.951457783 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.14873037e+26 ener added: 4.251851852e+22 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.030009071 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 5.112150176e+18 ymom added: -1.460517359e+19 zmom added: -8.220975468e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.147804289 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281876859e+29 ener added: 3.281875295e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02273402282 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.036240741e+24 ener added: 5.917722222e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.84607029 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.032300081 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.405423595 Gravity::multilevel_solve_for_new_phi() time = 0.534334653 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.813445987 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157842738e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 2.019904409 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.61219537 on level 0 Castro::construct_old_gravity() time = 0.035905699 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691488e+40 ener added: 1.212217726e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.444928332e+22 ymom added: 5.312662293e+21 zmom added: 7.10314939e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.19360318 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.807147144 on level 1 Castro::reflux() at level 0 : time = 2.271667031 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.641392717 Gravity::multilevel_solve_for_new_phi() time = 1.208722221 Castro::construct_new_gravity() time = 1.320845476 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.108598872e+21 ymom added: -7.90284783e+21 zmom added: 1.10682106e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.948215165 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.030129417 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -5.605374531e+17 ymom added: 2.17388754e+19 zmom added: 9.538641239e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.145414153 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.495038611e+26 ener added: 1.500090463e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02311266116 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.283851852e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.22013743 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.032374295 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.405765743 Gravity::multilevel_solve_for_new_phi() time = 0.535036567 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.816400047 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471615689e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 2.061416205 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.70288605 on level 0 Castro::construct_old_gravity() time = 0.032758682 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425501e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.200924858e+22 ymom added: 1.239621202e+22 zmom added: 8.023757211e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.193021503 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.843050773 on level 1 Castro::reflux() at level 0 : time = 2.24549643 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.595592466 Gravity::multilevel_solve_for_new_phi() time = 1.186126937 Castro::construct_new_gravity() time = 1.300040393 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405469507e+21 ymom added: -9.735672131e+21 zmom added: -7.437548959e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.948272422 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.029832893 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.521396665e+18 ymom added: -2.667031699e+19 zmom added: -1.338680709e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.142176544 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.851958796e+25 ener added: 2.794684722e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02300947356 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 47.38409435 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.033564647 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.409910628 Gravity::multilevel_solve_for_new_phi() time = 0.541812732 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.799323202 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819051789e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 2.056461907 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.65383106 on level 0 Castro::construct_old_gravity() time = 0.036224753 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109508e+40 ener added: 1.662186972e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013797881e+23 ymom added: 1.888946593e+22 zmom added: 8.797079311e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.194859484 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.839412274 on level 1 Castro::reflux() at level 0 : time = 2.265997661 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.603161868 Gravity::multilevel_solve_for_new_phi() time = 1.154162727 Castro::construct_new_gravity() time = 1.267794505 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.798425632e+21 ymom added: -1.178643014e+22 zmom added: -5.143756067e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.962802017 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.030511925 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 2.012736213e+18 ymom added: -2.460766836e+19 zmom added: -1.244731e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.14238065 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.005277778e+24 ener added: 3.882013889e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02306129577 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.814813006e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.36696544 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.574626319 seconds Ending run at 08:05:40 UTC on 2023-11-21. Run time = 243.0289383 Run time without initialization = 231.2177827 Average number of zones advanced per microsecond: 0.021 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 243 ... 243 ... 243 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 92.1646 103.5576 108.8227 44.78% Castro::computeTemp() 88 14.74 16.64 17.51 7.21% check_for_negative_density() 9 0.2195 5.493 16.96 6.98% amrex::Copy() 679 12.58 14.1 14.75 6.07% MultiFab::AddProduct() 4205 10.65 11.89 12.37 5.09% Castro::normalize_species() 87 9.951 11.18 11.64 4.79% FabArray::ParallelCopy_finish() 830 1.644 4.252 10.57 4.35% Castro::reset_internal_energy(Fab) 1476 8.273 9.249 9.609 3.95% Castro::estTimeStep() 26 3.952 5.381 8.817 3.63% FabArray::setVal() 5078 7.414 8.258 8.681 3.57% MultiFab::Subtract() 175 6.882 7.704 7.986 3.29% Gravity::fill_multipole_BCs() 11 4.304 4.703 5.491 2.26% Castro::reset_internal_energy(MultiFab) 88 3.404 4.067 4.507 1.85% FabArray::sum() 4205 3.319 3.709 3.86 1.59% FluxRegister::Reflux() 24 2.933 3.294 3.509 1.44% FabArray::ParallelCopy_nowait() 830 2.072 2.247 2.655 1.09% Castro::enforce_min_density() 87 1.65 2.018 2.46 1.01% MLPoisson::Fsmooth() 1376 2.094 2.326 2.42 1.00% Castro::FluxRegCrseInit 9 1.447 1.652 1.77 0.73% Castro::reflux() 4 1.464 1.685 1.765 0.73% Castro::wd_update() 6 0.4305 0.7723 1.542 0.63% FillPatchInterp(MF) 30 1.229 1.333 1.444 0.59% Amr::InitAmr() 1 1.205 1.205 1.205 0.50% Castro::initMFs() 5 1.076 1.138 1.203 0.50% amrex::Add() 143 0.9806 1.098 1.146 0.47% Castro::construct_old_gravity_source() 18 0.9043 1.037 1.129 0.46% FillBoundary_finish() 1561 0.4346 0.6885 1.118 0.46% MLPoisson::Fapply() 326 0.8889 1.011 1.091 0.45% Castro::problem_post_timestep() 10 0.8507 0.8887 0.9669 0.40% Castro::construct_new_gravity_source() 9 0.7549 0.8405 0.8825 0.36% FillPatchIterator::Initialize 62 0.7557 0.8354 0.8727 0.36% FabArray::Saxpy() 182 0.7054 0.7814 0.811 0.33% VisMF::Write(FabArray) 3 0.6321 0.7224 0.7921 0.33% Castro::volInBoundary() 42 0.6546 0.6733 0.6971 0.29% StateData::FillBoundary(geom) 50 0.6094 0.6177 0.6311 0.26% Castro::construct_new_source() 63 0.4944 0.5375 0.5555 0.23% MultiFab::max() 18 0.02352 0.1501 0.4832 0.20% MLCellLinOp::applyBC() 1750 0.3664 0.4254 0.4563 0.19% MultiFab::contains_nan() 18 0.3788 0.4215 0.4424 0.18% FillBoundary_nowait() 1561 0.3061 0.371 0.4285 0.18% FabArray::Xpay() 291 0.3437 0.3852 0.4007 0.16% Castro::initialize_do_advance() 9 0.2395 0.2749 0.3086 0.13% Amr::writePlotFile() 2 0.09575 0.1726 0.2867 0.12% Castro::construct_old_source() 63 0.1892 0.2115 0.2403 0.10% FabArray::setDomainBndry() 92 0.1977 0.2182 0.232 0.10% amrex::average_down 233 0.1918 0.2034 0.2099 0.09% Castro::fill_hybrid_hydro_source() 27 0.1671 0.1877 0.1995 0.08% MLCellLinOp::defineAuxData() 11 0.1527 0.1672 0.1727 0.07% MLPoisson::FFlux() 149 0.004598 0.04263 0.1567 0.06% Castro::initData() 1 0.1231 0.1377 0.1488 0.06% MLCellLinOp::compGrad() 18 0.1221 0.1339 0.1385 0.06% Castro::swap_state_time_levels() 9 0.0984 0.1139 0.1324 0.05% FabArray::mult() 56 0.1107 0.1241 0.1299 0.05% StateData::define() 25 0.1178 0.1229 0.1286 0.05% Castro::evaluate_source_change() 193 0.113 0.1192 0.1229 0.05% MLMG::addInterpCorrection() 155 0.1031 0.1161 0.1222 0.05% Gravity::get_old_grav_vector() 9 0.07721 0.09371 0.1127 0.05% Gravity::get_new_grav_vector() 10 0.08508 0.09322 0.09662 0.04% Castro::initialize_advance() 9 0.08188 0.08773 0.09347 0.04% MLMG::prepareForSolve() 10 0.07286 0.07996 0.09058 0.04% MLCellLinOp::updateSolBC() 25 0.0701 0.07045 0.07073 0.03% MLMG::solve() 10 0.0007437 0.0444 0.06983 0.03% AmrLevel::FillPatch() 62 0.05605 0.05891 0.06326 0.03% FabArray::norminf(mask) 21 0.05351 0.06013 0.06252 0.03% MLMG::ResNormInf() 54 0.003295 0.02728 0.05826 0.02% Castro::apply_tagging_restrictions() 6 0.05007 0.05512 0.05797 0.02% AmrMesh::MakeNewGrids() 6 0.04765 0.04996 0.05635 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.03965 0.04341 0.0514 0.02% MLCellLinOp::updateCorBC() 17 0.04762 0.04787 0.04821 0.02% BndryData::define() 27 0.04336 0.04463 0.04627 0.02% FabArray::norminf() 147 0.03676 0.04004 0.04199 0.02% MLCellLinOp::reflux() 30 0.004113 0.01237 0.03701 0.02% MLMG::MLResNormInf() 15 0.0001078 0.02477 0.03592 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01895 0.02228 0.02837 0.01% Castro::FluxRegFineAdd() 9 0.02534 0.02598 0.02749 0.01% MLCellLinOp::defineBC() 11 0.02694 0.02706 0.02718 0.01% MultiFab::Multiply() 8 0.02141 0.02413 0.02518 0.01% Castro::build_fine_mask() 44 0.01784 0.01977 0.02085 0.01% MLMG::interpCorrection_1 17 0.01821 0.01861 0.01947 0.01% Castro::linear_to_hybrid_momentum() 1 0.0163 0.01692 0.01858 0.01% Castro::buildMetrics() 5 0.01492 0.01667 0.01828 0.01% FabArrayBase::CPC::define() 416 0.01326 0.015 0.018 0.01% Castro::Castro() 5 0.01616 0.01675 0.01765 0.01% StateDataPhysBCFunct::() 95 0.001975 0.007278 0.01591 0.01% FillPatchTwoLevels 31 0.01355 0.0146 0.01585 0.01% Gravity::actual_multilevel_solve() 10 0.01055 0.01236 0.01532 0.01% MLLinOp::makeSubCommunicator() 11 0.001075 0.006939 0.01532 0.01% Castro::do_new_sources() 9 0.000479 0.004647 0.01449 0.01% Castro::construct_old_gravity() 9 0.0003446 0.00374 0.01369 0.01% MLCellLinOp::prepareForSolve() 11 0.01168 0.0118 0.0119 0.00% Castro::enforce_consistent_e() 1 0.009417 0.01055 0.01123 0.00% AMRErrorTag::operator() 6 0.008692 0.009907 0.01068 0.00% FabArrayBase::getCPC() 1440 0.01004 0.01036 0.01051 0.00% Gravity::update_max_rhs() 10 0.008764 0.009192 0.009729 0.00% Gravity::actual_solve_with_mlmg() 11 0.00915 0.009323 0.00963 0.00% Castro::do_old_sources() 9 0.0004684 0.002668 0.008795 0.00% DistributionMapping::LeastUsedCPUs() 5 0.004323 0.00561 0.008276 0.00% Castro::apply_problem_tags() 6 0.00718 0.007441 0.007782 0.00% Castro::finalize_do_advance() 9 0.006183 0.006797 0.00719 0.00% AmrLevel::derive() 6 0.004677 0.005343 0.005862 0.00% average_down_faces 48 0.005602 0.005636 0.005664 0.00% MLCellLinOp::setLevelBC() 19 0.003823 0.004659 0.00554 0.00% FabArray::FillBoundary() 1561 0.004091 0.004331 0.00479 0.00% AmrLevel::FillCoarsePatch() 5 0.0009209 0.002294 0.004355 0.00% MLCGSolver::bicgstab 7 0 0.001029 0.004118 0.00% MLCellLinOp::smooth() 688 0.003557 0.003685 0.003829 0.00% FabArrayBase::getFB() 1561 0.003099 0.003279 0.003673 0.00% amrex::Dot() 115 0 0.000874 0.003496 0.00% Gravity::test_composite_phi() 1 0.002239 0.002594 0.003275 0.00% Castro::construct_new_gravity() 9 0.0003821 0.001363 0.003153 0.00% FabArrayBase::FB::FB() 19 0.002543 0.00285 0.003141 0.00% FabArray::ParallelCopy() 830 0.0029 0.002968 0.003079 0.00% main() 1 0.001703 0.001747 0.00182 0.00% MLLinOp::defineGrids() 11 0.00141 0.001487 0.001666 0.00% Amr::regrid() 5 0.0009912 0.001185 0.001554 0.00% MLCellLinOp::apply() 326 0.0008448 0.001038 0.001549 0.00% Amr::coarseTimeStep() 5 0.0006333 0.001106 0.001268 0.00% Castro::clean_state() 87 0.00119 0.001212 0.001236 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0003062 0.001225 0.00% TagBoxArray::collate() 6 0.0002855 0.0009285 0.001151 0.00% OwnerMask() 6 0.0007301 0.0008339 0.0009123 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008917 0.0009004 0.0009085 0.00% MLCellLinOp::correctionResidual() 196 0.0006588 0.000706 0.0007939 0.00% MLMG::mgVcycle() 65 0.0005711 0.0005935 0.0006099 0.00% Castro::finalize_advance() 9 0.0003477 0.0004976 0.0006042 0.00% Castro::do_advance_ctu() 5 0.0005227 0.0005648 0.0005994 0.00% Amr::grid_places() 6 0.0003914 0.0004719 0.0005596 0.00% FillPatchSingleLevel 92 0.0005249 0.0005342 0.0005458 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003776 0.0004194 0.0004515 0.00% TagBoxArray::mapPRD 6 0.0004305 0.0004395 0.0004454 0.00% Castro::subcycle_advance_ctu() 5 8.602e-05 0.0001786 0.0004074 0.00% MLMG:computeResOfCorrection() 155 0.0003786 0.0003872 0.0004037 0.00% MLMG::actualBottomSolve() 31 3.946e-05 0.0001224 0.0003693 0.00% MLMG::oneIter() 31 0.0003209 0.0003328 0.0003514 0.00% MLCellLinOp::solutionResidual() 72 0.0003392 0.0003452 0.0003498 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002917 0.0003012 0.000312 0.00% Castro::computeNewDt() 8 0.0001094 0.0001552 0.0002885 0.00% Amr::timeStep() 9 0.000185 0.0002233 0.0002787 0.00% MLMG::mgVcycle_down::0 31 0.000255 0.0002692 0.0002766 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002241 0.0002431 0.0002711 0.00% Amr::defBaseLevel() 1 0.000235 0.0002487 0.0002659 0.00% Castro::enforce_speed_limit() 87 0.00023 0.0002467 0.0002587 0.00% MLMG::mgVcycle_down::1 31 0.00022 0.0002322 0.0002424 0.00% Castro::post_regrid() 10 0.0002041 0.0002117 0.0002234 0.00% MLMG::mgVcycle_bottom 65 0.0002066 0.0002131 0.0002163 0.00% Amr::FinalizeInit() 1 3.128e-05 8.386e-05 0.0002062 0.00% MLMG::mgVcycle_down::3 31 0.0001826 0.0001862 0.000192 0.00% AmrLevel::AmrLevel(dm) 5 0.0001797 0.0001862 0.0001904 0.00% MLMG::mgVcycle_down::2 31 0.0001802 0.0001867 0.0001899 0.00% MLMG::mgVcycle_down::4 31 0.0001604 0.0001682 0.0001877 0.00% makeSFC 33 0.000163 0.0001715 0.0001847 0.00% FabArrayBase::TheFPinfo() 25 0.0001651 0.000174 0.0001821 0.00% MLMG::mgVcycle_up::0 31 0.0001731 0.0001763 0.0001794 0.00% Castro::check_for_nan() 18 0.0001319 0.0001548 0.0001668 0.00% Castro::react_state() 18 0.0001187 0.0001427 0.0001514 0.00% MLMG::mgVcycle_up::1 31 0.0001261 0.0001304 0.0001366 0.00% FPinfo::FPinfo() 3 0.0001035 0.000123 0.0001351 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001203 0.0001268 0.0001318 0.00% MLMG::mgVcycle_up::2 31 0.0001239 0.0001266 0.0001287 0.00% MLMG::mgVcycle_up::4 31 0.000111 0.0001169 0.0001252 0.00% MLMG::mgVcycle_up::3 31 0.000119 0.0001225 0.0001247 0.00% Castro::expand_state() 13 0.0001049 0.0001129 0.0001233 0.00% Castro::errorEst() 6 9.988e-05 0.0001076 0.0001222 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001107 0.0001149 0.0001214 0.00% Amr::writeSmallPlotFile() 1 9.873e-05 0.0001082 0.0001208 0.00% FabArray::RecvLayoutMask() 8 8.973e-05 0.0001007 0.0001192 0.00% DistributionMapping::Distribute() 35 0.0001084 0.0001118 0.0001143 0.00% Castro::apply_source_to_state() 18 9.845e-05 0.000104 0.0001141 0.00% Castro::advance() 5 8.071e-05 9.23e-05 0.0001109 0.00% MLPoisson::define() 11 0.0001042 0.0001061 0.0001081 0.00% Gravity::swapTimeLevels() 9 8.481e-05 9.009e-05 0.0001023 0.00% Castro::post_timestep() 9 7.675e-05 8.596e-05 9.394e-05 0.00% MLLinOp::define() 11 6.374e-05 7.395e-05 9.179e-05 0.00% Castro::avgDown(state_indx) 20 6.639e-05 7.422e-05 9.135e-05 0.00% MLMG::computeResidual() 31 7.931e-05 8.294e-05 8.641e-05 0.00% ClusterList::intersect() 1 0 2.062e-05 8.25e-05 0.00% MLMG::computeMLResidual() 15 7.405e-05 7.715e-05 8.027e-05 0.00% knapsack() 3 6.215e-05 7.031e-05 7.463e-05 0.00% Castro::create_source_corrector() 9 6.202e-05 6.66e-05 7.025e-05 0.00% MLMG::miniCycle() 34 6.413e-05 6.644e-05 6.918e-05 0.00% Castro::retry_advance_ctu() 5 2.239e-05 3.722e-05 6.288e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 5.91e-05 6.075e-05 6.216e-05 0.00% MLMG::getGradSolution() 10 5.629e-05 5.877e-05 6.186e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.283e-05 4.884e-05 5.649e-05 0.00% amrex::average_down_w_geom 20 4.716e-05 5.047e-05 5.316e-05 0.00% Amr::initSubcycle() 1 5.133e-05 5.211e-05 5.249e-05 0.00% Castro::construct_new_hybrid_source() 9 4.218e-05 4.69e-05 5.084e-05 0.00% MLPoisson::prepareForSolve() 11 4.47e-05 4.698e-05 5.031e-05 0.00% Gravity::set_mass_offset() 6 4.49e-05 4.716e-05 4.919e-05 0.00% Castro::init(old) 3 4.172e-05 4.28e-05 4.425e-05 0.00% Castro::computeInitialDt() 2 1.607e-05 2.712e-05 3.934e-05 0.00% Castro::construct_old_hybrid_source() 9 3.375e-05 3.691e-05 3.869e-05 0.00% AmrMesh-cluster 1 0 9.136e-06 3.654e-05 0.00% Castro::avgDown() 4 2.917e-05 3.291e-05 3.636e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.824e-05 3.098e-05 3.446e-05 0.00% BoxList::parallelComplementIn() 12 2.492e-05 2.923e-05 3.411e-05 0.00% Castro::derive() 6 1.853e-05 2.244e-05 3.314e-05 0.00% MLMG::MLRhsNormInf() 10 2.961e-05 3.13e-05 3.295e-05 0.00% BoxList::complementIn 12 2.475e-05 2.808e-05 3.177e-05 0.00% Castro::post_init() 1 2.4e-05 2.767e-05 3.007e-05 0.00% Amr::bldFineLevels() 1 2.259e-05 2.574e-05 2.885e-05 0.00% MLMG::compResidual() 1 2.519e-05 2.654e-05 2.789e-05 0.00% Castro::construct_old_rotation_source() 9 2.169e-05 2.343e-05 2.554e-05 0.00% Castro::construct_new_rotation_source() 9 1.884e-05 2.097e-05 2.276e-05 0.00% Castro::init() 1 1.612e-05 1.683e-05 1.794e-05 0.00% Amr::initialInit() 1 4.255e-06 6.365e-06 1.068e-05 0.00% Amr::InitializeInit() 1 7.464e-06 9.204e-06 1.063e-05 0.00% ClusterList::chop() 1 0 2.488e-06 9.953e-06 0.00% Amr::init() 1 6.215e-06 6.946e-06 7.544e-06 0.00% knapsack()swap 1 3.523e-06 3.677e-06 3.836e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 243 243 243 100.00% Amr::coarseTimeStep() 5 230.6 230.6 230.6 94.90% Amr::timeStep() 9 229.3 229.4 229.4 94.37% Castro::advance() 5 202.2 202.2 202.2 83.18% Castro::subcycle_advance_ctu() 5 193.6 193.9 194.7 80.12% Castro::do_advance_ctu() 5 193.6 193.9 194.7 80.12% Castro::construct_ctu_hydro_source() 9 118.7 118.9 119 48.97% Castro::clean_state() 87 71.22 80.29 84.01 34.57% Castro::computeTemp() 88 45.14 50.91 53.26 21.91% Castro::reset_internal_energy(MultiFab) 88 30.4 34.27 35.74 14.71% Castro::evaluate_source_change() 193 17.83 19.89 20.7 8.52% Castro::enforce_min_density() 87 16.93 19.09 20.08 8.26% Castro::post_regrid() 10 15.89 17.37 17.92 7.37% Amr::regrid() 5 15.77 17.1 17.61 7.24% check_for_negative_density() 9 0.2195 5.493 16.96 6.98% Castro::initialize_do_advance() 9 14.52 15.04 15.4 6.34% amrex::Copy() 679 12.58 14.1 14.75 6.07% AmrLevel::FillPatch() 62 7.84 9.901 14.49 5.96% FillPatchIterator::Initialize 62 6.156 8.292 13.05 5.37% FabArray::ParallelCopy() 830 3.778 6.562 12.75 5.25% Gravity::solve_phi_with_mlmg() 11 11.37 11.77 12.56 5.17% MultiFab::AddProduct() 4205 10.65 11.89 12.37 5.09% Gravity::multilevel_solve_for_new_phi() 10 11.1 11.49 12.26 5.05% Gravity::actual_multilevel_solve() 10 11.05 11.44 12.22 5.03% Castro::normalize_species() 87 9.951 11.18 11.64 4.79% Castro::do_old_sources() 9 11.28 11.35 11.44 4.71% FillPatchSingleLevel 92 3.912 6.117 11.03 4.54% Castro::do_new_sources() 9 9.248 10.39 10.91 4.49% Amr::init() 1 10.61 10.61 10.61 4.36% FabArray::ParallelCopy_finish() 830 1.644 4.252 10.57 4.35% FillPatchIterator::FillFromLevel0() 31 3.538 5.692 10.52 4.33% Castro::post_timestep() 9 10.06 10.06 10.06 4.14% Amr::initialInit() 1 10.01 10.01 10.01 4.12% Castro::reset_internal_energy(Fab) 1476 8.273 9.249 9.609 3.95% Castro::reflux() 4 8.343 8.883 9.101 3.74% Castro::estTimeStep() 26 3.952 5.381 8.817 3.63% FabArray::setVal() 5078 7.414 8.258 8.681 3.57% Castro::initialize_advance() 9 7.441 8.264 8.581 3.53% MultiFab::Subtract() 175 6.882 7.704 7.986 3.29% Castro::construct_new_gravity() 9 6.488 6.846 7.609 3.13% Gravity::actual_solve_with_mlmg() 11 6.964 6.969 6.975 2.87% MLMG::solve() 10 6.46 6.47 6.49 2.67% Castro::expand_state() 13 5.068 5.563 6.354 2.61% Amr::FinalizeInit() 1 5.784 6.004 6.32 2.60% Gravity::fill_multipole_BCs() 11 4.385 4.782 5.564 2.29% MLMG::oneIter() 31 5.207 5.221 5.24 2.16% FluxRegister::Reflux() 24 3.778 4.285 4.498 1.85% MLMG::mgVcycle() 65 4.415 4.471 4.496 1.85% Castro::construct_old_gravity() 9 0.7256 1.815 4.455 1.83% Gravity::get_old_grav_vector() 9 0.7119 1.811 4.454 1.83% Amr::InitializeInit() 1 3.69 4.005 4.226 1.74% Amr::defBaseLevel() 1 3.69 4.005 4.226 1.74% Castro::finalize_do_advance() 9 1.305 2.091 4.056 1.67% FabArray::sum() 4205 3.319 3.709 3.86 1.59% Castro::problem_post_timestep() 10 2.69 3.036 3.787 1.56% Castro::computeNewDt() 8 2.102 2.525 3.679 1.51% MLCellLinOp::smooth() 688 3.33 3.409 3.488 1.44% Castro::post_init() 1 2.488 2.615 2.942 1.21% Castro::initData() 1 2.31 2.58 2.725 1.12% FabArray::ParallelCopy_nowait() 830 2.131 2.307 2.72 1.12% Castro::FluxRegCrseInit 9 1.728 2.024 2.681 1.10% MLPoisson::Fsmooth() 1376 2.094 2.326 2.42 1.00% Castro::wd_update() 6 1.099 1.446 2.197 0.90% MLMG::mgVcycle_down::0 31 2.035 2.077 2.117 0.87% Castro::construct_new_source() 63 1.796 1.985 2.055 0.85% MLCellLinOp::applyBC() 1750 1.281 1.495 1.924 0.79% FillPatchIterator::FillFromTwoLevels() 31 1.499 1.547 1.588 0.65% FillPatchTwoLevels 31 1.498 1.546 1.588 0.65% Amr::bldFineLevels() 1 1.308 1.491 1.571 0.65% FabArray::FillBoundary() 1561 0.8457 1.07 1.558 0.64% Castro::Castro() 5 1.311 1.389 1.465 0.60% FillPatchInterp(MF) 30 1.229 1.333 1.444 0.59% MLMG::mgVcycle_up::0 31 1.395 1.415 1.441 0.59% MLCellLinOp::apply() 326 1.304 1.348 1.409 0.58% Castro::initMFs() 5 1.081 1.145 1.21 0.50% Amr::InitAmr() 1 1.205 1.205 1.205 0.50% Castro::construct_old_source() 63 0.9262 1.056 1.173 0.48% Amr::writePlotFile() 2 1.17 1.171 1.171 0.48% amrex::Add() 143 0.9806 1.098 1.146 0.47% Castro::construct_old_gravity_source() 18 0.9043 1.037 1.129 0.46% FillBoundary_finish() 1561 0.4346 0.6885 1.118 0.46% MLPoisson::Fapply() 326 0.8889 1.011 1.091 0.45% Castro::computeInitialDt() 2 0.5521 0.7726 1.088 0.45% MLCellLinOp::solutionResidual() 72 1.019 1.049 1.08 0.44% Castro::construct_new_gravity_source() 9 0.7549 0.8405 0.8825 0.36% MLCellLinOp::correctionResidual() 196 0.8228 0.8414 0.8518 0.35% Gravity::get_new_grav_vector() 10 0.7792 0.7917 0.8226 0.34% FabArray::Saxpy() 182 0.7054 0.7814 0.811 0.33% VisMF::Write(FabArray) 3 0.6321 0.7224 0.7921 0.33% MLMG:computeResOfCorrection() 155 0.6575 0.7032 0.7355 0.30% Castro::volInBoundary() 42 0.6547 0.6734 0.6971 0.29% StateDataPhysBCFunct::() 95 0.6115 0.625 0.637 0.26% StateData::FillBoundary(geom) 50 0.6094 0.6177 0.6311 0.26% MLMG::computeResWithCrseSolFineCor() 17 0.4726 0.5052 0.589 0.24% Gravity::update_max_rhs() 10 0.0828 0.2074 0.5356 0.22% MLMG::computeMLResidual() 15 0.4322 0.4681 0.5167 0.21% Castro::init() 1 0.4837 0.4905 0.4973 0.20% AmrLevel::FillCoarsePatch() 5 0.4836 0.4905 0.4973 0.20% MultiFab::max() 18 0.02352 0.1501 0.4832 0.20% Castro::check_for_nan() 18 0.3789 0.4216 0.4426 0.18% MultiFab::contains_nan() 18 0.3788 0.4215 0.4424 0.18% FillBoundary_nowait() 1561 0.3118 0.3771 0.4348 0.18% MLMG::mgVcycle_down::1 31 0.3053 0.3589 0.4251 0.17% Gravity::test_composite_phi() 1 0.4051 0.4057 0.4064 0.17% FabArray::Xpay() 291 0.3437 0.3852 0.4007 0.16% MLMG::computeResidual() 31 0.3512 0.3647 0.3746 0.15% MLCellLinOp::reflux() 30 0.1908 0.2406 0.3692 0.15% Castro::init(old) 3 0.3606 0.3635 0.3685 0.15% Castro::react_state() 18 0.2396 0.2833 0.3444 0.14% Castro::apply_source_to_state() 18 0.2847 0.3134 0.3269 0.13% MLPoisson::define() 11 0.2843 0.2948 0.3025 0.12% amrex::average_down 233 0.2599 0.2754 0.2941 0.12% MLMG::prepareForSolve() 10 0.2013 0.2225 0.2359 0.10% FabArray::setDomainBndry() 92 0.1977 0.2182 0.232 0.10% MLMG::mgVcycle_up::1 31 0.2164 0.2205 0.2228 0.09% MLCellLinOp::defineAuxData() 11 0.1793 0.1967 0.2062 0.08% Amr::grid_places() 6 0.2038 0.204 0.204 0.08% AmrMesh::MakeNewGrids() 6 0.2034 0.2035 0.2035 0.08% Castro::fill_hybrid_hydro_source() 27 0.1671 0.1877 0.1995 0.08% MLMG::addInterpCorrection() 155 0.1274 0.1647 0.1835 0.08% MLMG::mgVcycle_bottom 65 0.1487 0.1564 0.1789 0.07% MLMG::getGradSolution() 10 0.1687 0.1702 0.171 0.07% MLCellLinOp::compGrad() 18 0.1686 0.1701 0.171 0.07% MLPoisson::FFlux() 149 0.004598 0.04263 0.1567 0.06% MLMG::miniCycle() 34 0.1483 0.1487 0.149 0.06% Castro::avgDown() 4 0.1186 0.1338 0.1447 0.06% Castro::avgDown(state_indx) 20 0.1185 0.1338 0.1446 0.06% amrex::average_down_w_geom 20 0.1185 0.1337 0.1446 0.06% MLCellLinOp::updateSolBC() 25 0.08004 0.1044 0.1387 0.06% Castro::construct_new_hybrid_source() 9 0.1114 0.1254 0.1356 0.06% Castro::swap_state_time_levels() 9 0.0984 0.1139 0.1324 0.05% FabArray::mult() 56 0.1107 0.1241 0.1299 0.05% AmrLevel::AmrLevel(dm) 5 0.118 0.1231 0.1288 0.05% StateData::define() 25 0.1178 0.1229 0.1286 0.05% Castro::errorEst() 6 0.1111 0.1213 0.1272 0.05% MLMG::ResNormInf() 54 0.07461 0.09676 0.1202 0.05% MLCellLinOp::defineBC() 11 0.08735 0.08912 0.09052 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08905 0.08927 0.08945 0.04% MLMG::mgVcycle_down::2 31 0.06969 0.0747 0.08334 0.03% MLMG::mgVcycle_up::3 31 0.02541 0.06748 0.08201 0.03% MLMG::MLResNormInf() 15 0.04356 0.06682 0.07523 0.03% Castro::construct_old_hybrid_source() 9 0.05578 0.06237 0.06667 0.03% FabArray::norminf(mask) 21 0.05351 0.06013 0.06252 0.03% BndryData::define() 27 0.059 0.06077 0.06206 0.03% MLMG::interpCorrection_1 17 0.02704 0.04141 0.0589 0.02% Castro::apply_tagging_restrictions() 6 0.05007 0.05512 0.05797 0.02% MLCellLinOp::updateCorBC() 17 0.05368 0.054 0.05448 0.02% MLMG::mgVcycle_up::2 31 0.05309 0.05368 0.05421 0.02% Castro::derive() 6 0.04506 0.04875 0.05129 0.02% AmrLevel::derive() 6 0.04504 0.04873 0.05127 0.02% Castro::buildMetrics() 5 0.03864 0.04266 0.04611 0.02% FabArray::norminf() 147 0.03676 0.04004 0.04199 0.02% Gravity::swapTimeLevels() 9 0.03434 0.03801 0.04085 0.02% MLMG::mgVcycle_down::3 31 0.03045 0.03149 0.03394 0.01% TagBoxArray::mapPRD 6 0.003954 0.01821 0.03312 0.01% MLMG::actualBottomSolve() 31 3.946e-05 0.007706 0.03071 0.01% MLCGSolver::bicgstab 7 0 0.007492 0.02997 0.01% MLMG::MLRhsNormInf() 10 0.02467 0.02756 0.02853 0.01% FabArrayBase::getCPC() 1440 0.0233 0.02536 0.02843 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01895 0.02228 0.02837 0.01% Castro::FluxRegFineAdd() 9 0.02534 0.02598 0.02749 0.01% MultiFab::Multiply() 8 0.02141 0.02413 0.02518 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01441 0.01624 0.02128 0.01% Castro::build_fine_mask() 44 0.01784 0.01977 0.02085 0.01% MLMG::compResidual() 1 0.0184 0.01962 0.02026 0.01% MLMG::mgVcycle_down::4 31 0.005043 0.008659 0.01869 0.01% Castro::linear_to_hybrid_momentum() 1 0.0163 0.01692 0.01858 0.01% FabArrayBase::CPC::define() 416 0.01326 0.015 0.018 0.01% MLLinOp::define() 11 0.002959 0.008886 0.01748 0.01% MLLinOp::defineGrids() 11 0.002892 0.008812 0.01739 0.01% MLLinOp::makeSubCommunicator() 11 0.001075 0.006939 0.01532 0.01% MLMG::mgVcycle_up::4 31 0.00422 0.006776 0.01404 0.01% MLPoisson::prepareForSolve() 11 0.01173 0.01185 0.01194 0.00% MLCellLinOp::prepareForSolve() 11 0.01168 0.0118 0.0119 0.00% Castro::enforce_consistent_e() 1 0.009417 0.01055 0.01123 0.00% AMRErrorTag::operator() 6 0.008692 0.009907 0.01068 0.00% average_down_faces 48 0.008944 0.009293 0.009642 0.00% FabArrayBase::TheFPinfo() 25 0.004647 0.005963 0.008637 0.00% FPinfo::FPinfo() 3 0.004466 0.005789 0.008469 0.00% DistributionMapping::KnapSackDoIt() 3 0.004363 0.005666 0.008334 0.00% DistributionMapping::LeastUsedCPUs() 5 0.004323 0.00561 0.008276 0.00% Castro::apply_problem_tags() 6 0.00718 0.007441 0.007782 0.00% FabArrayBase::getFB() 1561 0.005642 0.006129 0.006657 0.00% MLCellLinOp::setLevelBC() 19 0.003823 0.004659 0.00554 0.00% amrex::Dot() 115 0 0.000874 0.003496 0.00% FabArrayBase::FB::FB() 19 0.002543 0.00285 0.003141 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007727 0.003091 0.00% TagBoxArray::collate() 6 0.0002855 0.0009285 0.001151 0.00% OwnerMask() 6 0.0007301 0.0008339 0.0009123 0.00% Castro::finalize_advance() 9 0.0003477 0.0004976 0.0006042 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003789 0.000387 0.0004062 0.00% makeSFC 33 0.0002634 0.000272 0.0002848 0.00% Castro::enforce_speed_limit() 87 0.00023 0.0002467 0.0002587 0.00% AmrMesh-cluster 1 0 3.225e-05 0.000129 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 9.666e-05 0.0001088 0.000123 0.00% Amr::writeSmallPlotFile() 1 9.873e-05 0.0001082 0.0001208 0.00% FabArray::RecvLayoutMask() 8 8.973e-05 0.0001007 0.0001192 0.00% DistributionMapping::Distribute() 35 0.0001084 0.0001118 0.0001143 0.00% ClusterList::intersect() 1 0 2.062e-05 8.25e-05 0.00% knapsack() 3 6.567e-05 7.399e-05 7.838e-05 0.00% Castro::create_source_corrector() 9 6.202e-05 6.66e-05 7.025e-05 0.00% BoxList::parallelComplementIn() 12 4.99e-05 5.731e-05 6.379e-05 0.00% Castro::retry_advance_ctu() 5 2.239e-05 3.722e-05 6.288e-05 0.00% Amr::initSubcycle() 1 5.133e-05 5.211e-05 5.249e-05 0.00% Gravity::set_mass_offset() 6 4.49e-05 4.716e-05 4.919e-05 0.00% BoxList::complementIn 12 2.475e-05 2.808e-05 3.177e-05 0.00% Castro::construct_old_rotation_source() 9 2.169e-05 2.343e-05 2.554e-05 0.00% Castro::construct_new_rotation_source() 9 1.884e-05 2.097e-05 2.276e-05 0.00% ClusterList::chop() 1 0 2.488e-06 9.953e-06 0.00% knapsack()swap 1 3.523e-06 3.677e-06 3.836e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (23.11-17-g9e35dc19489d) finalized