Initializing AMReX (v2024-27-ga9b452ab5e8b)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (v2024-27-ga9b452ab5e8b) initialized Starting run at 07:25:10 UTC on 2024-03-21. Successfully read inputs file ... Castro git describe: 24.03-25-g09bde5fae AMReX git describe: v2024-27-ga9b452ab5 Microphysics git describe: 24.03-20-gb934b16f reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.030737212 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.030070061 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.435156388 Gravity::multilevel_solve_for_new_phi() time = 1.83834856 Gravity::fill_multipole_BCs() time = 0.370607465 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.532521404 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.860273991 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793356e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 2.298053807 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.42115983 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.707782733 Gravity::multilevel_solve_for_new_phi() time = 1.868853561 Castro::construct_new_gravity() time = 1.986283794 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207927e+26 ymom added: 1.108730196e+26 zmom added: -5.613112907e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 2.280176616 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.00947637e+27 ener added: 2.068773556e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.02429862888 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.191111111e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 38.54249702 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.033021451 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.403331908 Gravity::multilevel_solve_for_new_phi() time = 1.89277573 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849559407e+31 ener added: 2.785926976e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.853866182 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158322e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222074e+26 zmom added: -2.8670017e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 2.012347932 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.79364023 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.503703704e+23 ener added: 6.281481481e+23 Castro::construct_old_gravity() time = 0.033530532 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253778e+40 ener added: 1.791685121e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.574127282e+22 ymom added: 1.534769107e+22 zmom added: 6.441804792e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.193442931 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.808219321 on level 1 Castro::reflux() at level 0 : time = 2.265125501 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.434994295 Gravity::multilevel_solve_for_new_phi() time = 1.380144995 Castro::construct_new_gravity() time = 1.497271555 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46375568e+41 ener added: 7.485558923e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.861679586e+21 ymom added: -6.245616015e+21 zmom added: -1.245900444e+20 eint added: 0 ener added: -3.223710603e+41 Castro::do_new_sources() time = 2.006332735 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.149299259e+26 ener added: 4.592592593e+21 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025 Castro::construct_new_gravity() time = 0.031775694 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472528e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: 4.76487527e+18 ymom added: -1.361438167e+19 zmom added: -8.219114427e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.13691006 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.281889991e+29 ener added: 3.281888214e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02263936379 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135286e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.057555556e+24 ener added: 5.917703704e+24 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 49.96886825 [STEP 2] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.031240652 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.403264061 Gravity::multilevel_solve_for_new_phi() time = 0.528803847 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.84693833 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.083120164e+24 ymom added: 5.231862796e+26 zmom added: -3.157856713e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 1.998367891 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.31635871 on level 0 Castro::construct_old_gravity() time = 0.037882278 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.818691374e+40 ener added: 1.212217725e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.442950351e+22 ymom added: 5.017514388e+21 zmom added: 7.103293505e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.206132915 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.91074584 on level 1 Castro::reflux() at level 0 : time = 2.267647923 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.421361625 Gravity::multilevel_solve_for_new_phi() time = 0.976800566 Castro::construct_new_gravity() time = 1.093083034 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.106550538e+21 ymom added: -7.896156836e+21 zmom added: 1.108336323e+20 eint added: 0 ener added: -3.427512073e+41 Castro::do_new_sources() time = 1.975781694 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194 Castro::construct_new_gravity() time = 0.031053456 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.74625952e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -6.816100901e+17 ymom added: 2.351329365e+19 zmom added: 9.539392659e+19 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.139659542 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.490463796e+26 ener added: 1.495539352e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02317475626 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778748814e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.281481481e+24 ener added: 6.674962963e+24 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 47.09789605 [STEP 3] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.031535561 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.401570995 Gravity::multilevel_solve_for_new_phi() time = 0.528349359 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.741101595 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.694655893e+24 ymom added: 5.755038681e+26 zmom added: -3.471622784e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 1.957171121 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.24339201 on level 0 Castro::construct_old_gravity() time = 0.043576927 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425349e+40 ener added: 1.373884257e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.19818149e+22 ymom added: 1.239621202e+22 zmom added: 8.023757211e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.197542008 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881554e+48 ener added: 1.675623034e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.775684571 on level 1 Castro::reflux() at level 0 : time = 2.233069924 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.420424048 Gravity::multilevel_solve_for_new_phi() time = 0.957104207 Castro::construct_new_gravity() time = 1.070202704 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.405071346e+21 ymom added: -9.735495453e+21 zmom added: -7.674479504e+18 eint added: 0 ener added: -3.723007452e+41 Castro::do_new_sources() time = 1.94295881 on level 0 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171 Castro::construct_new_gravity() time = 0.028586915 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265925e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 3.598274775e+18 ymom added: -2.567952508e+19 zmom added: -1.336958338e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.137895942 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.942862037e+25 ener added: 2.885587963e+25 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02343728544 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056623696e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.365462963e+23 ener added: 4.62962963e+19 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 46.63321874 [STEP 4] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.031297925 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.402619213 Gravity::multilevel_solve_for_new_phi() time = 0.528058298 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.795236973 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366517551e+24 ymom added: 6.330531998e+26 zmom added: -3.819056992e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 1.983181281 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 21.55879555 on level 0 Castro::construct_old_gravity() time = 0.037262341 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109532e+40 ener added: 1.662186973e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.013662616e+23 ymom added: 1.829917012e+22 zmom added: 8.796070504e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.196889581 on level 1 ... Entering construct_ctu_hydro_source() on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 2.814815483 on level 1 Castro::reflux() at level 0 : time = 2.23000715 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.417814999 Gravity::multilevel_solve_for_new_phi() time = 0.958069289 Castro::construct_new_gravity() time = 1.074328917 on level 0 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.799171515e+21 ymom added: -1.177853541e+22 zmom added: -4.924059364e+18 eint added: 0 ener added: -4.148475158e+41 Castro::do_new_sources() time = 1.96138921 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901 Castro::construct_new_gravity() time = 0.029180862 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.726504457e+18 ymom added: -2.54093091e+19 zmom added: -1.252338631e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.141708963 on level 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.819509259e+24 ener added: 4.717583333e+24 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 0.02319655169 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286065e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.94814634e+22 ener added: 2.962962963e+20 Updating the rotational period from 30.52751552 s to 30.52751551 s. [STEP 5] Coarse TimeStep time: 47.08426128 [STEP 5] FAB kilobyte spread across MPI nodes: [833440 ... 934735] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.535297757 seconds Ending run at 07:29:11 UTC on 2024-03-21. Run time = 241.5162897 Run time without initialization = 229.8638729 Average number of zones advanced per microsecond: 0.022 Average number of zones advanced per microsecond per rank: 0.005 TinyProfiler total time across processes [min...avg...max]: 241.5 ... 241.5 ... 241.5 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 91.8986 103.2499 108.1788 44.79% Castro::computeTemp() 88 14.88 16.78 17.59 7.28% check_for_negative_density() 9 0.3057 5.34 16.79 6.95% amrex::Copy() 679 12.41 13.99 14.68 6.08% MultiFab::AddProduct() 4205 10.45 11.65 12.11 5.01% Castro::normalize_species() 87 10.73 11.4 11.66 4.83% FabArray::ParallelCopy_finish() 830 1.541 4.352 10.58 4.38% Castro::reset_internal_energy(Fab) 1476 8.314 9.262 9.618 3.98% Castro::estTimeStep() 26 3.987 5.363 8.681 3.59% FabArray::setVal() 5070 7.049 7.922 8.285 3.43% MultiFab::Subtract() 175 6.798 7.648 7.999 3.31% Gravity::fill_multipole_BCs() 11 4.305 4.484 4.748 1.97% Castro::reset_internal_energy(MultiFab) 88 3.29 3.828 4.128 1.71% FabArray::sum() 4205 3.429 3.842 3.993 1.65% FluxRegister::Reflux() 24 2.895 3.27 3.415 1.41% FabArray::ParallelCopy_nowait() 830 2.048 2.244 2.681 1.11% MLPoisson::Fsmooth() 1376 2.097 2.368 2.497 1.03% Castro::enforce_min_density() 87 1.508 1.761 1.94 0.80% Castro::wd_update() 6 0.513 0.9546 1.787 0.74% Castro::FluxRegCrseInit 9 1.479 1.639 1.732 0.72% Castro::reflux() 4 1.472 1.662 1.731 0.72% FillPatchInterp(MF) 30 1.234 1.341 1.467 0.61% Amr::InitAmr() 1 1.222 1.222 1.222 0.51% amrex::Add() 136 1.007 1.127 1.187 0.49% Castro::initMFs() 5 1.119 1.141 1.154 0.48% MLPoisson::Fapply() 318 0.8993 1.023 1.091 0.45% Castro::construct_old_gravity_source() 18 0.9043 1.019 1.06 0.44% FillBoundary_finish() 1553 0.2725 0.5205 0.9929 0.41% Castro::construct_new_gravity_source() 9 0.7425 0.8361 0.8812 0.36% Castro::problem_post_timestep() 10 0.7621 0.7951 0.8711 0.36% FillPatchIterator::Initialize 62 0.7346 0.8215 0.8552 0.35% Castro::volInBoundary() 42 0.6549 0.7396 0.8307 0.34% FabArray::Saxpy() 182 0.6882 0.7753 0.8128 0.34% VisMF::Write(FabArray) 3 0.5851 0.6565 0.6977 0.29% StateData::FillBoundary(geom) 50 0.5805 0.5916 0.6041 0.25% Castro::construct_new_source() 63 0.475 0.5281 0.5507 0.23% MultiFab::max() 18 0.01933 0.1916 0.5327 0.22% MLCellLinOp::applyBC() 1742 0.357 0.4296 0.4708 0.19% MultiFab::contains_nan() 18 0.3951 0.4434 0.4646 0.19% FillBoundary_nowait() 1553 0.3218 0.3791 0.414 0.17% FabArray::Xpay() 283 0.3415 0.3852 0.402 0.17% Castro::initialize_do_advance() 9 0.2384 0.2576 0.2647 0.11% FabArray::setDomainBndry() 92 0.1963 0.2188 0.2369 0.10% Amr::writePlotFile() 2 0.07994 0.135 0.2351 0.10% amrex::average_down 233 0.1923 0.2058 0.2183 0.09% Castro::construct_old_source() 63 0.1798 0.1974 0.2044 0.08% Castro::fill_hybrid_hydro_source() 27 0.1699 0.1901 0.1983 0.08% MLCellLinOp::defineAuxData() 11 0.1442 0.1582 0.1657 0.07% Castro::initData() 1 0.1219 0.1433 0.1611 0.07% MLPoisson::FFlux() 150 0.004587 0.04224 0.1551 0.06% MLCellLinOp::compGrad() 18 0.124 0.138 0.1441 0.06% FabArray::mult() 56 0.1145 0.1281 0.133 0.06% Castro::swap_state_time_levels() 9 0.08433 0.1008 0.1326 0.05% MLMG::addInterpCorrection() 155 0.1047 0.1204 0.1276 0.05% StateData::define() 25 0.1063 0.1187 0.1241 0.05% Castro::evaluate_source_change() 193 0.1018 0.1096 0.1134 0.05% Castro::initialize_advance() 9 0.0661 0.08279 0.1058 0.04% Gravity::get_new_grav_vector() 10 0.08598 0.0954 0.1008 0.04% Gravity::get_old_grav_vector() 9 0.07893 0.08743 0.09211 0.04% MLMG::solve() 10 0.02117 0.05761 0.09135 0.04% MLMG::prepareForSolve() 10 0.06089 0.06704 0.07072 0.03% MLCellLinOp::updateSolBC() 25 0.06666 0.06704 0.06753 0.03% FabArray::norminf(mask) 21 0.05408 0.06108 0.06388 0.03% AmrLevel::FillPatch() 62 0.04884 0.05516 0.05964 0.02% Castro::apply_tagging_restrictions() 6 0.04729 0.05252 0.0546 0.02% AmrMesh::MakeNewGrids() 6 0.04533 0.04687 0.05117 0.02% MLMG::ResNormInf() 54 0.00324 0.01612 0.04759 0.02% MLCellLinOp::updateCorBC() 17 0.04544 0.04575 0.04609 0.02% Gravity::multilevel_solve_for_new_phi() 10 0.02848 0.03598 0.04484 0.02% BndryData::define() 27 0.04153 0.04275 0.04396 0.02% FabArray::norminf() 147 0.03634 0.03925 0.04225 0.02% MLMG::MLResNormInf() 15 0.0001169 0.02448 0.03568 0.01% MLCellLinOp::reflux() 30 0.004247 0.01212 0.03559 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01779 0.02144 0.02798 0.01% Castro::FluxRegFineAdd() 9 0.02593 0.02606 0.02628 0.01% MLCellLinOp::defineBC() 11 0.02581 0.02599 0.02617 0.01% MultiFab::Multiply() 8 0.02148 0.02409 0.02521 0.01% Castro::build_fine_mask() 44 0.01707 0.0183 0.02024 0.01% MLMG::interpCorrection_1 17 0.01841 0.01874 0.01948 0.01% FabArrayBase::CPC::define() 416 0.01295 0.01473 0.01778 0.01% StateDataPhysBCFunct::() 95 0.001899 0.008341 0.01738 0.01% MLLinOp::makeSubCommunicator() 11 0.0004438 0.008149 0.01721 0.01% Castro::Castro() 5 0.01462 0.01636 0.0172 0.01% Castro::buildMetrics() 5 0.01503 0.0163 0.01707 0.01% FillPatchTwoLevels 31 0.01292 0.01415 0.01669 0.01% Castro::linear_to_hybrid_momentum() 1 0.01374 0.01544 0.01609 0.01% Castro::construct_old_gravity() 9 0.0003785 0.005362 0.01352 0.01% MLCellLinOp::prepareForSolve() 11 0.01137 0.01161 0.01189 0.00% Castro::enforce_consistent_e() 1 0.009107 0.01031 0.01082 0.00% Gravity::actual_multilevel_solve() 10 0.007795 0.00971 0.01076 0.00% AMRErrorTag::operator() 6 0.00875 0.009916 0.01041 0.00% FabArrayBase::getCPC() 1440 0.009563 0.009856 0.0102 0.00% Gravity::actual_solve_with_mlmg() 11 0.009059 0.009315 0.009602 0.00% Castro::finalize_do_advance() 9 0.007288 0.0088 0.009576 0.00% Gravity::update_max_rhs() 10 0.007402 0.008479 0.009113 0.00% Castro::apply_problem_tags() 6 0.006327 0.007119 0.007416 0.00% average_down_faces 48 0.005634 0.005661 0.00569 0.00% MLCellLinOp::setLevelBC() 19 0.003931 0.004695 0.005411 0.00% FabArray::FillBoundary() 1553 0.004151 0.004372 0.004795 0.00% AmrLevel::derive() 6 0.003631 0.004091 0.004278 0.00% MLCellLinOp::smooth() 688 0.00369 0.003817 0.003934 0.00% MLCGSolver::bicgstab 7 0 0.000959 0.003836 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0004452 0.002716 0.003803 0.00% amrex::Dot() 115 0 0.0009282 0.003713 0.00% FabArrayBase::getFB() 1553 0.00308 0.003229 0.003546 0.00% Gravity::test_composite_phi() 1 0.002186 0.002643 0.003313 0.00% FabArrayBase::FB::FB() 19 0.002474 0.002814 0.003082 0.00% FabArray::ParallelCopy() 830 0.002848 0.002996 0.003061 0.00% Castro::do_new_sources() 9 0.0005341 0.0011 0.002639 0.00% Castro::do_old_sources() 9 0.0004991 0.0009004 0.001776 0.00% main() 1 0.001523 0.001593 0.001759 0.00% AmrLevel::FillCoarsePatch() 5 0.0008773 0.001135 0.001754 0.00% MLLinOp::defineGrids() 11 0.001346 0.001439 0.001591 0.00% MLCellLinOp::apply() 318 0.0008825 0.001044 0.001491 0.00% Amr::regrid() 5 0.0009333 0.00109 0.001418 0.00% TagBoxArray::collate() 6 0.0004707 0.001012 0.001354 0.00% Castro::clean_state() 87 0.001238 0.001276 0.001319 0.00% Castro::construct_new_gravity() 9 0.0003679 0.0006659 0.001231 0.00% Amr::coarseTimeStep() 5 0.0004765 0.0009648 0.001133 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0002799 0.001119 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0008989 0.0009036 0.0009097 0.00% OwnerMask() 6 0.0007039 0.0008117 0.0008816 0.00% MLCellLinOp::correctionResidual() 189 0.0006682 0.0007085 0.0007821 0.00% MLMG::mgVcycle() 65 0.0005857 0.0006199 0.0006761 0.00% FillPatchSingleLevel 92 0.0005297 0.0005375 0.0005554 0.00% Castro::do_advance_ctu() 5 0.0004926 0.0005218 0.0005383 0.00% Amr::grid_places() 6 0.0004068 0.0004738 0.0005227 0.00% FillPatchIterator::FillFromLevel0() 31 0.0003585 0.0004218 0.0004618 0.00% TagBoxArray::mapPRD 6 0.0004183 0.0004319 0.0004401 0.00% Castro::subcycle_advance_ctu() 5 7.05e-05 0.0001737 0.0004346 0.00% MLMG:computeResOfCorrection() 155 0.0003864 0.0003972 0.0004113 0.00% MLMG::actualBottomSolve() 31 4.022e-05 0.0001231 0.000368 0.00% Castro::finalize_advance() 9 0.000352 0.0003568 0.0003614 0.00% MLMG::oneIter() 31 0.0003305 0.0003457 0.0003546 0.00% MLCellLinOp::solutionResidual() 72 0.0003321 0.0003413 0.0003476 0.00% MLMG::mgVcycle_down::0 31 0.0002838 0.0002959 0.0003265 0.00% FillPatchIterator::FillFromTwoLevels() 31 0.0002847 0.0002963 0.0003049 0.00% Castro::computeNewDt() 8 0.0001219 0.0001682 0.0002989 0.00% Gravity::solve_phi_with_mlmg() 11 0.0002312 0.0002534 0.000288 0.00% Castro::enforce_speed_limit() 87 0.0002596 0.0002709 0.0002807 0.00% Amr::timeStep() 9 0.0001835 0.0002333 0.0002795 0.00% MLMG::mgVcycle_down::1 31 0.0002398 0.0002465 0.0002567 0.00% Amr::defBaseLevel() 1 0.0002298 0.0002327 0.0002345 0.00% Castro::post_regrid() 10 0.0001941 0.0002053 0.0002144 0.00% MLMG::mgVcycle_bottom 65 0.0002114 0.0002124 0.0002134 0.00% MLMG::mgVcycle_down::2 31 0.0001883 0.0001941 0.000204 0.00% AmrLevel::AmrLevel(dm) 5 0.0001774 0.0001902 0.0001994 0.00% MLMG::mgVcycle_down::3 31 0.00019 0.0001928 0.0001964 0.00% MLMG::mgVcycle_up::0 31 0.0001801 0.0001898 0.0001959 0.00% makeSFC 33 0.0001616 0.0001764 0.0001952 0.00% FabArrayBase::TheFPinfo() 25 0.0001688 0.000178 0.0001878 0.00% MLMG::mgVcycle_down::4 31 0.000162 0.0001679 0.0001795 0.00% Amr::FinalizeInit() 1 2.935e-05 7.32e-05 0.0001773 0.00% Castro::check_for_nan() 18 0.0001295 0.0001379 0.0001491 0.00% Castro::react_state() 18 0.0001189 0.000131 0.0001427 0.00% MLMG::mgVcycle_up::1 31 0.0001348 0.0001382 0.0001404 0.00% MLMG::computeResWithCrseSolFineCor() 17 0.0001251 0.0001301 0.0001385 0.00% MLMG::mgVcycle_up::2 31 0.0001285 0.0001302 0.0001329 0.00% FPinfo::FPinfo() 3 0.0001126 0.000121 0.0001307 0.00% MLMG::mgVcycle_up::3 31 0.0001203 0.0001226 0.0001244 0.00% MLMG::mgVcycle_up::4 31 0.0001108 0.0001169 0.000124 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0001074 0.0001134 0.0001221 0.00% DistributionMapping::Distribute() 35 0.0001099 0.0001154 0.0001197 0.00% Castro::expand_state() 13 0.0001125 0.0001159 0.0001193 0.00% FabArray::RecvLayoutMask() 8 9.244e-05 9.937e-05 0.0001148 0.00% Castro::advance() 5 8.009e-05 9.665e-05 0.000114 0.00% MLPoisson::define() 11 9.899e-05 0.0001064 0.0001139 0.00% Castro::errorEst() 6 0.0001001 0.0001047 0.0001096 0.00% Amr::writeSmallPlotFile() 1 8.824e-05 9.89e-05 0.0001094 0.00% Castro::apply_source_to_state() 18 9.789e-05 0.0001027 0.0001087 0.00% Gravity::swapTimeLevels() 9 8.45e-05 8.774e-05 9.073e-05 0.00% MLMG::computeMLResidual() 15 7.745e-05 8.274e-05 8.915e-05 0.00% MLMG::computeResidual() 31 8.237e-05 8.511e-05 8.828e-05 0.00% Castro::post_timestep() 9 7.384e-05 8.041e-05 8.733e-05 0.00% ClusterList::intersect() 1 0 2.08e-05 8.321e-05 0.00% Castro::avgDown(state_indx) 20 6.228e-05 6.964e-05 8.067e-05 0.00% Castro::retry_advance_ctu() 5 3.546e-05 5.158e-05 7.331e-05 0.00% knapsack() 3 6.609e-05 6.91e-05 7.098e-05 0.00% MLLinOp::define() 11 6.159e-05 6.529e-05 7.075e-05 0.00% MLMG::miniCycle() 34 6.527e-05 6.762e-05 7.039e-05 0.00% MLMG::getGradSolution() 10 6.016e-05 6.332e-05 6.522e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 6.21e-05 6.32e-05 6.5e-05 0.00% Castro::create_source_corrector() 9 5.63e-05 5.919e-05 6.2e-05 0.00% Amr::initSubcycle() 1 4.975e-05 5.415e-05 5.841e-05 0.00% amrex::average_down_w_geom 20 4.828e-05 5.205e-05 5.732e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 4.954e-05 5.311e-05 5.704e-05 0.00% MLPoisson::prepareForSolve() 11 4.437e-05 4.784e-05 5.25e-05 0.00% Castro::construct_new_hybrid_source() 9 4.585e-05 4.904e-05 5.16e-05 0.00% Castro::init(old) 3 3.966e-05 4.324e-05 4.613e-05 0.00% Gravity::set_mass_offset() 6 3.167e-05 3.765e-05 4.279e-05 0.00% BoxList::complementIn 12 2.202e-05 2.968e-05 3.923e-05 0.00% Castro::avgDown() 4 2.947e-05 3.285e-05 3.781e-05 0.00% AmrMesh-cluster 1 0 9.284e-06 3.713e-05 0.00% Castro::construct_old_hybrid_source() 9 3.362e-05 3.499e-05 3.661e-05 0.00% DistributionMapping::KnapSackDoIt() 3 2.965e-05 3.264e-05 3.519e-05 0.00% MLMG::MLRhsNormInf() 10 3.15e-05 3.218e-05 3.289e-05 0.00% MLMG::compResidual() 1 2.618e-05 2.904e-05 3.28e-05 0.00% BoxList::parallelComplementIn() 12 2.449e-05 2.654e-05 3.186e-05 0.00% Castro::computeInitialDt() 2 1.192e-05 2.123e-05 3.123e-05 0.00% Castro::post_init() 1 2.681e-05 2.894e-05 3.118e-05 0.00% Amr::bldFineLevels() 1 1.797e-05 2.294e-05 2.727e-05 0.00% Castro::construct_new_rotation_source() 9 1.911e-05 2.184e-05 2.565e-05 0.00% Castro::construct_old_rotation_source() 9 2.034e-05 2.249e-05 2.474e-05 0.00% Castro::derive() 6 1.812e-05 1.932e-05 2.04e-05 0.00% Castro::init() 1 1.598e-05 1.622e-05 1.64e-05 0.00% Amr::init() 1 6.86e-06 9.318e-06 1.607e-05 0.00% ClusterList::chop() 1 0 2.563e-06 1.025e-05 0.00% Amr::InitializeInit() 1 7.463e-06 8.438e-06 9.295e-06 0.00% Amr::initialInit() 1 4.15e-06 5.796e-06 6.653e-06 0.00% knapsack()swap 1 3.686e-06 3.767e-06 3.909e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 241.5 241.5 241.5 100.00% Amr::coarseTimeStep() 5 229.3 229.3 229.3 94.95% Amr::timeStep() 9 228 228 228 94.40% Castro::advance() 5 200.9 200.9 200.9 83.18% Castro::subcycle_advance_ctu() 5 192.3 192.7 193.5 80.11% Castro::do_advance_ctu() 5 192.3 192.7 193.5 80.11% Castro::construct_ctu_hydro_source() 9 118.3 118.4 118.5 49.07% Castro::clean_state() 87 71.49 79.81 83.31 34.50% Castro::computeTemp() 88 45.04 50.68 53 21.94% Castro::reset_internal_energy(MultiFab) 88 30.16 33.9 35.41 14.66% Castro::evaluate_source_change() 193 17.58 19.65 20.41 8.45% Castro::enforce_min_density() 87 16.52 18.61 19.54 8.09% Castro::post_regrid() 10 15.74 17.24 17.83 7.38% Amr::regrid() 5 15.59 16.92 17.45 7.22% check_for_negative_density() 9 0.3057 5.34 16.79 6.95% Castro::initialize_do_advance() 9 14.55 15.04 15.41 6.38% amrex::Copy() 679 12.41 13.99 14.68 6.08% AmrLevel::FillPatch() 62 8.213 9.665 13.68 5.67% FabArray::ParallelCopy() 830 4.288 6.658 12.74 5.28% FillPatchIterator::Initialize 62 6.553 8.063 12.26 5.08% MultiFab::AddProduct() 4205 10.45 11.65 12.11 5.01% Gravity::solve_phi_with_mlmg() 11 11.33 11.51 11.77 4.87% Castro::normalize_species() 87 10.73 11.4 11.66 4.83% Gravity::multilevel_solve_for_new_phi() 10 11.04 11.21 11.45 4.74% Gravity::actual_multilevel_solve() 10 11 11.17 11.42 4.73% Castro::do_old_sources() 9 10.9 10.96 11.04 4.57% Castro::do_new_sources() 9 9.16 10.25 10.72 4.44% FabArray::ParallelCopy_finish() 830 1.541 4.352 10.58 4.38% Amr::init() 1 10.43 10.43 10.43 4.32% FillPatchSingleLevel 92 4.334 5.897 10.25 4.24% Castro::post_timestep() 9 10.01 10.01 10.01 4.15% Amr::initialInit() 1 9.897 9.897 9.897 4.10% FillPatchIterator::FillFromLevel0() 31 3.966 5.488 9.8 4.06% Castro::reset_internal_energy(Fab) 1476 8.314 9.262 9.618 3.98% Castro::reflux() 4 8.922 8.971 8.991 3.72% Castro::estTimeStep() 26 3.987 5.363 8.681 3.59% Castro::initialize_advance() 9 7.414 8.214 8.552 3.54% FabArray::setVal() 5070 7.049 7.922 8.285 3.43% MultiFab::Subtract() 175 6.798 7.648 7.999 3.31% Gravity::actual_solve_with_mlmg() 11 6.919 6.926 6.932 2.87% Castro::construct_new_gravity() 9 6.448 6.606 6.841 2.83% MLMG::solve() 10 6.42 6.436 6.456 2.67% Amr::FinalizeInit() 1 5.886 5.999 6.283 2.60% Castro::expand_state() 13 5.182 5.512 6.165 2.55% MLMG::oneIter() 31 5.193 5.215 5.225 2.16% Gravity::fill_multipole_BCs() 11 4.386 4.561 4.817 1.99% MLMG::mgVcycle() 65 4.364 4.397 4.414 1.83% FluxRegister::Reflux() 24 4.355 4.368 4.374 1.81% Castro::construct_old_gravity() 9 0.6844 1.811 4.227 1.75% Gravity::get_old_grav_vector() 9 0.6709 1.805 4.227 1.75% Amr::InitializeInit() 1 3.613 3.898 4.011 1.66% Amr::defBaseLevel() 1 3.613 3.898 4.011 1.66% Castro::finalize_do_advance() 9 1.343 2.132 4 1.66% FabArray::sum() 4205 3.429 3.842 3.993 1.65% Castro::problem_post_timestep() 10 2.764 3.182 3.93 1.63% Castro::computeNewDt() 8 2.15 2.623 3.787 1.57% MLCellLinOp::smooth() 688 3.31 3.344 3.381 1.40% Castro::post_init() 1 2.651 2.821 3.155 1.31% FabArray::ParallelCopy_nowait() 830 2.106 2.303 2.744 1.14% Castro::FluxRegCrseInit 9 1.79 2.06 2.643 1.09% Castro::initData() 1 2.204 2.464 2.557 1.06% MLPoisson::Fsmooth() 1376 2.097 2.368 2.497 1.03% Castro::wd_update() 6 1.276 1.694 2.442 1.01% Castro::construct_new_source() 63 1.753 1.965 2.053 0.85% MLMG::mgVcycle_down::0 31 1.913 1.993 2.044 0.85% MLCellLinOp::applyBC() 1742 1.121 1.34 1.774 0.73% FillPatchIterator::FillFromTwoLevels() 31 1.506 1.535 1.568 0.65% FillPatchTwoLevels 31 1.505 1.535 1.568 0.65% Amr::bldFineLevels() 1 1.255 1.415 1.474 0.61% FillPatchInterp(MF) 30 1.234 1.341 1.467 0.61% MLMG::mgVcycle_up::0 31 1.396 1.427 1.442 0.60% FabArray::FillBoundary() 1553 0.6499 0.91 1.417 0.59% Castro::Castro() 5 1.354 1.389 1.414 0.59% MLCellLinOp::apply() 318 1.287 1.325 1.352 0.56% Amr::InitAmr() 1 1.222 1.222 1.222 0.51% amrex::Add() 136 1.007 1.127 1.187 0.49% Castro::initMFs() 5 1.124 1.146 1.159 0.48% MLPoisson::Fapply() 318 0.8993 1.023 1.091 0.45% Amr::writePlotFile() 2 1.068 1.069 1.069 0.44% MLCellLinOp::solutionResidual() 72 0.9707 1.03 1.061 0.44% Castro::construct_old_gravity_source() 18 0.9043 1.019 1.06 0.44% Castro::construct_old_source() 63 0.9091 1.017 1.057 0.44% FillBoundary_finish() 1553 0.2725 0.5205 0.9929 0.41% Castro::computeInitialDt() 2 0.5042 0.6172 0.9015 0.37% Castro::construct_new_gravity_source() 9 0.7425 0.8361 0.8812 0.36% Castro::volInBoundary() 42 0.6549 0.7397 0.8308 0.34% MLCellLinOp::correctionResidual() 189 0.8071 0.8141 0.8265 0.34% Gravity::get_new_grav_vector() 10 0.795 0.8045 0.8146 0.34% FabArray::Saxpy() 182 0.6882 0.7753 0.8128 0.34% MLMG:computeResOfCorrection() 155 0.6505 0.69 0.7142 0.30% VisMF::Write(FabArray) 3 0.5851 0.6565 0.6977 0.29% StateDataPhysBCFunct::() 95 0.5824 0.5999 0.6148 0.25% StateData::FillBoundary(geom) 50 0.5805 0.5916 0.6041 0.25% Gravity::update_max_rhs() 10 0.0779 0.2488 0.5841 0.24% MLMG::computeResWithCrseSolFineCor() 17 0.5771 0.5778 0.5792 0.24% MultiFab::max() 18 0.01933 0.1916 0.5327 0.22% MLMG::computeMLResidual() 15 0.4229 0.4444 0.4939 0.20% Castro::init() 1 0.4837 0.4857 0.4874 0.20% AmrLevel::FillCoarsePatch() 5 0.4837 0.4857 0.4874 0.20% Castro::check_for_nan() 18 0.3953 0.4435 0.4647 0.19% MultiFab::contains_nan() 18 0.3951 0.4434 0.4646 0.19% MLMG::mgVcycle_down::1 31 0.3072 0.3555 0.4285 0.18% FillBoundary_nowait() 1553 0.3274 0.3851 0.4202 0.17% Gravity::test_composite_phi() 1 0.4127 0.4141 0.4158 0.17% FabArray::Xpay() 283 0.3415 0.3852 0.402 0.17% amrex::average_down 233 0.2694 0.3631 0.4017 0.17% MLCellLinOp::reflux() 30 0.1774 0.2296 0.3676 0.15% MLMG::computeResidual() 31 0.3525 0.3593 0.3625 0.15% Castro::init(old) 3 0.3565 0.3581 0.3589 0.15% Castro::apply_source_to_state() 18 0.2755 0.3105 0.3254 0.13% MLPoisson::define() 11 0.2727 0.2824 0.2914 0.12% Castro::react_state() 18 0.2139 0.2458 0.2704 0.11% FabArray::setDomainBndry() 92 0.1963 0.2188 0.2369 0.10% MLMG::prepareForSolve() 10 0.2144 0.2217 0.228 0.09% MLMG::mgVcycle_up::1 31 0.2213 0.2242 0.2259 0.09% MLCellLinOp::defineAuxData() 11 0.1697 0.1867 0.1992 0.08% Castro::fill_hybrid_hydro_source() 27 0.1699 0.1901 0.1983 0.08% Amr::grid_places() 6 0.1879 0.188 0.188 0.08% AmrMesh::MakeNewGrids() 6 0.1874 0.1875 0.1876 0.08% MLMG::addInterpCorrection() 155 0.133 0.1683 0.1865 0.08% MLMG::mgVcycle_bottom 65 0.1495 0.1569 0.1787 0.07% MLMG::getGradSolution() 10 0.1703 0.1727 0.1747 0.07% MLCellLinOp::compGrad() 18 0.1702 0.1727 0.1747 0.07% MLPoisson::FFlux() 150 0.004587 0.04224 0.1551 0.06% MLMG::miniCycle() 34 0.1495 0.1496 0.1499 0.06% FabArray::mult() 56 0.1145 0.1281 0.133 0.06% Castro::construct_new_hybrid_source() 9 0.113 0.1267 0.1327 0.05% Castro::swap_state_time_levels() 9 0.08433 0.1008 0.1326 0.05% AmrLevel::AmrLevel(dm) 5 0.1065 0.1189 0.1243 0.05% StateData::define() 25 0.1063 0.1187 0.1241 0.05% Castro::avgDown() 4 0.1175 0.12 0.1228 0.05% Castro::avgDown(state_indx) 20 0.1175 0.12 0.1228 0.05% amrex::average_down_w_geom 20 0.1174 0.1199 0.1227 0.05% Castro::errorEst() 6 0.1018 0.1141 0.1201 0.05% MLMG::ResNormInf() 54 0.07486 0.08519 0.1092 0.05% MLCellLinOp::updateSolBC() 25 0.07722 0.0856 0.09821 0.04% MLCellLinOp::defineBC() 11 0.0836 0.08557 0.0895 0.04% MLMG::computeResWithCrseCorFineCor() 17 0.08493 0.08574 0.08743 0.04% MLMG::mgVcycle_up::3 31 0.02597 0.0673 0.08121 0.03% MLMG::mgVcycle_down::2 31 0.06815 0.07204 0.07877 0.03% MLMG::MLResNormInf() 15 0.04431 0.06699 0.07518 0.03% Castro::construct_old_hybrid_source() 9 0.05699 0.0635 0.06605 0.03% FabArray::norminf(mask) 21 0.05408 0.06108 0.06388 0.03% BndryData::define() 27 0.05638 0.05835 0.0621 0.03% MLMG::interpCorrection_1 17 0.02738 0.03651 0.05988 0.02% Castro::apply_tagging_restrictions() 6 0.04729 0.05252 0.0546 0.02% MLMG::mgVcycle_up::2 31 0.05254 0.05282 0.05303 0.02% MLCellLinOp::updateCorBC() 17 0.04936 0.05021 0.05173 0.02% Castro::derive() 6 0.03935 0.04444 0.04758 0.02% AmrLevel::derive() 6 0.03933 0.04442 0.04756 0.02% FabArray::norminf() 147 0.03634 0.03925 0.04225 0.02% Castro::buildMetrics() 5 0.0371 0.03862 0.0413 0.02% Gravity::swapTimeLevels() 9 0.03167 0.03722 0.03968 0.02% MLMG::mgVcycle_down::3 31 0.0312 0.03226 0.03482 0.01% MLMG::MLRhsNormInf() 10 0.02488 0.02816 0.02958 0.01% TagBoxArray::mapPRD 6 0.003913 0.01441 0.02924 0.01% MLMG::actualBottomSolve() 31 4.022e-05 0.007329 0.02919 0.01% MLCGSolver::bicgstab 7 0 0.007107 0.02843 0.01% FabArrayBase::getCPC() 1440 0.02251 0.02458 0.02798 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 622 0.01779 0.02144 0.02798 0.01% Castro::FluxRegFineAdd() 9 0.02593 0.02606 0.02628 0.01% MultiFab::Multiply() 8 0.02148 0.02409 0.02521 0.01% Gravity::average_fine_ec_onto_crse_ec() 8 0.01935 0.01992 0.02149 0.01% MLMG::compResidual() 1 0.01919 0.02052 0.02107 0.01% Castro::build_fine_mask() 44 0.01707 0.0183 0.02024 0.01% MLLinOp::define() 11 0.002314 0.01005 0.01929 0.01% MLLinOp::defineGrids() 11 0.002248 0.009982 0.01922 0.01% MLMG::mgVcycle_down::4 31 0.005407 0.008802 0.01893 0.01% FabArrayBase::CPC::define() 416 0.01295 0.01473 0.01778 0.01% MLLinOp::makeSubCommunicator() 11 0.0004438 0.008149 0.01721 0.01% Castro::linear_to_hybrid_momentum() 1 0.01374 0.01544 0.01609 0.01% MLMG::mgVcycle_up::4 31 0.004414 0.006953 0.01449 0.01% MLPoisson::prepareForSolve() 11 0.01143 0.01166 0.01194 0.00% MLCellLinOp::prepareForSolve() 11 0.01137 0.01161 0.01189 0.00% Castro::enforce_consistent_e() 1 0.009107 0.01031 0.01082 0.00% AMRErrorTag::operator() 6 0.00875 0.009916 0.01041 0.00% average_down_faces 48 0.008976 0.009229 0.009554 0.00% Castro::apply_problem_tags() 6 0.006327 0.007119 0.007416 0.00% FabArrayBase::getFB() 1553 0.005578 0.006043 0.006498 0.00% MLCellLinOp::setLevelBC() 19 0.003931 0.004695 0.005411 0.00% FabArrayBase::TheFPinfo() 25 0.0007698 0.003048 0.004146 0.00% FPinfo::FPinfo() 3 0.0006009 0.00287 0.003958 0.00% DistributionMapping::KnapSackDoIt() 3 0.0004858 0.002749 0.003827 0.00% DistributionMapping::LeastUsedCPUs() 5 0.0004452 0.002716 0.003803 0.00% amrex::Dot() 115 0 0.0009282 0.003713 0.00% FabArrayBase::FB::FB() 19 0.002474 0.002814 0.003082 0.00% MLCGSolver::ParallelAllReduce 183 0 0.0007681 0.003072 0.00% TagBoxArray::collate() 6 0.0004707 0.001012 0.001354 0.00% OwnerMask() 6 0.0007039 0.0008117 0.0008816 0.00% MLLinOp::makeAgglomeratedDMap 11 0.0003677 0.0003942 0.000424 0.00% Castro::finalize_advance() 9 0.000352 0.0003568 0.0003614 0.00% makeSFC 33 0.0002603 0.0002808 0.0003019 0.00% Castro::enforce_speed_limit() 87 0.0002596 0.0002709 0.0002807 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001258 0.0001373 0.0001476 0.00% AmrMesh-cluster 1 0 3.265e-05 0.0001306 0.00% DistributionMapping::Distribute() 35 0.0001099 0.0001154 0.0001197 0.00% FabArray::RecvLayoutMask() 8 9.244e-05 9.937e-05 0.0001148 0.00% Amr::writeSmallPlotFile() 1 8.824e-05 9.89e-05 0.0001094 0.00% ClusterList::intersect() 1 0 2.08e-05 8.321e-05 0.00% knapsack() 3 6.978e-05 7.287e-05 7.476e-05 0.00% Castro::retry_advance_ctu() 5 3.546e-05 5.158e-05 7.331e-05 0.00% BoxList::parallelComplementIn() 12 4.65e-05 5.622e-05 6.436e-05 0.00% Castro::create_source_corrector() 9 5.63e-05 5.919e-05 6.2e-05 0.00% Amr::initSubcycle() 1 4.975e-05 5.415e-05 5.841e-05 0.00% Gravity::set_mass_offset() 6 3.167e-05 3.765e-05 4.279e-05 0.00% BoxList::complementIn 12 2.202e-05 2.968e-05 3.923e-05 0.00% Castro::construct_new_rotation_source() 9 1.911e-05 2.184e-05 2.565e-05 0.00% Castro::construct_old_rotation_source() 9 2.034e-05 2.249e-05 2.474e-05 0.00% ClusterList::chop() 1 0 2.563e-06 1.025e-05 0.00% knapsack()swap 1 3.686e-06 3.767e-06 3.909e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (v2024-27-ga9b452ab5e8b) finalized