MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (22.11-34-g88fe04f00600) initialized Starting run at 09:28:21 UTC on 2022-11-29. Successfully read inputs file ... Castro git describe: 22.11-19-g093c32c96 AMReX git describe: 22.11-34-g88fe04f00 Microphysics git describe: 22.11-47-g1539b018 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Warning: the following castro.* parameters are ignored castro.center = 0.0e0 0.0e0 0.0e0 Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.065003882 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.061895299 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.744292723 Gravity::fill_multipole_BCs() time = 0.741909208 ... norm of composite residual at level 0 3.509358608e-10 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.764773662 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (64,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.287624606e+47 ener added: 6.129070894e+49 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.44936293e+25 ymom added: -1.046793351e+26 zmom added: -4.133074948e+24 eint added: 0 ener added: -1.270533196e+50 Castro::do_old_sources() time = 4.225962092 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 38.35477038 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.00547245e+47 ener added: 1.651139801e+46 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 1.245111228 Gravity::solve_for_phi() time = 3.38238531 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.440206272e+49 ener added: 2.911791643e+45 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.451207933e+26 ymom added: 1.075144417e+26 zmom added: -5.613375295e+26 eint added: 0 ener added: 1.260457978e+50 Castro::do_new_sources() time = 4.713958428 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.184461852e+27 ener added: 2.243834889e+27 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.01251569808 [Level 0 step 1] Advanced 884736 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.190352593e+26 ener added: 6.213783704e+26 Updating the rotational period from 30.54368673 s to 30.45939751 s. [STEP 1] Coarse TimeStep time: 74.51539842 [STEP 1] FAB kilobyte spread across MPI nodes: [679617 ... 761594] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.06284287 new finest: 1 Castro::numpts_1d at level 1 is 340 Warning: the following castro.* parameters are ignored castro.center = 0.0e0 0.0e0 0.0e0 Gravity::fill_multipole_BCs() time = 0.779008716 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.306135866e+46 ener added: 9.211440366e+51 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135286e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (87,103,92) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135286e-09 [Level 0 step 2] ADVANCE with dt = 2.526135286e-09  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 2.13183832 Gravity::solve_for_phi() time = 4.538365087 ... testing grad_phi_curr after doing single level solve Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.200514105e+40 ener added: 6.433158191e+33 Contributions to the state from old-time sources: mass added: 0 xmom added: 5.533896722e+24 ymom added: 4.756222073e+26 zmom added: -2.866993882e+23 eint added: 0 ener added: -1.721443002e+45 Castro::do_old_sources() time = 3.618063683 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 38.15482417 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 1.103743688 Gravity::solve_for_phi() time = 1.841274639 ... testing grad_phi_curr before adding comp_minus_level_grad_phi ... testing grad_phi_curr after adding comp_minus_level_grad_phi Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.4637558e+41 ener added: 7.48556853e+35 Contributions to the state from new-time sources: mass added: 0 xmom added: -7.716487915e+19 ymom added: -6.58763018e+21 zmom added: -4.706669256e+19 eint added: 0 ener added: -3.22371126e+41 Castro::do_new_sources() time = 3.847239611 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.147587407e+26 ener added: 8.096296296e+22 ...estimated hydro-limited timestep at level 0: 0.05461716025 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716025  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.01238543088 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE with dt = 2.526135286e-09 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.849569753e+31 ener added: 2.785937322e+31  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135286e-09 Estimated number of subcycles remaining: 1 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.147330623e+31 ener added: 1.147330701e+31 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.551111111e+23 ener added: 6.281481481e+23 ... old-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.054829234 ... testing grad_phi_curr after doing single level solve Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.197253775e+40 ener added: 1.79168512e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: 7.575964413e+22 ymom added: 1.534769107e+22 zmom added: 6.442093022e+21 eint added: 0 ener added: -2.586789965e+47 Castro::do_old_sources() time = 0.251163516 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 4.218976197 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.204912576e+51 ener added: 6.747117479e+48 ... new-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.197495282 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564837764e+47 ener added: 4.559472601e+41 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.428286313e+20 ymom added: 9.887202622e+19 zmom added: 4.139444922e+19 eint added: 0 ener added: 2.564611685e+47 Castro::do_new_sources() time = 0.286593503 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.182413467e+29 ener added: 3.182412331e+29 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.00947139178 [Level 1 step 2] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.0315e+24 ener added: 5.917722222e+24 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.564833212e+47 ener added: 1.552243693e+40 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.564833238e+47 ener added: 7.225621114e+31 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.447965841e+20 ymom added: 1.08852003e+20 zmom added: 4.139587955e+19 eint added: 0 ener added: 2.564611684e+47 Castro::do_new_sources() time = 0.2853838 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.46376329e+41 ener added: 4.846972222e+26 Contributions to the state from new-time sources: mass added: 0 xmom added: -7.823471791e+19 ymom added: -6.593355454e+21 zmom added: -4.706705798e+19 eint added: 0 ener added: -3.22371126e+41 Castro::do_new_sources() time = 3.889644365 Castro::reflux() at level 0 : time = 11.71214275 Updating the rotational period from 30.45939751 s to 30.52749735 s. [STEP 2] Coarse TimeStep time: 105.4709207 [STEP 2] FAB kilobyte spread across MPI nodes: [730873 ... 816637] STEP = 2 TIME = 0.005426075163 DT = 2.526135286e-09 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008131491259 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.287037368e-07 ... but limiting to: 2.778748814e-09 = 1.1 * 2.526135286e-09 Now regridding at level lbase = 0 grid_places() time: 0.066857429 new finest: 1 Castro::numpts_1d at level 1 is 340 Warning: the following castro.* parameters are ignored castro.center = 0.0e0 0.0e0 0.0e0 Gravity::fill_multipole_BCs() time = 0.840452396 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.290843056e+25 ener added: 5.2756875e+25 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008131491259 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131491259 ...estimated hydro-limited timestep at level 1: 6.287037368e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.287037368e-07 [Level 0 step 3] ADVANCE with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 2.298105846 Gravity::solve_for_phi() time = 4.635901526 ... testing grad_phi_curr after doing single level solve Contributions to the state from old-time sources: mass added: 0 xmom added: 6.082514737e+24 ymom added: 5.231863911e+26 zmom added: -3.188631137e+23 eint added: 0 ener added: -1.937110674e+45 Castro::do_old_sources() time = 3.581166489 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 38.55862631 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 1.192216576 Gravity::solve_for_phi() time = 1.977918029 ... testing grad_phi_curr before adding comp_minus_level_grad_phi ... testing grad_phi_curr after adding comp_minus_level_grad_phi Contributions to the state from new-time sources: mass added: 0 xmom added: -8.784437903e+19 ymom added: -8.206350317e+21 zmom added: 1.85662131e+19 eint added: 0 ener added: -3.427512069e+41 Castro::do_new_sources() time = 3.969617282 ...estimated hydro-limited timestep at level 0: 0.0008131520194 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008131520194  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.01231324669 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE with dt = 2.778748814e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778748814e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.053628572 ... testing grad_phi_curr after doing single level solve Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.81869149e+40 ener added: 1.21221773e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.463160029e+22 ymom added: 6.198106009e+21 zmom added: 4.010293339e+21 eint added: 0 ener added: -2.344787748e+45 Castro::do_old_sources() time = 0.247282384 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 4.875929261 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.83673455e+48 ener added: 2.077900047e+42 ... new-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.113985326 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244373444e+43 ener added: 1.746259518e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.399175116e+20 ymom added: 3.332663724e+19 zmom added: 1.373026898e+21 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.280921954 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.111194167e+26 ener added: 1.111475463e+26 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.008613743365 [Level 1 step 3] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.381037037e+24 ener added: 6.674962963e+24 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -5.244355982e+43 ener added: 1.039595775e+40 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 5.244357361e+43 ener added: 4.093450833e+26 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.427566985e+20 ymom added: 3.795183406e+19 zmom added: 1.373027012e+21 eint added: 0 ener added: 5.303047905e+43 Castro::do_new_sources() time = 0.28213848 Contributions to the state from new-time sources: mass added: 0 xmom added: -9.987953374e+19 ymom added: -8.28346278e+21 zmom added: 1.85695193e+19 eint added: 0 ener added: -3.427512069e+41 Castro::do_new_sources() time = 3.958085202 Castro::reflux() at level 0 : time = 11.45746097 Updating the rotational period from 30.52749735 s to 30.52751553 s. [STEP 3] Coarse TimeStep time: 104.2485528 [STEP 3] FAB kilobyte spread across MPI nodes: [730873 ... 816637] STEP = 3 TIME = 0.005426077942 DT = 2.778748814e-09 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008130329735 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.564309766e-07 ... but limiting to: 3.056623696e-09 = 1.1 * 2.778748814e-09 Now regridding at level lbase = 0 grid_places() time: 0.066595982 new finest: 1 Castro::numpts_1d at level 1 is 340 Warning: the following castro.* parameters are ignored castro.center = 0.0e0 0.0e0 0.0e0 Gravity::fill_multipole_BCs() time = 0.804553286 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.149562963e+25 ener added: 2.174896296e+25 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0008130329735 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008130329735 ...estimated hydro-limited timestep at level 1: 6.564309766e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.564309766e-07 [Level 0 step 4] ADVANCE with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 2.272269634 Gravity::solve_for_phi() time = 4.58635681 ... testing grad_phi_curr after doing single level solve Contributions to the state from old-time sources: mass added: 0 xmom added: 6.695019149e+24 ymom added: 5.755040134e+26 zmom added: -3.473144628e+23 eint added: 0 ener added: -2.130751807e+45 Castro::do_old_sources() time = 3.567959834 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 36.43380305 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 1.295182145 Gravity::solve_for_phi() time = 2.074497606 ... testing grad_phi_curr before adding comp_minus_level_grad_phi ... testing grad_phi_curr after adding comp_minus_level_grad_phi Contributions to the state from new-time sources: mass added: 0 xmom added: -1.121260155e+20 ymom added: -1.001412316e+22 zmom added: 2.146366892e+19 eint added: 0 ener added: -3.723007447e+41 Castro::do_new_sources() time = 3.947062911 ...estimated hydro-limited timestep at level 0: 0.000813036171 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000813036171  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.01286111794 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE with dt = 3.056623696e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056623695e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.068122536 ... testing grad_phi_curr after doing single level solve Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.150425294e+40 ener added: 1.373884894e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 9.20986439e+22 ymom added: 1.180591621e+22 zmom added: 7.87315684e+21 eint added: 0 ener added: -2.520808161e+45 Castro::do_old_sources() time = 0.248168997 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 4.126290978 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.318881551e+48 ener added: 1.675623034e+42 ... new-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.134410781 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.983601425e+43 ener added: 1.617265924e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.743510049e+20 ymom added: 3.277719809e+19 zmom added: -4.057743965e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.283728048 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.022010509e+26 ener added: 1.016131991e+26 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.009680199307 [Level 1 step 4] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.038425926e+23 ener added: 7.586111111e+22 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.983585253e+43 ener added: 1.072608177e+39 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.74160253e+20 ymom added: 3.852379121e+19 zmom added: -4.057846343e+19 eint added: 0 ener added: 4.042918443e+43 Castro::do_new_sources() time = 0.280376843 Contributions to the state from new-time sources: mass added: 0 xmom added: -9.899019135e+19 ymom added: -1.001339929e+22 zmom added: 2.146665326e+19 eint added: 0 ener added: -3.723007447e+41 Castro::do_new_sources() time = 3.666224999 Castro::reflux() at level 0 : time = 11.18312364 Updating the rotational period from 30.52751553 s to 30.52751552 s. [STEP 4] Coarse TimeStep time: 99.23116317 [STEP 4] FAB kilobyte spread across MPI nodes: [730873 ... 816637] STEP = 4 TIME = 0.005426080998 DT = 3.056623696e-09 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002506596503 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.863488975e-07 ... but limiting to: 3.362286065e-09 = 1.1 * 3.056623696e-09 Now regridding at level lbase = 0 grid_places() time: 0.058974 new finest: 1 Castro::numpts_1d at level 1 is 340 Warning: the following castro.* parameters are ignored castro.center = 0.0e0 0.0e0 0.0e0 Gravity::fill_multipole_BCs() time = 0.773278127 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 4 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 ...estimated hydro-limited timestep at level 0: 0.0002506596503 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002506596503 ...estimated hydro-limited timestep at level 1: 6.863488975e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.863488975e-07 [Level 0 step 5] ADVANCE with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 2.039421038 Gravity::solve_for_phi() time = 4.418552875 ... testing grad_phi_curr after doing single level solve Contributions to the state from old-time sources: mass added: 0 xmom added: 7.366717342e+24 ymom added: 6.330531845e+26 zmom added: -3.819676135e+23 eint added: 0 ener added: -2.343742535e+45 Castro::do_old_sources() time = 3.62519849 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 38.53726074 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 1.228150607 Gravity::solve_for_phi() time = 1.983806123 ... testing grad_phi_curr before adding comp_minus_level_grad_phi ... testing grad_phi_curr after adding comp_minus_level_grad_phi Contributions to the state from new-time sources: mass added: 0 xmom added: -1.306164815e+20 ymom added: -1.196819457e+22 zmom added: 2.60298934e+19 eint added: 0 ener added: -4.148475152e+41 Castro::do_new_sources() time = 4.038644451 ...estimated hydro-limited timestep at level 0: 0.0006828000901 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006828000901  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.01229397443 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE with dt = 3.362286065e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286065e-09 Estimated number of subcycles remaining: 1 ... old-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.057694816 ... testing grad_phi_curr after doing single level solve Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -2.560109373e+40 ener added: 1.662186954e+36 Contributions to the state from old-time sources: mass added: 0 xmom added: 1.004646015e+23 ymom added: 1.770887431e+22 zmom added: 8.567359701e+21 eint added: 0 ener added: -2.728291723e+45 Castro::do_old_sources() time = 0.258425931 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 4.173784075 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.05099824e+48 ener added: 1.448876286e+42 ... new-time level Poisson gravity solve at level 1 Gravity::solve_for_phi() time = 0.128549567 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.326552265e+43 ener added: 1.577214716e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -3.118154583e+20 ymom added: -4.647714815e+18 zmom added: -2.699847517e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.292346883 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.32370463e+25 ener added: 2.319497685e+25 ...estimated hydro-limited timestep at level 1: 7.169281148e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.169281148e-07  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.009336934881 [Level 1 step 5] Advanced 65536 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.3e+22 ener added: 3.333333333e+20 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.326536493e+43 ener added: 7.40740893e+38 Contributions to the state from new-time sources: mass added: 0 xmom added: -3.123342543e+20 ymom added: 1.93834928e+19 zmom added: -2.699906052e+19 eint added: 0 ener added: 3.390623254e+43 Castro::do_new_sources() time = 0.284435534 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.405558708e+20 ymom added: -1.207441756e+22 zmom added: 2.603347761e+19 eint added: 0 ener added: -4.148475152e+41 Castro::do_new_sources() time = 3.92347352 Castro::reflux() at level 0 : time = 11.68311229 Updating the rotational period from 30.52751552 s to 30.52751552 s. [STEP 5] Coarse TimeStep time: 102.7637448 [STEP 5] FAB kilobyte spread across MPI nodes: [730873 ... 816637] STEP = 5 TIME = 0.005426084361 DT = 3.362286065e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.805507393 seconds Ending run at 09:36:48 UTC on 2022-11-29. Run time = 507.4161834 Run time without initialization = 487.0373845 Average number of zones advanced per microsecond: 0.010 Average number of zones advanced per microsecond per rank: 0.002 TinyProfiler total time across processes [min...avg...max]: 507.4 ... 507.4 ... 507.4 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 9 181.0735 198.6776 207.4410 40.88% Castro::computeTemp() 104 35.52 39.74 41.37 8.15% amrex::Copy() 977 27.04 30.42 31.76 6.26% Castro::normalize_species() 103 25.42 28.63 29.92 5.90% Castro::do_advance_ctu() 9 0.4339 9.29 27.01 5.32% MultiFab::AddProduct() 5021 22.61 25.54 26.63 5.25% Castro::reset_internal_energy(Fab) 1708 17.13 19.44 20.54 4.05% Gravity::fill_multipole_BCs() 15 11.14 13.68 19.35 3.81% Castro::estTimeStep() 26 8.374 11.72 19.15 3.77% MultiFab::Subtract() 227 15.12 17.08 17.83 3.51% FabArray::setVal() 6319 14.68 16.58 17.28 3.41% FabArray::ParallelCopy_finish() 934 3.012 7.313 17.09 3.37% Castro::reset_internal_energy(MultiFab) 104 8.237 8.997 9.993 1.97% FabArray::sum() 5021 8.149 9.163 9.52 1.88% MLPoisson::Fsmooth() 2800 6.998 7.84 8.236 1.62% FluxRegister::Reflux() 24 4.966 5.734 6.036 1.19% Castro::reflux() 4 4.307 4.589 4.829 0.95% FabArray::ParallelCopy_nowait() 934 4.101 4.317 4.63 0.91% FillBoundary_finish() 3098 2.142 2.861 4.447 0.88% MLPoisson::Fapply() 607 3.664 4.093 4.285 0.84% Castro::enforce_min_density() 103 2.529 2.999 3.989 0.79% Castro::construct_new_gravity_source() 17 2.768 3.11 3.229 0.64% FabArray::Saxpy() 77 2.507 2.857 2.982 0.59% Castro::wd_update() 6 0.8236 1.494 2.854 0.56% Castro::FluxRegCrseInit 9 2.494 2.539 2.559 0.50% FillPatchInterp(MF) 30 1.503 1.863 2.339 0.46% amrex::Add() 174 1.908 2.154 2.294 0.45% Castro::construct_old_gravity_source() 18 1.972 2.174 2.271 0.45% Castro::construct_new_source() 119 1.874 2.109 2.193 0.43% Castro::problem_post_timestep() 10 1.298 1.369 1.521 0.30% FillBoundary_nowait() 3098 1.161 1.283 1.433 0.28% MLCellLinOp::applyBC() 3450 1.037 1.223 1.368 0.27% Amr::InitAmr() 1 1.36 1.36 1.36 0.27% FabArray::Xpay() 487 1.146 1.271 1.353 0.27% Castro::initMFs() 5 1.208 1.234 1.271 0.25% StateData::FillBoundary(geom) 50 0.8291 0.9828 1.087 0.21% Castro::volInBoundary() 42 0.8531 0.9678 1.058 0.21% VisMF::Write(FabArray) 3 0.7426 0.827 0.9151 0.18% FillPatchIterator::Initialize 62 0.8541 0.8921 0.9125 0.18% MultiFab::max() 18 0.04916 0.3596 0.8798 0.17% MultiFab::contains_nan() 18 0.725 0.8294 0.8759 0.17% Castro::fill_hybrid_hydro_source() 43 0.5928 0.6669 0.6939 0.14% MLMG::ResNormInf() 99 0.333 0.443 0.6143 0.12% amrex::average_down 392 0.4902 0.5318 0.5583 0.11% MLMG::addInterpCorrection() 340 0.4276 0.4776 0.507 0.10% Castro::initData() 1 0.4716 0.4871 0.4973 0.10% FabArray::setDomainBndry() 92 0.3308 0.3903 0.4327 0.09% Castro::initialize_do_advance() 9 0.3042 0.3558 0.4321 0.09% Castro::construct_old_source() 63 0.3622 0.3991 0.423 0.08% MLCellLinOp::compGrad() 26 0.3335 0.378 0.402 0.08% Amr::writePlotFile() 2 0.1282 0.2091 0.3582 0.07% FabArray::mult() 80 0.257 0.2922 0.3464 0.07% MLCellLinOp::defineAuxData() 23 0.1686 0.1954 0.2177 0.04% Gravity::get_new_grav_vector() 10 0.1672 0.1898 0.1997 0.04% Castro::evaluate_source_change() 233 0.1577 0.1717 0.1789 0.04% Gravity::get_old_grav_vector() 9 0.1533 0.1667 0.1735 0.03% StateDataPhysBCFunct::() 95 0.02076 0.09556 0.1633 0.03% Gravity::test_level_grad_phi_curr() 8 0.1327 0.1542 0.1621 0.03% MLLinOp::makeSubCommunicator() 23 0.00363 0.06948 0.1612 0.03% Castro::swap_state_time_levels() 9 0.1358 0.1426 0.1558 0.03% MLPoisson::FFlux() 81 0.003867 0.0412 0.1513 0.03% MLMG::prepareForSolve() 22 0.1216 0.1349 0.1474 0.03% StateData::define() 25 0.1238 0.1369 0.1453 0.03% MLMG::solve() 22 0.01602 0.07298 0.1402 0.03% Castro::initialize_advance() 9 0.09182 0.1087 0.1373 0.03% Castro::apply_tagging_restrictions() 6 0.1155 0.1213 0.1249 0.02% FabArray::setVal(val, thecmd, scomp, ncomp) 998 0.07394 0.08591 0.109 0.02% AmrMesh::MakeNewGrids() 6 0.08681 0.09627 0.104 0.02% BndryData::define() 31 0.08988 0.09499 0.09848 0.02% Castro::do_new_sources() 17 0.000988 0.02537 0.09656 0.02% Gravity::test_level_grad_phi_prev() 8 0.0748 0.08389 0.08747 0.02% MLCellLinOp::updateSolBC() 14 0.0578 0.06571 0.07867 0.02% Castro::expand_state() 13 0.05528 0.06591 0.07748 0.02% MLMG::MLRhsNormInf() 22 0.05423 0.06281 0.07036 0.01% MLCellLinOp::updateCorBC() 10 0.04137 0.04653 0.0571 0.01% Castro::finalize_do_advance() 9 0.01003 0.02271 0.05357 0.01% MLCellLinOp::setLevelBC() 27 0.04081 0.04661 0.05112 0.01% Castro::FluxRegFineAdd() 9 0.0352 0.03952 0.04458 0.01% Castro::linear_to_hybrid_momentum() 1 0.03164 0.03689 0.04095 0.01% MLMG::MLResNormInf() 25 0.0001923 0.02587 0.03856 0.01% MLCellLinOp::reflux() 17 0.003859 0.01163 0.03369 0.01% Castro::do_old_sources() 9 0.0004801 0.008621 0.03109 0.01% Castro::build_fine_mask() 44 0.02491 0.02796 0.03054 0.01% MLCellLinOp::prepareForSolve() 23 0.028 0.02861 0.02999 0.01% DistributionMapping::LeastUsedCPUs() 5 0.01001 0.02157 0.02959 0.01% Castro::construct_old_gravity() 9 0.01808 0.02447 0.02881 0.01% FabArrayBase::CPC::define() 670 0.01828 0.02234 0.02786 0.01% MLCellLinOp::defineBC() 23 0.02218 0.02551 0.02781 0.01% MLMG::interpCorrection_1 10 0.02337 0.024 0.02501 0.00% Castro::enforce_consistent_e() 1 0.02104 0.02344 0.0245 0.00% Gravity::actual_solve_with_mlmg() 23 0.01992 0.02127 0.02386 0.00% AMRErrorTag::operator() 6 0.01993 0.02246 0.02357 0.00% Gravity::multilevel_solve_for_new_phi() 5 0.0167 0.02094 0.02337 0.00% MultiFab::Multiply() 4 0.01973 0.02167 0.02273 0.00% Castro::Castro() 5 0.01649 0.01897 0.01987 0.00% Amr::coarseTimeStep() 5 0.0008446 0.01408 0.01905 0.00% Castro::buildMetrics() 5 0.01612 0.01796 0.01901 0.00% FabArrayBase::getCPC() 1920 0.01566 0.01671 0.01838 0.00% Gravity::solve_for_phi() 17 0.00989 0.01319 0.01794 0.00% FabArray::LinComb() 334 0 0.004077 0.01631 0.00% MLCGSolver::bicgstab 17 0 0.003968 0.01587 0.00% FabArray::FillBoundary() 3098 0.01125 0.01281 0.01559 0.00% FillPatchTwoLevels 31 0.01314 0.0141 0.01491 0.00% MLCellLinOp::smooth() 1400 0.01112 0.01206 0.0136 0.00% Gravity::create_comp_minus_level_grad_phi() 4 0.01078 0.01192 0.01284 0.00% Castro::apply_problem_tags() 6 0.01194 0.01238 0.01268 0.00% Gravity::update_max_rhs() 10 0.01023 0.0111 0.01192 0.00% FabArray::norminf() 138 0.003892 0.007559 0.01132 0.00% FabArrayBase::getFB() 3098 0.008098 0.009131 0.01126 0.00% amrex::Dot() 239 0 0.002614 0.01046 0.00% Castro::derive() 6 0.005581 0.006896 0.009283 0.00% Castro::finalize_advance() 9 0.000763 0.002729 0.008415 0.00% Gravity::GetCrsePhi() 8 0.006575 0.007283 0.008144 0.00% FabArrayBase::FB::FB() 44 0.004663 0.005948 0.007491 0.00% Gravity::actual_multilevel_solve() 5 0.005829 0.006096 0.006325 0.00% average_down_faces 24 0.004898 0.005313 0.005531 0.00% MLCGSolver::ParallelAllReduce 321 0 0.00128 0.00512 0.00% FabArray::ParallelCopy() 934 0.004216 0.004451 0.004839 0.00% MLCellLinOp::apply() 607 0.002145 0.00286 0.004684 0.00% Gravity::test_composite_phi() 1 0.00237 0.003139 0.004546 0.00% MLLinOp::defineGrids() 23 0.003647 0.0038 0.004019 0.00% CGSolver::sxay() 334 0 0.0009759 0.003903 0.00% MLCellLinOp::correctionResidual() 377 0.001746 0.002037 0.002575 0.00% Castro::clean_state() 103 0.002206 0.002276 0.002353 0.00% TagBoxArray::collate() 6 0.00106 0.001358 0.002052 0.00% Amr::regrid() 5 0.001418 0.001668 0.00205 0.00% MLMG::mgVcycle() 90 0.0015 0.001642 0.00183 0.00% main() 1 0.001744 0.001795 0.001819 0.00% MLMG:computeResOfCorrection() 340 0.001438 0.001539 0.001743 0.00% MLMG::mgVcycle_down::0 70 0.001281 0.001378 0.001519 0.00% OwnerMask() 6 0.0009999 0.001294 0.00144 0.00% AmrLevel::FillCoarsePatch() 5 0.0009633 0.001204 0.001362 0.00% MLMG::mgVcycle_down::1 70 0.001006 0.001119 0.001255 0.00% Castro::construct_new_gravity() 9 0.0005703 0.0008265 0.001241 0.00% MLMG::oneIter() 70 0.000931 0.0009947 0.00108 0.00% MLCellLinOp::solutionResidual() 110 0.0008177 0.0008965 0.0009845 0.00% MLMG::mgVcycle_down::2 70 0.0007544 0.0008366 0.0009803 0.00% Castro::subcycle_advance_ctu() 9 0.000137 0.0003789 0.0009693 0.00% MLMG::mgVcycle_up::0 70 0.0007896 0.0008551 0.0009448 0.00% MLMG::actualBottomSolve() 70 0.0001074 0.0003171 0.000924 0.00% Gravity::solve_phi_with_mlmg() 23 0.0007162 0.0008131 0.0009111 0.00% MLMG::mgVcycle_down::3 70 0.0006927 0.0007479 0.0008338 0.00% FillPatchSingleLevel 92 0.0007098 0.0007141 0.0007199 0.00% TagBoxArray::mapPRD 6 0.0005832 0.0006616 0.0006976 0.00% MLMG::mgVcycle_down::4 60 0.0004975 0.0005674 0.0006893 0.00% Gravity::average_fine_ec_onto_crse_ec() 4 0.0005337 0.0006112 0.0006854 0.00% Amr::defBaseLevel() 1 0.0001559 0.0002876 0.0006409 0.00% MultiFab::Add() 100 0.0005249 0.0005684 0.000637 0.00% MLMG::mgVcycle_up::1 70 0.0005329 0.0005665 0.0006232 0.00% Amr::grid_places() 6 0.0005158 0.0005359 0.0005603 0.00% MLMG::mgVcycle_up::2 70 0.0004397 0.0004741 0.000533 0.00% makeSFC 69 0.0004702 0.0004862 0.0005126 0.00% MLMG::mgVcycle_up::3 70 0.0004072 0.0004452 0.0005065 0.00% Castro::computeNewDt() 8 0.0001365 0.0002246 0.0004612 0.00% MLMG::computeResidual() 70 0.0003477 0.0003851 0.0004344 0.00% Castro::enforce_speed_limit() 103 0.0003763 0.0004152 0.0004314 0.00% MLMG::mgVcycle_bottom 90 0.0003404 0.0003681 0.0004223 0.00% MLMG::mgVcycle_up::4 60 0.0003194 0.0003527 0.0004204 0.00% Castro::init(old) 3 0.0003685 0.0003907 0.000419 0.00% Castro::post_regrid() 10 0.0002658 0.0003092 0.0003407 0.00% DistributionMapping::Distribute() 71 0.0002728 0.0002933 0.000339 0.00% MLLinOp::define() 23 0.0002883 0.0003045 0.0003341 0.00% Castro::apply_source_to_state() 34 0.0002994 0.0003132 0.0003317 0.00% Amr::timeStep() 9 0.0002769 0.0003064 0.0003282 0.00% MLLinOp::makeAgglomeratedDMap 23 0.0002935 0.0003058 0.0003167 0.00% Castro::check_for_nan() 18 0.0001951 0.0002257 0.0002857 0.00% MLPoisson::define() 23 0.0002475 0.0002646 0.0002815 0.00% AmrLevel::AmrLevel(dm) 5 0.000235 0.0002524 0.000268 0.00% MultiFab::Copy() 46 0.0002201 0.0002365 0.0002663 0.00% MLMG::computeResWithCrseSolFineCor() 10 0.0001784 0.0002106 0.0002495 0.00% Castro::react_state() 18 0.000214 0.000229 0.0002445 0.00% MLMG::computeMLResidual() 25 0.0001662 0.000188 0.0002189 0.00% MLMG::getGradSolution() 22 0.0001793 0.0001864 0.0001914 0.00% FabArrayBase::TheFPinfo() 25 0.0001709 0.0001841 0.0001912 0.00% FPinfo::FPinfo() 3 0.0001389 0.0001564 0.0001706 0.00% Castro::avgDown(state_indx) 20 9.136e-05 0.000118 0.0001674 0.00% Castro::errorEst() 6 0.0001539 0.000161 0.0001668 0.00% Castro::construct_new_hybrid_source() 17 0.000143 0.0001495 0.0001619 0.00% Castro::advance() 9 0.0001302 0.0001438 0.0001542 0.00% MLPoisson::prepareForSolve() 23 0.0001375 0.0001445 0.0001486 0.00% Gravity::swapTimeLevels() 9 0.0001241 0.0001369 0.0001453 0.00% Castro::post_timestep() 9 0.0001185 0.0001292 0.0001373 0.00% ClusterList::intersect() 1 0 2.954e-05 0.0001182 0.00% Amr::writeSmallPlotFile() 1 0.0001049 0.0001076 0.0001096 0.00% MLMG::computeResWithCrseCorFineCor() 10 7.326e-05 8.421e-05 0.0001007 0.00% Amr::FinalizeInit() 1 3.793e-05 5.458e-05 9.484e-05 0.00% Castro::create_source_corrector() 9 6.481e-05 7.642e-05 8.972e-05 0.00% Gravity::set_mass_offset() 6 6.546e-05 7.545e-05 8.916e-05 0.00% amrex::average_down_w_geom 20 6.867e-05 7.59e-05 8.676e-05 0.00% knapsack() 3 7.008e-05 7.717e-05 8.498e-05 0.00% FabArray::RecvLayoutMask() 4 6.669e-05 7.091e-05 7.74e-05 0.00% MLMG::miniCycle() 20 5.295e-05 6.14e-05 7.239e-05 0.00% Castro::avgDown() 4 4.728e-05 5.523e-05 6.75e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 5.655e-05 6.022e-05 6.455e-05 0.00% Castro::retry_advance_ctu() 9 3.373e-05 4.368e-05 6.297e-05 0.00% Castro::construct_new_rotation_source() 17 5.286e-05 5.39e-05 5.49e-05 0.00% Amr::initSubcycle() 1 5.093e-05 5.152e-05 5.207e-05 0.00% AmrMesh-cluster 1 0 1.272e-05 5.087e-05 0.00% Castro::post_init() 1 3.313e-05 4.021e-05 5.075e-05 0.00% MLMG::compResidual() 1 3.837e-05 4.267e-05 4.78e-05 0.00% DistributionMapping::KnapSackDoIt() 3 3.711e-05 4.052e-05 4.645e-05 0.00% Castro::construct_old_hybrid_source() 9 4.039e-05 4.438e-05 4.605e-05 0.00% BoxList::parallelComplementIn() 12 3.06e-05 3.813e-05 4.421e-05 0.00% Castro::computeInitialDt() 2 1.891e-05 2.708e-05 3.393e-05 0.00% Castro::construct_old_rotation_source() 9 2.907e-05 3.087e-05 3.256e-05 0.00% BoxList::complementIn 12 2.787e-05 2.946e-05 3.177e-05 0.00% Amr::bldFineLevels() 1 2.512e-05 2.646e-05 2.732e-05 0.00% Castro::init() 1 1.506e-05 1.915e-05 2.235e-05 0.00% Amr::init() 1 7.178e-06 9.877e-06 1.566e-05 0.00% ClusterList::chop() 1 0 2.886e-06 1.154e-05 0.00% Amr::InitializeInit() 1 8.596e-06 9.5e-06 1.031e-05 0.00% Amr::initialInit() 1 6.012e-06 7.008e-06 8.447e-06 0.00% knapsack()swap 1 4.272e-06 5.087e-06 5.735e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 507.4 507.4 507.4 100.00% Amr::coarseTimeStep() 5 486.2 486.2 486.2 95.82% Amr::timeStep() 9 483.5 483.5 483.5 95.29% Castro::advance() 9 383 383.1 383.1 75.49% Castro::subcycle_advance_ctu() 9 363.4 364.2 365.8 72.09% Castro::do_advance_ctu() 9 363.4 364.2 365.8 72.09% Castro::construct_ctu_hydro_source() 9 181.1 198.7 207.4 40.88% Castro::clean_state() 103 161.8 181.3 190.4 37.52% Castro::computeTemp() 104 102.3 114.2 119.7 23.58% Castro::reset_internal_energy(MultiFab) 104 66.83 74.49 78.46 15.46% Castro::post_timestep() 9 65.02 65.02 65.03 12.82% Castro::reflux() 4 40.64 44.44 46.01 9.07% Castro::evaluate_source_change() 233 38.87 43.82 45.6 8.99% Castro::enforce_min_density() 103 35.49 40.08 42.53 8.38% Gravity::solve_phi_with_mlmg() 23 34.16 36.76 42.51 8.38% Castro::do_new_sources() 17 31.96 36.27 38.2 7.53% Castro::post_regrid() 10 32.87 35.74 37.13 7.32% Amr::regrid() 5 32.18 34.83 36.06 7.11% amrex::Copy() 977 27.04 30.42 31.76 6.26% Gravity::solve_for_phi() 17 21.89 24.49 30.22 5.96% Castro::normalize_species() 103 25.42 28.63 29.92 5.90% Castro::initialize_do_advance() 9 27.22 28.56 29.32 5.78% MultiFab::AddProduct() 5021 22.61 25.54 26.63 5.25% Gravity::actual_solve_with_mlmg() 23 22.73 22.76 22.82 4.50% Castro::construct_old_gravity() 9 14.22 16.74 22.05 4.35% MLMG::solve() 22 21.62 21.67 21.72 4.28% FabArray::ParallelCopy() 934 7.753 11.74 21.3 4.20% Castro::reset_internal_energy(Fab) 1708 17.13 19.44 20.54 4.05% Castro::do_old_sources() 9 16.7 18.64 19.61 3.86% Gravity::fill_multipole_BCs() 15 11.27 13.8 19.46 3.83% Castro::estTimeStep() 26 8.374 11.72 19.15 3.77% Amr::init() 1 19.02 19.02 19.02 3.75% Amr::initialInit() 1 18.25 18.25 18.25 3.60% MultiFab::Subtract() 227 15.12 17.08 17.83 3.51% MLMG::oneIter() 70 17.19 17.31 17.37 3.42% FillPatchIterator::Initialize 62 10.88 12.89 17.3 3.41% FabArray::setVal() 6319 14.68 16.58 17.28 3.41% FabArray::ParallelCopy_finish() 934 3.012 7.313 17.09 3.37% Castro::initialize_advance() 9 14.49 16.06 16.89 3.33% MLMG::mgVcycle() 90 16.1 16.16 16.23 3.20% FillPatchSingleLevel 92 7.53 9.55 14.07 2.77% Castro::construct_new_gravity() 9 11.19 11.92 13.41 2.64% Amr::FinalizeInit() 1 11.14 11.45 12.11 2.39% MLCellLinOp::smooth() 1400 11.65 11.8 11.95 2.36% Gravity::multilevel_solve_for_new_phi() 5 11.61 11.62 11.63 2.29% Gravity::actual_multilevel_solve() 5 11.59 11.6 11.61 2.29% Castro::expand_state() 13 8.76 9.492 10.49 2.07% FabArray::sum() 5021 8.149 9.163 9.52 1.88% MLPoisson::Fsmooth() 2800 6.998 7.84 8.236 1.62% Castro::computeNewDt() 8 4.475 5.564 7.948 1.57% FluxRegister::Reflux() 24 6.417 7.42 7.871 1.55% Castro::construct_new_source() 119 6.66 7.536 7.836 1.54% MLMG::mgVcycle_down::0 70 7.455 7.674 7.78 1.53% Amr::InitializeInit() 1 6.142 6.802 7.11 1.40% Amr::defBaseLevel() 1 6.142 6.802 7.11 1.40% MLCellLinOp::applyBC() 3450 4.702 5.313 6.678 1.32% Castro::problem_post_timestep() 10 4.657 5.239 6.498 1.28% amrex::average_down 392 0.6989 2.239 5.979 1.18% FabArray::FillBoundary() 3098 3.416 4.172 5.767 1.14% Castro::avgDown() 4 0.2263 1.805 5.589 1.10% Castro::avgDown(state_indx) 20 0.2263 1.805 5.589 1.10% amrex::average_down_w_geom 20 0.2261 1.805 5.589 1.10% Castro::post_init() 1 4.782 5.078 5.578 1.10% Castro::initData() 1 4.638 5.195 5.454 1.07% MLCellLinOp::apply() 607 5.209 5.329 5.451 1.07% MLMG::mgVcycle_up::0 70 5.135 5.219 5.266 1.04% FabArray::ParallelCopy_nowait() 934 4.199 4.419 4.736 0.93% FillBoundary_finish() 3098 2.142 2.861 4.447 0.88% MLPoisson::Fapply() 607 3.664 4.093 4.285 0.84% Castro::wd_update() 6 1.882 2.462 3.707 0.73% MLCellLinOp::solutionResidual() 110 3.507 3.594 3.67 0.72% MLCellLinOp::correctionResidual() 377 3.087 3.185 3.332 0.66% Castro::construct_new_gravity_source() 17 2.768 3.11 3.229 0.64% MLMG:computeResOfCorrection() 340 2.849 3 3.148 0.62% Gravity::get_old_grav_vector() 9 1.267 1.987 3.142 0.62% Amr::bldFineLevels() 1 2.573 2.843 2.988 0.59% FabArray::Saxpy() 77 2.507 2.857 2.982 0.59% FillPatchTwoLevels 31 2.774 2.803 2.838 0.56% Castro::finalize_advance() 9 2.764 2.78 2.799 0.55% Castro::FluxRegCrseInit 9 2.622 2.64 2.658 0.52% FillPatchInterp(MF) 30 1.503 1.863 2.339 0.46% MLMG::computeResidual() 70 2.228 2.292 2.334 0.46% amrex::Add() 174 1.908 2.154 2.294 0.45% Castro::construct_old_gravity_source() 18 1.972 2.174 2.271 0.45% Castro::construct_old_source() 63 1.958 2.152 2.254 0.44% Castro::computeInitialDt() 2 1.074 1.381 2.041 0.40% Castro::Castro() 5 1.505 1.593 1.653 0.33% MLMG::mgVcycle_down::1 70 1.22 1.36 1.577 0.31% Amr::writePlotFile() 2 1.571 1.571 1.571 0.31% Gravity::get_new_grav_vector() 10 1.421 1.442 1.46 0.29% FillBoundary_nowait() 3098 1.174 1.298 1.45 0.29% Amr::InitAmr() 1 1.36 1.36 1.36 0.27% FabArray::Xpay() 487 1.146 1.271 1.353 0.27% Castro::apply_source_to_state() 34 1.113 1.266 1.323 0.26% Castro::initMFs() 5 1.217 1.242 1.279 0.25% StateDataPhysBCFunct::() 95 0.8499 1.078 1.161 0.23% StateData::FillBoundary(geom) 50 0.8291 0.9828 1.087 0.21% Castro::volInBoundary() 42 0.8532 0.9679 1.059 0.21% MLMG::computeMLResidual() 25 0.9689 0.9845 1.028 0.20% Gravity::update_max_rhs() 10 0.1512 0.4622 0.9731 0.19% VisMF::Write(FabArray) 3 0.7426 0.827 0.9151 0.18% MLMG::mgVcycle_up::1 70 0.8254 0.8498 0.8841 0.17% MultiFab::max() 18 0.04916 0.3596 0.8798 0.17% Castro::check_for_nan() 18 0.7253 0.8297 0.8761 0.17% MultiFab::contains_nan() 18 0.725 0.8294 0.8759 0.17% Gravity::test_composite_phi() 1 0.8161 0.824 0.8276 0.16% MLMG::addInterpCorrection() 340 0.527 0.6748 0.7589 0.15% Castro::init(old) 3 0.7158 0.7228 0.735 0.14% MLMG::computeResWithCrseSolFineCor() 10 0.5922 0.6262 0.7104 0.14% Castro::fill_hybrid_hydro_source() 43 0.5928 0.6669 0.6939 0.14% MLMG::ResNormInf() 99 0.333 0.443 0.6143 0.12% Castro::init() 1 0.5856 0.5944 0.6007 0.12% AmrLevel::FillCoarsePatch() 5 0.5856 0.5943 0.6007 0.12% MLPoisson::define() 23 0.4578 0.5077 0.5515 0.11% Castro::construct_new_hybrid_source() 17 0.4616 0.5222 0.5442 0.11% MLMG::prepareForSolve() 22 0.4284 0.483 0.5261 0.10% Castro::react_state() 18 0.4073 0.4669 0.4923 0.10% MLMG::getGradSolution() 22 0.4345 0.4648 0.4902 0.10% MLCellLinOp::compGrad() 26 0.4343 0.4646 0.4901 0.10% FabArray::setDomainBndry() 92 0.3308 0.3903 0.4327 0.09% Amr::grid_places() 6 0.3819 0.3822 0.3823 0.08% AmrMesh::MakeNewGrids() 6 0.3813 0.3816 0.3818 0.08% MultiFab::Add() 100 0.3176 0.3575 0.3775 0.07% MLCellLinOp::reflux() 17 0.1968 0.2488 0.3732 0.07% FabArray::mult() 80 0.257 0.2922 0.3464 0.07% MLMG::mgVcycle_down::2 70 0.2634 0.2855 0.3126 0.06% MLMG::mgVcycle_up::3 70 0.09007 0.2289 0.2763 0.05% MLMG::mgVcycle_bottom 90 0.1633 0.1911 0.2734 0.05% MLCellLinOp::defineAuxData() 23 0.2114 0.2418 0.2717 0.05% Castro::errorEst() 6 0.2231 0.2392 0.2495 0.05% Gravity::test_level_grad_phi_curr() 8 0.1898 0.2242 0.2384 0.05% MLMG::mgVcycle_up::2 70 0.1867 0.1996 0.2253 0.04% Gravity::create_comp_minus_level_grad_phi() 4 0.1892 0.208 0.2197 0.04% MLCellLinOp::defineBC() 23 0.1732 0.1909 0.214 0.04% MLCellLinOp::updateSolBC() 14 0.08844 0.1319 0.1885 0.04% BndryData::define() 31 0.1499 0.1641 0.1851 0.04% Gravity::GetCrsePhi() 8 0.1071 0.147 0.1846 0.04% MLLinOp::define() 23 0.008808 0.07466 0.1666 0.03% MLLinOp::defineGrids() 23 0.008511 0.07435 0.1663 0.03% MLMG::miniCycle() 20 0.1631 0.1636 0.164 0.03% MLLinOp::makeSubCommunicator() 23 0.00363 0.06948 0.1612 0.03% Castro::swap_state_time_levels() 9 0.1358 0.1426 0.1558 0.03% Castro::construct_old_hybrid_source() 9 0.1314 0.1448 0.1522 0.03% MLPoisson::FFlux() 81 0.003867 0.0412 0.1513 0.03% AmrLevel::AmrLevel(dm) 5 0.124 0.1372 0.1456 0.03% StateData::define() 25 0.1238 0.1369 0.1453 0.03% Gravity::test_level_grad_phi_prev() 8 0.1088 0.1225 0.1292 0.03% Castro::apply_tagging_restrictions() 6 0.1155 0.1213 0.1249 0.02% MLMG::mgVcycle_down::3 70 0.1032 0.1092 0.1246 0.02% MLMG::MLResNormInf() 25 0.08126 0.1056 0.1201 0.02% MLMG::computeResWithCrseCorFineCor() 10 0.0929 0.102 0.1099 0.02% MLMG::actualBottomSolve() 70 0.0001074 0.02743 0.1094 0.02% FabArray::setVal(val, thecmd, scomp, ncomp) 998 0.07394 0.08591 0.109 0.02% MLCGSolver::bicgstab 17 0 0.02682 0.1073 0.02% MultiFab::Copy() 46 0.08014 0.09071 0.09623 0.02% Castro::derive() 6 0.07475 0.08294 0.0886 0.02% MLMG::MLRhsNormInf() 22 0.05423 0.06281 0.07036 0.01% Gravity::swapTimeLevels() 9 0.05599 0.06286 0.06641 0.01% MLMG::interpCorrection_1 10 0.03284 0.04622 0.06496 0.01% MLCellLinOp::updateCorBC() 10 0.04787 0.05215 0.06344 0.01% MLCellLinOp::setLevelBC() 27 0.05278 0.05596 0.06133 0.01% MLMG::mgVcycle_down::4 60 0.01313 0.02516 0.06047 0.01% Castro::finalize_do_advance() 9 0.01003 0.02271 0.05357 0.01% Castro::buildMetrics() 5 0.04828 0.05067 0.05223 0.01% TagBoxArray::mapPRD 6 0.01302 0.02455 0.0516 0.01% MLMG::compResidual() 1 0.03931 0.04356 0.04706 0.01% FabArrayBase::getCPC() 1920 0.03393 0.03905 0.04623 0.01% Castro::FluxRegFineAdd() 9 0.0352 0.03952 0.04458 0.01% MLMG::mgVcycle_up::4 60 0.01082 0.01936 0.0443 0.01% Castro::linear_to_hybrid_momentum() 1 0.03164 0.03689 0.04095 0.01% Castro::build_fine_mask() 44 0.02491 0.02796 0.03054 0.01% MLPoisson::prepareForSolve() 23 0.02815 0.02876 0.03014 0.01% MLCellLinOp::prepareForSolve() 23 0.028 0.02861 0.02999 0.01% DistributionMapping::LeastUsedCPUs() 5 0.01001 0.02157 0.02959 0.01% FabArrayBase::TheFPinfo() 25 0.01044 0.02162 0.02955 0.01% FPinfo::FPinfo() 3 0.01025 0.02144 0.02937 0.01% DistributionMapping::KnapSackDoIt() 3 0.01008 0.02128 0.0292 0.01% FabArrayBase::CPC::define() 670 0.01828 0.02234 0.02786 0.01% Castro::enforce_consistent_e() 1 0.02104 0.02344 0.0245 0.00% AMRErrorTag::operator() 6 0.01993 0.02246 0.02357 0.00% MultiFab::Multiply() 4 0.01973 0.02167 0.02273 0.00% Gravity::average_fine_ec_onto_crse_ec() 4 0.01435 0.01642 0.02119 0.00% CGSolver::sxay() 334 0 0.005052 0.02021 0.00% FabArrayBase::getFB() 3098 0.01276 0.01508 0.01725 0.00% FabArray::LinComb() 334 0 0.004077 0.01631 0.00% Castro::apply_problem_tags() 6 0.01194 0.01238 0.01268 0.00% average_down_faces 24 0.008319 0.01028 0.01214 0.00% FabArray::norminf() 138 0.003892 0.007559 0.01132 0.00% MLCGSolver::ParallelAllReduce 321 0 0.002691 0.01076 0.00% amrex::Dot() 239 0 0.002614 0.01046 0.00% FabArrayBase::FB::FB() 44 0.004663 0.005948 0.007491 0.00% TagBoxArray::collate() 6 0.00106 0.001358 0.002052 0.00% OwnerMask() 6 0.0009999 0.001294 0.00144 0.00% MLLinOp::makeAgglomeratedDMap 23 0.001023 0.001071 0.001123 0.00% makeSFC 69 0.000729 0.0007649 0.0008067 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 0.0001072 0.0004792 0.0006079 0.00% Castro::enforce_speed_limit() 103 0.0003763 0.0004152 0.0004314 0.00% DistributionMapping::Distribute() 71 0.0002728 0.0002933 0.000339 0.00% AmrMesh-cluster 1 0 4.515e-05 0.0001806 0.00% ClusterList::intersect() 1 0 2.954e-05 0.0001182 0.00% Amr::writeSmallPlotFile() 1 0.0001049 0.0001076 0.0001096 0.00% knapsack() 3 7.435e-05 8.225e-05 9.072e-05 0.00% Castro::create_source_corrector() 9 6.481e-05 7.642e-05 8.972e-05 0.00% Gravity::set_mass_offset() 6 6.546e-05 7.545e-05 8.916e-05 0.00% FabArray::RecvLayoutMask() 4 6.669e-05 7.091e-05 7.74e-05 0.00% BoxList::parallelComplementIn() 12 5.86e-05 6.759e-05 7.439e-05 0.00% Castro::retry_advance_ctu() 9 3.373e-05 4.368e-05 6.297e-05 0.00% Castro::construct_new_rotation_source() 17 5.286e-05 5.39e-05 5.49e-05 0.00% Amr::initSubcycle() 1 5.093e-05 5.152e-05 5.207e-05 0.00% Castro::construct_old_rotation_source() 9 2.907e-05 3.087e-05 3.256e-05 0.00% BoxList::complementIn 12 2.787e-05 2.946e-05 3.177e-05 0.00% ClusterList::chop() 1 0 2.886e-06 1.154e-05 0.00% knapsack()swap 1 4.272e-06 5.087e-06 5.735e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::castro.center(nvals = 3) :: [0.0e0, 0.0e0, 0.0e0] [TOP]::mg.v(nvals = 1) :: [2] AMReX (22.11-34-g88fe04f00600) finalized