MPI initialized with 16 MPI processes MPI initialized with thread support level 3 AMReX (23.06-29-g0236a3732cc8) initialized Starting run at 07:14:32 UTC on 2023-06-28. Successfully read inputs file ... Castro git describe: 23.06-37-g46f8579c6 AMReX git describe: 23.06-29-g0236a3732 Microphysics git describe: 23.06-33-g716dc7f9 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Generated initial model for primary WD of mass 0.64 solar masses, central density 4.500e+06 g cm**-3, and radius 8.734e+08 cm. Generated initial model for secondary WD of mass 0.64 solar masses, central density 4.500e+06 g cm**-3, and radius 8.734e+08 cm. Castro::numpts_1d at level 0 is 294 Setting the gravity type to PoissonGrav Warning: the following castro.* parameters are ignored castro.dtnuc_mode = 1 Initializing the data at level 0 Done initializing the level 0 data grid_places() time: 0.132597633 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.000982636 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.010418461 Gravity::multilevel_solve_for_new_phi() time = 0.097124371 Gravity::fill_multipole_BCs() time = 0.010301246 ... norm of composite residual at level 0 1.193955541e-11 TIME= 0 MASS = 2.549996568e+33 TIME= 0 XMOM = 0 TIME= 0 YMOM = 0 TIME= 0 ZMOM = 0 TIME= 0 ANG MOM X = 0 TIME= 0 ANG MOM Y = 0 TIME= 0 ANG MOM Z = -3.115378115e+34 TIME= 0 RHO*e = 1.325915365e+50 TIME= 0 RHO*K = 3.09960185e+49 TIME= 0 RHO*E = 1.63587555e+50 TIME= 0 RHO*PHI = -5.434813569e+50 TIME= 0 TOTAL ENERGY= -1.081531234e+50 TIME= 0 CENTER OF MASS X-LOC = 316439254.4 TIME= 0 CENTER OF MASS X-VEL = 0 TIME= 0 CENTER OF MASS Y-LOC = -4.854673355e-07 TIME= 0 CENTER OF MASS Y-VEL = 0 TIME= 0 CENTER OF MASS Z-LOC = 0 TIME= 0 CENTER OF MASS Z-VEL = 0 TIME= 0 MAXIMUM TEMPERATURE = 10000000 TIME= 0 MAXIMUM DENSITY = 4476474 TIME= 0 MAXIMUM T_S / T_E = 0 INITIAL GRIDS Level 0 32 grids 32768 cells 100 % of domain smallest grid: 32 x 32 biggest grid: 32 x 32 PLOTFILE: file = wdmerger_retry-SDC_plt00000 Write plotfile time = 0.050687818 seconds SMALL PLOTFILE: file = smallplt00000 Write small plotfile time = 0.007881568 seconds [Level 0 step 1] ADVANCE with dt = 0.25  Beginning subcycle 1 starting at time 0 with dt = 0.25 Estimated number of subcycles remaining: 1 Beginning SDC iteration 1 of 2. Castro::construct_old_gravity() time = 0.003149986 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -5.111789233e+47 ener added: 3.959219145e+48 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.174137392e+41 ymom added: 6.498822529e+28 zmom added: 0 eint added: 0 ener added: 6.931398274e+47 Castro::do_old_sources() time = 0.063864117 ... Entering construct_ctu_hydro_source() Invalid density = -9.947567811e-05 at index 0, 62, 0 Invalid density = -1.942940124 at index 0, 63, 0 Invalid density = -1.164962232 at index 1, 63, 0 Invalid density = -0.767071355 at index 2, 63, 0 Invalid density = -0.08878802337 at index 3, 63, 0 Invalid density = -0.01174181879 at index 4, 63, 0 Invalid density = -0.0008017363671 at index 5, 63, 0 Invalid density = -1.942940124 at index 0, 192, 0 Invalid density = -1.164962232 at index 1, 192, 0 Invalid density = -0.767071355 at index 2, 192, 0 Invalid density = -0.08878802337 at index 3, 192, 0 Invalid density = -0.01174181879 at index 4, 192, 0 Invalid density = -0.0008017363671 at index 5, 192, 0 Invalid density = -9.947567811e-05 at index 0, 193, 0 Invalid density = -618.7284155 at index 0, 64, 0 Invalid density = -540.6137805 at index 1, 64, 0 Invalid density = -443.905673 at index 2, 64, 0 Invalid density = -302.5298758 at index 3, 64, 0 Invalid density = -167.9940382 at index 4, 64, 0 Invalid density = -55.18215035 at index 5, 64, 0 Invalid density = -1.089487306 at index 6, 64, 0 Invalid density = -625.0368072 at index 5, 65, 0 Invalid density = -691.1269281 at index 6, 65, 0 Invalid density = -318.5467376 at index 7, 65, 0 Invalid density = -11.08517908 at index 8, 65, 0 Invalid density = -820.8014889 at index 8, 66, 0 Invalid density = -456.4708539 at index 9, 66, 0 Invalid density = -6.239071149 at index 10, 66, 0 Invalid density = -907.0826581 at index 10, 67, 0 Invalid density = -241.113047 at index 11, 67, 0 Invalid density = -606.7538937 at index 12, 68, 0 Invalid density = -859.0333271 at index 13, 69, 0 Invalid density = -908.1731819 at index 14, 70, 0 Invalid density = -764.1508347 at index 15, 71, 0 Invalid density = -415.2911996 at index 16, 72, 0 Invalid density = -552.7694089 at index 17, 74, 0 Invalid density = -160.2179808 at index 18, 75, 0 Invalid density = -80.16477042 at index 18, 76, 0 Invalid density = -61.58489855 at index 19, 77, 0 Invalid density = -108.2035501 at index 19, 78, 0 Invalid density = -15.15828486 at index 20, 80, 0 Invalid density = -8.076810125 at index 20, 81, 0 Invalid density = -8.076810125 at index 20, 174, 0 Invalid density = -15.15828486 at index 20, 175, 0 Invalid density = -108.2035501 at index 19, 177, 0 Invalid density = -61.58489855 at index 19, 178, 0 Invalid density = -80.16477042 at index 18, 179, 0 Invalid density = -160.2179808 at index 18, 180, 0 Invalid density = -552.7694089 at index 17, 181, 0 Invalid density = -415.2911996 at index 16, 183, 0 Invalid density = -764.1508347 at index 15, 184, 0 Invalid density = -908.1731819 at index 14, 185, 0 Invalid density = -859.0333271 at index 13, 186, 0 Invalid density = -606.7538937 at index 12, 187, 0 Invalid density = -907.0826581 at index 10, 188, 0 Invalid density = -241.113047 at index 11, 188, 0 Invalid density = -820.8014889 at index 8, 189, 0 Invalid density = -456.4708539 at index 9, 189, 0 Invalid density = -6.239071149 at index 10, 189, 0 Invalid density = -625.0368072 at index 5, 190, 0 Invalid density = -691.1269281 at index 6, 190, 0 Invalid density = -318.5467376 at index 7, 190, 0 Invalid density = -11.08517908 at index 8, 190, 0 Invalid density = -618.7284155 at index 0, 191, 0 Invalid density = -540.6137805 at index 1, 191, 0 Invalid density = -443.905673 at index 2, 191, 0 Invalid density = -302.5298758 at index 3, 191, 0 Invalid density = -167.9940382 at index 4, 191, 0 Invalid density = -55.18215035 at index 5, 191, 0 Invalid density = -1.089487306 at index 6, 191, 0 Advance was unsuccessful with reason: invalid density; proceeding to a retry.  Subcycle completed  Timestep 0.25 rejected at level 0. Performing a retry, with subcycled timesteps of maximum length dt = 0.125   Beginning subcycle 1 starting at time 0 with dt = 0.125 Estimated number of subcycles remaining: 2 Adding an SDC iteration due to the retry. Beginning SDC iteration 1 of 3. Castro::construct_old_gravity() time = 0.006818875 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -5.024842118e+47 ener added: 6.151153052e+47 Contributions to the state from old-time sources: mass added: 0 xmom added: -5.870686959e+40 ymom added: 3.249411264e+28 zmom added: 0 eint added: 0 ener added: 3.465699137e+47 Castro::do_old_sources() time = 0.064469253 ... Entering construct_ctu_hydro_source() Invalid density = -1.592544331 at index 20, 79, 0 Invalid density = -0.02612720466 at index 21, 82, 0 Invalid density = -0.02612720466 at index 21, 173, 0 Invalid density = -1.592544331 at index 20, 176, 0 Advance was unsuccessful with reason: invalid density; proceeding to a retry.  Subcycle completed  Timestep 0.125 rejected at level 0. Performing a retry, with subcycled timesteps of maximum length dt = 0.0625   Beginning subcycle 1 starting at time 0 with dt = 0.0625 Estimated number of subcycles remaining: 4 Adding an SDC iteration due to the retry. Beginning SDC iteration 1 of 3. Castro::construct_old_gravity() time = 0.004921501 Contributions to the state from old-time sources: mass added: 0 xmom added: -2.93534348e+40 ymom added: 1.624705632e+28 zmom added: 0 eint added: 0 ener added: 1.732849568e+47 Castro::do_old_sources() time = 0.060852291 ... Entering construct_ctu_hydro_source() Invalid density = -0.1060744788 at index 20, 78, 0 Invalid density = -0.1060744788 at index 20, 177, 0 Advance was unsuccessful with reason: invalid density; proceeding to a retry.  Subcycle completed  Timestep 0.0625 rejected at level 0. Performing a retry, with subcycled timesteps of maximum length dt = 0.03125   Beginning subcycle 1 starting at time 0 with dt = 0.03125 Estimated number of subcycles remaining: 8 Adding an SDC iteration due to the retry. Beginning SDC iteration 1 of 3. Castro::construct_old_gravity() time = 0.004118244 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.46767174e+40 ymom added: 8.123528161e+27 zmom added: 0 eint added: 0 ener added: 8.664247842e+46 Castro::do_old_sources() time = 0.060679525 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.324085629 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015420279 Gravity::solve_for_phi() time = 0.086050481 Castro::construct_new_gravity() time = 0.088077096 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.674192048e+44 ener added: 3.847280946e+42 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.752957884e+35 ymom added: -3.586520465e+27 zmom added: 0 eint added: 0 ener added: 6.209484931e+43 Castro::do_new_sources() time = 0.060278468 Ending SDC iteration 1 of 3. Beginning SDC iteration 2 of 3. Castro::construct_old_gravity() time = 0.002732105 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.46767174e+40 ymom added: 8.123528161e+27 zmom added: 0 eint added: 0 ener added: 8.664247842e+46 Castro::do_old_sources() time = 0.058598423 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.322290643 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015493525 Gravity::solve_for_phi() time = 0.087908255 Castro::construct_new_gravity() time = 0.089965264 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.838148647e+44 ener added: 3.819761762e+42 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.742583287e+35 ymom added: -3.588335034e+27 zmom added: 0 eint added: 0 ener added: 4.570264853e+43 Castro::do_new_sources() time = 0.060591218 Ending SDC iteration 2 of 3. Beginning SDC iteration 3 of 3. Castro::construct_old_gravity() time = 0.003452283 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.46767174e+40 ymom added: 8.123528161e+27 zmom added: 0 eint added: 0 ener added: 8.664247842e+46 Castro::do_old_sources() time = 0.058839403 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.32807111 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015498024 Gravity::solve_for_phi() time = 0.086301674 Castro::construct_new_gravity() time = 0.0883282 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.838245244e+44 ener added: 3.819942812e+42 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.742573524e+35 ymom added: -3.587065898e+27 zmom added: 0 eint added: 0 ener added: 4.569279535e+43 Castro::do_new_sources() time = 0.060255512 Ending SDC iteration 3 of 3.  Subcycle completed  Beginning subcycle 2 starting at time 0.03125 with dt = 0.03125 Estimated number of subcycles remaining: 7 Beginning SDC iteration 1 of 2. Castro::construct_old_gravity() time = 0.00279748 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.985021785e+46 ener added: 5.435221686e+40 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467706591e+40 ymom added: 9.489312105e+26 zmom added: 0 eint added: 0 ener added: 8.721813579e+46 Castro::do_old_sources() time = 0.058657254 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.985598081e+46 ener added: 2.745852676e+42 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.330007481 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015832696 Gravity::solve_for_phi() time = 0.086525778 Castro::construct_new_gravity() time = 0.088607895 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.274072206e+44 ener added: 4.891864863e+43 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.763054894e+35 ymom added: -9.059860097e+24 zmom added: 0 eint added: 0 ener added: 5.968608016e+43 Castro::do_new_sources() time = 0.059788188 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Castro::construct_old_gravity() time = 0.002790658 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.985021785e+46 ener added: 5.435221686e+40 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467706591e+40 ymom added: 9.489312105e+26 zmom added: 0 eint added: 0 ener added: 8.721813579e+46 Castro::do_old_sources() time = 0.059961827 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.986297782e+46 ener added: 2.752525031e+42 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.337769935 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015365754 Gravity::solve_for_phi() time = 0.087881682 Castro::construct_new_gravity() time = 0.089979469 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.347573489e+44 ener added: 4.888153391e+43 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.756949739e+35 ymom added: -3.751920171e+24 zmom added: 0 eint added: 0 ener added: 5.2374099e+43 Castro::do_new_sources() time = 0.059608161 Ending SDC iteration 2 of 2.  Subcycle completed  Beginning subcycle 3 starting at time 0.0625 with dt = 0.03125 Estimated number of subcycles remaining: 6 Beginning SDC iteration 1 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.881810984e+29 ener added: 1.881810984e+29 Castro::construct_old_gravity() time = 0.003337821 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.984353173e+46 ener added: 6.990361455e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.46774173e+40 ymom added: 9.41450982e+26 zmom added: 0 eint added: 0 ener added: 8.780735352e+46 Castro::do_old_sources() time = 0.058835214 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.98733239e+46 ener added: 2.180535114e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.327475474 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.014754463 Gravity::solve_for_phi() time = 0.08545248 Castro::construct_new_gravity() time = 0.087510852 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.058042977e+43 ener added: 1.184398423e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.782398841e+35 ymom added: -3.291489439e+24 zmom added: 0 eint added: 0 ener added: 5.968491079e+43 Castro::do_new_sources() time = 0.059200746 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.881810984e+29 ener added: 1.881810984e+29 Castro::construct_old_gravity() time = 0.002533056 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.984353173e+46 ener added: 6.990361455e+42 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.46774173e+40 ymom added: 9.41450982e+26 zmom added: 0 eint added: 0 ener added: 8.780735352e+46 Castro::do_old_sources() time = 0.058322389 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.988039066e+46 ener added: 2.185480971e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.328332723 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.01587273 Gravity::solve_for_phi() time = 0.087661642 Castro::construct_new_gravity() time = 0.089763443 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.800848844e+43 ener added: 1.183591058e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.776272322e+35 ymom added: 1.291567233e+24 zmom added: 0 eint added: 0 ener added: 5.233809214e+43 Castro::do_new_sources() time = 0.059142312 Ending SDC iteration 2 of 2.  Subcycle completed  Beginning subcycle 4 starting at time 0.09375 with dt = 0.03125 Estimated number of subcycles remaining: 5 Beginning SDC iteration 1 of 2. Castro::construct_old_gravity() time = 0.002800533 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.983917922e+46 ener added: 1.154265495e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467777256e+40 ymom added: 9.44246623e+26 zmom added: 0 eint added: 0 ener added: 8.841018465e+46 Castro::do_old_sources() time = 0.058619989 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.98498831e+46 ener added: 2.292089986e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.322815793 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.014763473 Gravity::solve_for_phi() time = 0.084627827 Castro::construct_new_gravity() time = 0.086726324 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -4.614759295e+43 ener added: 1.488589688e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.803591267e+35 ymom added: 6.776595903e+23 zmom added: 0 eint added: 0 ener added: 4.637081169e+43 Castro::do_new_sources() time = 0.058750715 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Castro::construct_old_gravity() time = 0.002717044 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.983917922e+46 ener added: 1.154265495e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467777256e+40 ymom added: 9.44246623e+26 zmom added: 0 eint added: 0 ener added: 8.841018465e+46 Castro::do_old_sources() time = 0.058585975 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.985698801e+46 ener added: 2.298024982e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.333790431 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.016484667 Gravity::solve_for_phi() time = 0.088771646 Castro::construct_new_gravity() time = 0.090853269 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -5.359798245e+43 ener added: 1.487684259e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.797448342e+35 ymom added: 5.252452121e+24 zmom added: 0 eint added: 0 ener added: 3.901081726e+43 Castro::do_new_sources() time = 0.059780608 Ending SDC iteration 2 of 2.  Subcycle completed  Beginning subcycle 5 starting at time 0.125 with dt = 0.03125 Estimated number of subcycles remaining: 4 Beginning SDC iteration 1 of 2. Castro::construct_old_gravity() time = 0.002698 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.98356532e+46 ener added: 1.521222674e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467813205e+40 ymom added: 9.540691453e+26 zmom added: 0 eint added: 0 ener added: 8.902573838e+46 Castro::do_old_sources() time = 0.058366339 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.992363162e+46 ener added: 2.321339475e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.328608835 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.016175401 Gravity::solve_for_phi() time = 0.088278321 Castro::construct_new_gravity() time = 0.090396267 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.068921344e+43 ener added: 1.620777103e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.829776437e+35 ymom added: -4.179294337e+24 zmom added: 0 eint added: 0 ener added: 2.123241033e+43 Castro::do_new_sources() time = 0.059563581 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Castro::construct_old_gravity() time = 0.002527874 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.98356532e+46 ener added: 1.521222674e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467813205e+40 ymom added: 9.540691453e+26 zmom added: 0 eint added: 0 ener added: 8.902573838e+46 Castro::do_old_sources() time = 0.058583925 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.993083003e+46 ener added: 2.326773931e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.329218768 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015118757 Gravity::solve_for_phi() time = 0.085894049 Castro::construct_new_gravity() time = 0.087961115 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.821028634e+43 ener added: 1.619931887e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.823603759e+35 ymom added: 1.205384045e+24 zmom added: 0 eint added: 0 ener added: 1.379533843e+43 Castro::do_new_sources() time = 0.060566939 Ending SDC iteration 2 of 2.  Subcycle completed  Beginning subcycle 6 starting at time 0.15625 with dt = 0.03125 Estimated number of subcycles remaining: 3 Beginning SDC iteration 1 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.908356001e+29 ener added: 4.908356001e+29 Castro::construct_old_gravity() time = 0.004043317 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.983073759e+46 ener added: 2.019352467e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467849677e+40 ymom added: 9.586026171e+26 zmom added: 0 eint added: 0 ener added: 8.96490842e+46 Castro::do_old_sources() time = 0.058496687 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 7.000609588e+46 ener added: 2.406064512e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.323667025 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015238833 Gravity::solve_for_phi() time = 0.086596193 Castro::construct_new_gravity() time = 0.088716644 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -8.854195608e+43 ener added: 1.71395957e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.848617495e+35 ymom added: -3.296211805e+24 zmom added: 0 eint added: 0 ener added: -1.366537222e+43 Castro::do_new_sources() time = 0.059055176 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.908356001e+29 ener added: 4.908356001e+29 Castro::construct_old_gravity() time = 0.002511454 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.983073759e+46 ener added: 2.019352467e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467849677e+40 ymom added: 9.586026171e+26 zmom added: 0 eint added: 0 ener added: 8.96490842e+46 Castro::do_old_sources() time = 0.058396145 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 7.001333674e+46 ener added: 2.41251015e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.329310853 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015065104 Gravity::solve_for_phi() time = 0.086133562 Castro::construct_new_gravity() time = 0.088188038 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -9.610031177e+43 ener added: 1.713054017e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.842414694e+35 ymom added: -1.986935698e+24 zmom added: 0 eint added: 0 ener added: -2.113374984e+43 Castro::do_new_sources() time = 0.058813789 Ending SDC iteration 2 of 2.  Subcycle completed  Beginning subcycle 7 starting at time 0.1875 with dt = 0.03125 Estimated number of subcycles remaining: 2 Beginning SDC iteration 1 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.009461182e+29 ener added: 2.009461182e+29 Castro::construct_old_gravity() time = 0.002522202 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.982914566e+46 ener added: 2.174746794e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467886525e+40 ymom added: 9.502912521e+26 zmom added: 0 eint added: 0 ener added: 9.027789929e+46 Castro::do_old_sources() time = 0.058278666 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.993854135e+46 ener added: 2.676087223e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.324157883 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015423483 Gravity::solve_for_phi() time = 0.086868473 Castro::construct_new_gravity() time = 0.088907082 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.152849071e+44 ener added: 1.743211419e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.877263956e+35 ymom added: 2.991619167e+24 zmom added: 0 eint added: 0 ener added: -4.049549507e+43 Castro::do_new_sources() time = 0.059172971 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.009461182e+29 ener added: 2.009461182e+29 Castro::construct_old_gravity() time = 0.002674229 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.982914566e+46 ener added: 2.174746794e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467886525e+40 ymom added: 9.502912521e+26 zmom added: 0 eint added: 0 ener added: 9.027789929e+46 Castro::do_old_sources() time = 0.058424963 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.994586452e+46 ener added: 2.682600444e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.328148663 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015202281 Gravity::solve_for_phi() time = 0.086157899 Castro::construct_new_gravity() time = 0.088207928 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -1.229235664e+44 ener added: 1.74231031e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.871026436e+35 ymom added: -4.264296934e+24 zmom added: 0 eint added: 0 ener added: -4.804462625e+43 Castro::do_new_sources() time = 0.059489019 Ending SDC iteration 2 of 2.  Subcycle completed  Beginning subcycle 8 starting at time 0.21875 with dt = 0.03125 Estimated number of subcycles remaining: 1 Beginning SDC iteration 1 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.20728525e+29 ener added: 9.20728525e+29 Castro::construct_old_gravity() time = 0.002667554 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.982548267e+46 ener added: 2.529322566e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467923946e+40 ymom added: 9.435666022e+26 zmom added: 0 eint added: 0 ener added: 9.091353867e+46 Castro::do_old_sources() time = 0.058113266 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.993555556e+46 ener added: 2.937230463e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.32352912 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.016115418 Gravity::solve_for_phi() time = 0.086721119 Castro::construct_new_gravity() time = 0.088761516 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -8.731399969e+43 ener added: 1.766195089e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.939762635e+35 ymom added: 4.946678891e+24 zmom added: 0 eint added: 0 ener added: -1.225922477e+43 Castro::do_new_sources() time = 0.05975034 Ending SDC iteration 1 of 2. Beginning SDC iteration 2 of 2. Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.20728525e+29 ener added: 9.20728525e+29 Castro::construct_old_gravity() time = 0.002736926 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -6.982548267e+46 ener added: 2.529322566e+43 Contributions to the state from old-time sources: mass added: 0 xmom added: -1.467923946e+40 ymom added: 9.435666022e+26 zmom added: 0 eint added: 0 ener added: 9.091353867e+46 Castro::do_old_sources() time = 0.058497456 ... Entering construct_ctu_hydro_source() Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 6.994288712e+46 ener added: 2.943682879e+43 ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.329116729 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.015293585 Gravity::solve_for_phi() time = 0.086709507 Castro::construct_new_gravity() time = 0.088812334 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -9.496809054e+43 ener added: 1.76531212e+44 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.933525523e+35 ymom added: -4.247768651e+24 zmom added: 0 eint added: 0 ener added: -1.982551467e+43 Castro::do_new_sources() time = 0.059276378 Ending SDC iteration 2 of 2.  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.002801959958 [Level 0 step 1] Advanced 32768 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.117748323e+27 ener added: 4.117748323e+27 TIME= 0.25 MASS = 2.549996568e+33 TIME= 0.25 XMOM = -4.340632835e+38 TIME= 0.25 YMOM = 1.160568787e+28 TIME= 0.25 ZMOM = 0 TIME= 0.25 ANG MOM X = 0 TIME= 0.25 ANG MOM Y = 0 TIME= 0.25 ANG MOM Z = -1.471393084e+38 TIME= 0.25 RHO*e = 1.325894499e+50 TIME= 0.25 RHO*K = 3.170941995e+49 TIME= 0.25 RHO*E = 1.642988698e+50 TIME= 0.25 RHO*PHI = -5.449011351e+50 TIME= 0.25 TOTAL ENERGY= -1.081516977e+50 TIME= 0.25 CENTER OF MASS X-LOC = 316424375.9 TIME= 0.25 CENTER OF MASS X-VEL = -170221.1246 TIME= 0.25 CENTER OF MASS Y-LOC = 7.282010032e-07 TIME= 0.25 CENTER OF MASS Y-VEL = 4.55125627e-06 TIME= 0.25 CENTER OF MASS Z-LOC = 0 TIME= 0.25 CENTER OF MASS Z-VEL = 0 TIME= 0.25 MAXIMUM TEMPERATURE = 58251090.19 TIME= 0.25 MAXIMUM DENSITY = 4458791.937 TIME= 0.25 MAXIMUM T_S / T_E = 0 [STEP 1] Coarse TimeStep time: 11.75278938 [STEP 1] FAB kilobyte spread across MPI nodes: [6973 ... 6973] STEP = 1 TIME = 0.25 DT = 0.25 PLOTFILE: file = wdmerger_retry-SDC_plt00001 Write plotfile time = 0.052779404 seconds SMALL PLOTFILE: file = smallplt00001 Write small plotfile time = 0.007793705 seconds Ending run at 07:14:46 UTC on 2023-06-28. Run time = 14.25435252 Run time without initialization = 11.81435796 Average number of zones advanced per microsecond: 0.003 Average number of zones advanced per microsecond per rank: 0.000 TinyProfiler total time across processes [min...avg...max]: 14.25 ... 14.25 ... 14.25 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 20 5.4048 5.4513 5.5186 38.71% Amr::InitAmr() 1 1.576 1.576 1.576 11.06% Castro::computeTemp() 80 0.8609 0.9675 1.128 7.92% amrex::Copy() 795 0.7687 0.7788 0.7881 5.53% FabArray::ParallelCopy_finish() 1429 0.05029 0.4814 0.6883 4.83% Castro::normalize_species() 79 0.5407 0.5457 0.5689 3.99% MultiFab::AddProduct() 3439 0.5484 0.5514 0.5586 3.92% Castro::reset_internal_energy(Fab) 160 0.5259 0.5306 0.5398 3.79% Castro::initMFs() 1 0.3625 0.3722 0.4214 2.96% MLPoisson::Fsmooth() 6144 0.3132 0.3413 0.4147 2.91% FabArray::setVal() 5021 0.3439 0.3498 0.3598 2.52% MultiFab::Subtract() 159 0.3365 0.3398 0.3422 2.40% Gravity::fill_multipole_BCs() 19 0.1956 0.2384 0.2787 1.95% FabArray::ParallelCopy_nowait() 1429 0.1826 0.2002 0.2218 1.56% FabArray::sum() 3439 0.1827 0.1837 0.1862 1.31% Castro::reset_internal_energy(MultiFab) 80 0.02251 0.05425 0.1736 1.22% check_for_negative_density() 20 0.05719 0.1161 0.1675 1.18% Castro::initData() 1 0.003269 0.0224 0.1283 0.90% FillBoundary_nowait() 6388 0.07261 0.09789 0.1254 0.88% MLPoisson::Fapply() 994 0.09446 0.09912 0.1153 0.81% FillBoundary_finish() 6388 0.08139 0.09919 0.1137 0.80% MLCellLinOp::applyBC() 7156 0.04679 0.06224 0.111 0.78% FabArray::Saxpy() 192 0.06914 0.0698 0.0747 0.52% Castro::do_new_sources() 17 0.0003623 0.01282 0.07283 0.51% Castro::construct_new_gravity_source() 17 0.0678 0.0687 0.06939 0.49% FabArray::Xpay() 953 0.05033 0.05256 0.0603 0.42% Castro::construct_old_gravity_source() 20 0.046 0.04629 0.04705 0.33% MultiFab::contains_nan() 37 0.04597 0.04646 0.04692 0.33% Castro::derive() 18 0.04169 0.04211 0.04429 0.31% MLMG::addInterpCorrection() 768 0.03358 0.03603 0.04422 0.31% Castro::do_old_sources() 20 0.0007431 0.008973 0.04031 0.28% Castro::enforce_min_density() 79 0.01724 0.02214 0.03732 0.26% StateData::FillBoundary(geom) 143 0 0.01764 0.03415 0.24% amrex::average_down 768 0.02504 0.02709 0.03387 0.24% VisMF::Write(FabArray) 4 0.01655 0.01846 0.03167 0.22% MLCGSolver::bicgstab 8 0 0.001508 0.02413 0.17% Amr::writePlotFile() 2 0.009742 0.02065 0.02306 0.16% MLCellLinOp::defineAuxData() 19 0.01526 0.01595 0.01923 0.13% FabArray::FillBoundary() 6388 0.01398 0.01451 0.01813 0.13% Castro::construct_new_source() 102 0.01531 0.01549 0.01595 0.11% Castro::construct_old_gravity() 20 0.0003456 0.003338 0.01511 0.11% MLCellLinOp::smooth() 3072 0.01206 0.01255 0.01461 0.10% amrex::Add() 136 0.01385 0.01398 0.01452 0.10% FabArray::norminf() 244 0.0109 0.01128 0.01451 0.10% Castro::evaluate_source_change() 196 0.0134 0.01383 0.0144 0.10% FillPatchIterator::Initialize 147 0.01324 0.01361 0.01416 0.10% FabArrayBase::getFB() 6388 0.01008 0.0105 0.01402 0.10% MLMG::prepareForSolve() 18 0.01053 0.01089 0.01194 0.08% FabArray::setDomainBndry() 147 0.0004574 0.006174 0.01169 0.08% Castro::wd_update() 2 0.004845 0.008136 0.01039 0.07% Castro::construct_old_source() 120 0.009787 0.009923 0.01033 0.07% Amr::writeSmallPlotFile() 2 0.007626 0.009601 0.01008 0.07% TagBoxArray::collate() 2 0.0001037 0.001669 0.009865 0.07% MLMG::solve() 18 0.0007827 0.004135 0.008633 0.06% Gravity::actual_solve_with_mlmg() 19 0.006922 0.007253 0.008544 0.06% MLLinOp::makeSubCommunicator() 19 0.0009506 0.004531 0.008426 0.06% FabArrayBase::getCPC() 2005 0.007296 0.007544 0.008163 0.06% Castro::volWgtSum() 46 0.007354 0.007414 0.007447 0.05% FabArray::mult() 59 0.006847 0.006943 0.007204 0.05% MLMG::ResNormInf() 146 0.003546 0.005437 0.007158 0.05% FabArrayBase::CPC::define() 576 0.002229 0.003475 0.006979 0.05% Gravity::get_old_grav_vector() 20 0.006625 0.006749 0.006897 0.05% Gravity::get_new_grav_vector() 18 0.006222 0.006336 0.006758 0.05% MLCGSolver::ParallelAllReduce 189 0 0.0004217 0.006747 0.05% amrex::Dot() 142 0 0.0004013 0.00642 0.05% MLCellLinOp::compGrad() 18 0.005753 0.005892 0.006032 0.04% Castro::initialize_do_advance() 20 0.004495 0.005147 0.005852 0.04% MLCellLinOp::apply() 994 0.002133 0.002407 0.005735 0.04% Castro::volInBoundary() 14 0.003452 0.00372 0.005576 0.04% MultiFab::max() 2 0.0001469 0.004235 0.004953 0.03% FabArray::ParallelCopy() 1429 0.003756 0.003886 0.004349 0.03% FabArray::LinComb() 71 0 0.0002362 0.003779 0.03% MLPoisson::normalize() 79 0 0.0002105 0.003368 0.02% BndryData::define() 19 0.003138 0.003211 0.003341 0.02% MLCellLinOp::correctionResidual() 776 0.001935 0.00202 0.002576 0.02% MLLinOp::defineGrids() 19 0.001864 0.001987 0.002334 0.02% MLCellLinOp::prepareForSolve() 19 0.001706 0.001853 0.002236 0.02% Castro::sum_integrated_quantities() 2 0.001661 0.002023 0.002208 0.02% Castro::save_data_for_retry() 3 0.001268 0.001357 0.002024 0.01% MLMG::mgVcycle() 128 0.001643 0.001735 0.002004 0.01% FabArray::setVal(val, thecmd, scomp, ncomp) 608 0.001325 0.001581 0.001876 0.01% MLCellLinOp::defineBC() 19 0.001564 0.001632 0.001868 0.01% StateData::define() 6 0.001403 0.001531 0.001699 0.01% MLMG:computeResOfCorrection() 768 0.001427 0.001476 0.001615 0.01% FabArrayBase::FB::FB() 75 0.0002836 0.0007091 0.001575 0.01% main() 1 0.0009256 0.001211 0.001401 0.01% MLMG::actualBottomSolve() 128 0.0001357 0.000211 0.001258 0.01% Gravity::solve_for_phi() 17 0.0007921 0.0009505 0.001229 0.01% Castro::locWgtSum() 6 0.001142 0.001174 0.001203 0.01% Castro::construct_new_gravity() 17 0.0002757 0.0004398 0.0009833 0.01% StateDataPhysBCFunct::() 147 0.0006551 0.0008416 0.0009648 0.01% Castro::subcycle_advance_ctu() 1 0.0001915 0.0002713 0.0008946 0.01% MLCellLinOp::setLevelBC() 19 0.0005487 0.0006784 0.0008542 0.01% Castro::do_advance_ctu() 20 0.0006852 0.0007477 0.0008522 0.01% MLMG::mgVcycle_down::3 128 0.0006271 0.0006739 0.0008079 0.01% MLMG::mgVcycle_down::2 128 0.0006691 0.000721 0.0008012 0.01% MLMG::mgVcycle_down::4 128 0.0006088 0.0006338 0.0007916 0.01% MLMG::mgVcycle_down::5 128 0.0006308 0.0006547 0.0007807 0.01% MLMG::mgVcycle_down::0 128 0.0007249 0.0007349 0.0007747 0.01% MLMG::mgVcycle_down::1 128 0.0007096 0.0007261 0.0007587 0.01% Castro::swap_state_time_levels() 8 0.000594 0.0006572 0.0007174 0.01% Castro::Castro() 1 0.0004667 0.0005528 0.0006845 0.00% Castro::buildMetrics() 1 0.0005704 0.0005792 0.0006161 0.00% Castro::clean_state() 79 0.0004398 0.0004913 0.000582 0.00% MLMG::mgVcycle_up::5 128 0.0004324 0.0004512 0.0005543 0.00% FillPatchSingleLevel 147 0.0005004 0.0005195 0.0005525 0.00% MLMG::mgVcycle_up::0 128 0.0005068 0.0005213 0.000551 0.00% MLMG::mgVcycle_up::1 128 0.0004826 0.000507 0.0005461 0.00% Castro::apply_tagging_restrictions() 2 0.0004045 0.0004676 0.0005327 0.00% AmrMesh::MakeNewGrids() 2 0.0004996 0.0005123 0.0005321 0.00% MLMG::mgVcycle_up::3 128 0.0004179 0.0004344 0.0005269 0.00% MLMG::mgVcycle_up::2 128 0.0004201 0.0004638 0.000526 0.00% MLMG::mgVcycle_up::4 128 0.000423 0.0004348 0.000521 0.00% MLCellLinOp::solutionResidual() 147 0.0004513 0.0004691 0.0004972 0.00% Gravity::applyMetricTerms() 2 0.000477 0.0004845 0.0004958 0.00% makeSFC 95 0.0003964 0.0004267 0.000495 0.00% Castro::expand_state() 20 0.0004162 0.0004537 0.0004886 0.00% MLMG::oneIter() 128 0.0004031 0.000416 0.0004463 0.00% Castro::volProductSum() 2 0.0003942 0.0004071 0.0004221 0.00% Castro::problem_post_timestep() 2 1.081e-05 3.691e-05 0.0004095 0.00% Castro::enforce_consistent_e() 1 0.0003556 0.0003657 0.0003958 0.00% MLLinOp::makeAgglomeratedDMap 19 0.000349 0.0003661 0.0003902 0.00% Castro::create_source_corrector() 17 0.0002885 0.0003124 0.0003577 0.00% Gravity::solve_phi_with_mlmg() 19 0.0002313 0.0002519 0.0002958 0.00% MLMG::mgVcycle_bottom 128 0.0002396 0.0002542 0.0002706 0.00% DistributionMapping::Distribute() 95 0.0002195 0.0002336 0.0002664 0.00% MLMG::computeResidual() 128 0.0002476 0.000256 0.0002654 0.00% Castro::finalize_do_advance() 17 0.0001607 0.000191 0.000232 0.00% Amr::coarseTimeStep() 1 0.0001496 0.0002004 0.0002089 0.00% Gravity::update_max_rhs() 2 0.0001646 0.000172 0.0001822 0.00% OwnerMask() 2 0.0001356 0.0001507 0.000175 0.00% Castro::initialize_advance() 1 0.0001267 0.0001456 0.0001655 0.00% Castro::check_for_nan() 37 0.0001065 0.0001206 0.0001608 0.00% MLMG::compResidual() 1 0.000156 0.0001567 0.0001576 0.00% Amr::defBaseLevel() 1 0.0001012 0.0001293 0.0001561 0.00% Castro::retry_advance_ctu() 11 5.388e-05 6.509e-05 0.0001547 0.00% MLPoisson::define() 19 0.000133 0.0001434 0.0001513 0.00% Castro::enforce_speed_limit() 79 0.0001105 0.0001176 0.0001294 0.00% Gravity::test_composite_phi() 1 9.281e-05 0.0001028 0.0001293 0.00% Amr::grid_places() 2 8.327e-05 9.754e-05 0.0001252 0.00% TagBoxArray::mapPRD 2 0.0001119 0.0001173 0.0001242 0.00% Castro::apply_source_to_state() 37 0.0001018 0.0001128 0.000122 0.00% Amr::FinalizeInit() 1 8.762e-06 1.63e-05 0.0001039 0.00% MLLinOp::define() 19 8.078e-05 9.025e-05 0.0001001 0.00% Castro::apply_problem_tags() 2 9.151e-05 9.328e-05 9.567e-05 0.00% MultiFab::Swap() 5 7.027e-05 7.872e-05 9.416e-05 0.00% MLPoisson::prepareForSolve() 19 7.093e-05 7.715e-05 8.565e-05 0.00% Gravity::multilevel_solve_for_new_phi() 1 6.113e-05 6.839e-05 8.334e-05 0.00% Amr::initSubcycle() 1 6.185e-05 6.392e-05 6.892e-05 0.00% knapsack() 1 5.969e-05 6.174e-05 6.467e-05 0.00% MLMG::computeMLResidual() 18 4.012e-05 4.678e-05 5.665e-05 0.00% DistributionMapping::LeastUsedCPUs() 1 1.58e-05 3.649e-05 5.313e-05 0.00% Gravity::swapTimeLevels() 8 3.32e-05 4.042e-05 4.965e-05 0.00% MLMG::getGradSolution() 18 4.079e-05 4.29e-05 4.55e-05 0.00% MLMG::MLResNormInf() 18 3.504e-05 3.776e-05 4.419e-05 0.00% MLMG::MLRhsNormInf() 18 3.293e-05 3.388e-05 3.917e-05 0.00% AmrLevel::AmrLevel(dm) 1 3.442e-05 3.52e-05 3.634e-05 0.00% Castro::construct_old_rotation_source() 20 2.958e-05 3.088e-05 3.458e-05 0.00% Amr::regrid() 1 1.193e-05 1.407e-05 3.19e-05 0.00% Gravity::actual_multilevel_solve() 1 2.773e-05 2.945e-05 3.129e-05 0.00% Castro::post_init() 1 1.244e-05 1.456e-05 3.091e-05 0.00% Castro::finalize_advance() 1 1.765e-05 2.06e-05 2.968e-05 0.00% Castro::construct_new_rotation_source() 17 2.221e-05 2.296e-05 2.741e-05 0.00% Amr::timeStep() 1 1.08e-05 1.53e-05 2.602e-05 0.00% DistributionMapping::KnapSackDoIt() 1 1.657e-05 1.736e-05 1.902e-05 0.00% Castro::estTimeStep() 10 1.22e-05 1.391e-05 1.558e-05 0.00% Castro::post_regrid() 2 1.108e-05 1.275e-05 1.513e-05 0.00% Amr::InitializeInit() 1 6.886e-06 9.428e-06 1.439e-05 0.00% Castro::computeInitialDt() 2 7.502e-06 8.256e-06 1.056e-05 0.00% Amr::bldFineLevels() 1 6.972e-06 8.2e-06 1.033e-05 0.00% Castro::advance() 1 6.826e-06 8.307e-06 1.011e-05 0.00% Amr::init() 1 7.311e-06 8.049e-06 8.713e-06 0.00% Gravity::set_mass_offset() 2 4.399e-06 6.097e-06 8.233e-06 0.00% Castro::post_timestep() 1 5.776e-06 6.489e-06 7.834e-06 0.00% BoxList::parallelComplementIn() 4 7.194e-06 7.485e-06 7.827e-06 0.00% Castro::errorEst() 2 6.13e-06 6.555e-06 7.242e-06 0.00% Castro::gwstrain() 2 4.487e-06 5.058e-06 6.985e-06 0.00% BoxList::complementIn 4 6.051e-06 6.223e-06 6.541e-06 0.00% Amr::initialInit() 1 2.906e-06 3.356e-06 4.954e-06 0.00% Castro::FluxRegCrseInit 1 1.647e-06 1.934e-06 2.309e-06 0.00% Castro::FluxRegFineAdd() 1 1.384e-06 1.578e-06 1.848e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 14.25 14.25 14.25 100.00% Amr::coarseTimeStep() 1 11.81 11.81 11.81 82.88% Amr::timeStep() 1 11.75 11.75 11.75 82.45% Castro::advance() 1 11.69 11.69 11.69 82.04% Castro::subcycle_advance_ctu() 1 11.64 11.64 11.65 81.70% Castro::do_advance_ctu() 20 11.6 11.61 11.61 81.44% Castro::construct_ctu_hydro_source() 20 6.311 6.345 6.384 44.79% Castro::clean_state() 79 3.679 3.778 3.918 27.49% Castro::computeTemp() 80 2.383 2.496 2.69 18.87% Castro::reset_internal_energy(MultiFab) 80 1.488 1.528 1.639 11.50% Gravity::solve_phi_with_mlmg() 19 1.491 1.536 1.577 11.07% Amr::InitAmr() 1 1.576 1.576 1.576 11.06% Castro::construct_new_gravity() 17 1.422 1.465 1.506 10.56% Gravity::solve_for_phi() 17 1.384 1.429 1.47 10.31% Castro::initialize_do_advance() 20 1.354 1.386 1.418 9.95% Gravity::actual_solve_with_mlmg() 19 1.29 1.294 1.299 9.11% MLMG::solve() 18 1.237 1.243 1.244 8.73% Castro::do_old_sources() 20 1.144 1.166 1.183 8.30% MLMG::oneIter() 128 1.108 1.11 1.113 7.81% MLMG::mgVcycle() 128 1.093 1.095 1.099 7.71% Castro::do_new_sources() 17 0.9324 0.9764 1.013 7.10% Castro::evaluate_source_change() 196 0.9401 0.9455 0.9581 6.72% FabArray::ParallelCopy() 1429 0.2763 0.7051 0.9087 6.38% Amr::init() 1 0.8636 0.8636 0.8636 6.06% Amr::initialInit() 1 0.8041 0.8041 0.8043 5.64% Castro::enforce_min_density() 79 0.7802 0.7893 0.8036 5.64% amrex::Copy() 795 0.7687 0.7788 0.7881 5.53% MLCellLinOp::smooth() 3072 0.5343 0.5819 0.7097 4.98% FabArray::ParallelCopy_finish() 1429 0.05029 0.4814 0.6883 4.83% Amr::InitializeInit() 1 0.4513 0.5027 0.582 4.08% Amr::defBaseLevel() 1 0.4513 0.5027 0.582 4.08% Castro::normalize_species() 79 0.5407 0.5457 0.5689 3.99% MultiFab::AddProduct() 3439 0.5484 0.5514 0.5586 3.92% Castro::reset_internal_energy(Fab) 160 0.5259 0.5306 0.5398 3.79% FillPatchIterator::Initialize 147 0.2952 0.3627 0.4472 3.14% MLMG::mgVcycle_up::1 128 0.09006 0.2647 0.4389 3.08% FillPatchSingleLevel 147 0.2697 0.3429 0.4294 3.01% Castro::Castro() 1 0.369 0.378 0.4267 2.99% Castro::initMFs() 1 0.3627 0.3724 0.4216 2.96% MLPoisson::Fsmooth() 6144 0.3132 0.3413 0.4147 2.91% MLMG::addInterpCorrection() 768 0.06462 0.3262 0.3935 2.76% FabArray::setVal() 5021 0.3439 0.3498 0.3598 2.52% MLCellLinOp::applyBC() 7156 0.2578 0.285 0.3576 2.51% Amr::FinalizeInit() 1 0.2221 0.3014 0.3529 2.48% MultiFab::Subtract() 159 0.3365 0.3398 0.3422 2.40% MLMG::mgVcycle_up::2 128 0.01113 0.1254 0.3017 2.12% Gravity::fill_multipole_BCs() 19 0.1977 0.2405 0.2808 1.97% MLMG::mgVcycle_down::0 128 0.2573 0.2593 0.261 1.83% FabArray::FillBoundary() 6388 0.1994 0.2228 0.2552 1.79% FabArray::ParallelCopy_nowait() 1429 0.2014 0.2198 0.2425 1.70% Castro::expand_state() 20 0.1748 0.1873 0.215 1.51% Castro::initData() 1 0.07779 0.1229 0.2107 1.48% MLMG::mgVcycle_up::3 128 0.01106 0.02561 0.2061 1.45% MLCellLinOp::apply() 994 0.1466 0.1569 0.1968 1.38% MLMG::mgVcycle_up::0 128 0.189 0.1908 0.1934 1.36% FabArray::sum() 3439 0.1827 0.1837 0.1862 1.31% Amr::bldFineLevels() 1 0.04571 0.125 0.1764 1.24% check_for_negative_density() 20 0.05719 0.1161 0.1675 1.18% MLCellLinOp::correctionResidual() 776 0.11 0.1189 0.1465 1.03% MLMG:computeResOfCorrection() 768 0.1114 0.1201 0.1436 1.01% FillBoundary_nowait() 6388 0.08311 0.1091 0.137 0.96% Castro::post_init() 1 0.1292 0.1333 0.134 0.94% Amr::grid_places() 2 0.002872 0.08218 0.1336 0.94% AmrMesh::MakeNewGrids() 2 0.002784 0.08208 0.1335 0.94% TagBoxArray::mapPRD 2 0.0006471 0.07917 0.1314 0.92% Castro::construct_new_source() 102 0.1221 0.1233 0.1246 0.87% MLMG::mgVcycle_down::1 128 0.1089 0.1124 0.116 0.81% MLPoisson::Fapply() 994 0.09446 0.09912 0.1153 0.81% FillBoundary_finish() 6388 0.08139 0.09919 0.1137 0.80% Amr::writePlotFile() 2 0.104 0.1041 0.1041 0.73% Gravity::multilevel_solve_for_new_phi() 1 0.0971 0.09711 0.09714 0.68% Gravity::actual_multilevel_solve() 1 0.09703 0.09705 0.09707 0.68% MLCellLinOp::solutionResidual() 147 0.08573 0.09127 0.09696 0.68% MLMG::mgVcycle_bottom 128 0.0003786 0.006223 0.09366 0.66% MLMG::actualBottomSolve() 128 0.0001357 0.005969 0.09339 0.66% MLCGSolver::bicgstab 8 0 0.005726 0.09162 0.64% Castro::create_source_corrector() 17 0.04762 0.06962 0.09003 0.63% Castro::post_regrid() 2 0.08516 0.08588 0.08995 0.63% Castro::construct_old_source() 120 0.07884 0.07936 0.08016 0.56% MLMG::computeResidual() 128 0.07335 0.07477 0.07626 0.53% FabArray::Saxpy() 192 0.06914 0.0698 0.0747 0.52% Castro::derive() 18 0.06994 0.0705 0.07281 0.51% Castro::construct_new_gravity_source() 17 0.0678 0.0687 0.06939 0.49% MLMG::mgVcycle_down::2 128 0.01288 0.03952 0.06896 0.48% Castro::construct_old_gravity() 20 0.0412 0.05618 0.06264 0.44% Castro::post_timestep() 1 0.05801 0.06032 0.06241 0.44% Gravity::get_old_grav_vector() 20 0.03896 0.05284 0.06148 0.43% MLMG::mgVcycle_down::3 128 0.01229 0.01851 0.06085 0.43% FabArray::Xpay() 953 0.05033 0.05256 0.0603 0.42% amrex::average_down 768 0.03344 0.03925 0.0554 0.39% Castro::initialize_advance() 1 0.04835 0.04929 0.05161 0.36% Castro::construct_old_gravity_source() 20 0.046 0.04629 0.04705 0.33% Castro::check_for_nan() 37 0.04608 0.04658 0.04704 0.33% MultiFab::contains_nan() 37 0.04597 0.04646 0.04692 0.33% Amr::regrid() 1 0.0436 0.04379 0.04408 0.31% MLPoisson::define() 19 0.03009 0.03466 0.04032 0.28% Castro::retry_advance_ctu() 11 0.03549 0.03618 0.03879 0.27% MLMG::mgVcycle_down::4 128 0.01219 0.01388 0.03645 0.26% Gravity::get_new_grav_vector() 18 0.03481 0.03524 0.03635 0.26% StateDataPhysBCFunct::() 147 0.0006551 0.01849 0.03511 0.25% StateData::FillBoundary(geom) 143 0 0.01764 0.03415 0.24% VisMF::Write(FabArray) 4 0.01655 0.01846 0.03167 0.22% Castro::apply_source_to_state() 37 0.02824 0.02844 0.02989 0.21% MLMG::mgVcycle_up::4 128 0.01115 0.01233 0.02875 0.20% MLCellLinOp::defineAuxData() 19 0.01952 0.02096 0.02656 0.19% MLMG::mgVcycle_down::5 128 0.01223 0.01312 0.02493 0.17% MLMG::prepareForSolve() 18 0.02074 0.0217 0.02471 0.17% MLMG::mgVcycle_up::5 128 0.01122 0.01192 0.02147 0.15% MLMG::computeMLResidual() 18 0.01047 0.0161 0.02146 0.15% Castro::sum_integrated_quantities() 2 0.02105 0.02131 0.02133 0.15% Castro::save_data_for_retry() 3 0.01858 0.01902 0.02053 0.14% MLMG::ResNormInf() 146 0.01321 0.0151 0.01682 0.12% Amr::writeSmallPlotFile() 2 0.0161 0.0161 0.0161 0.11% FabArrayBase::getFB() 6388 0.01042 0.01121 0.01559 0.11% FabArrayBase::getCPC() 2005 0.009572 0.01102 0.01514 0.11% amrex::Add() 136 0.01385 0.01398 0.01452 0.10% FabArray::norminf() 244 0.0109 0.01128 0.01451 0.10% Castro::problem_post_timestep() 2 0.009375 0.01189 0.01388 0.10% Castro::wd_update() 2 0.009206 0.01186 0.01386 0.10% Gravity::test_composite_phi() 1 0.0138 0.01383 0.01384 0.10% MLLinOp::define() 19 0.004299 0.007635 0.01177 0.08% FabArray::setDomainBndry() 147 0.0004574 0.006174 0.01169 0.08% MLLinOp::defineGrids() 19 0.004199 0.007545 0.01169 0.08% MLCGSolver::ParallelAllReduce 189 0 0.0006417 0.01027 0.07% TagBoxArray::collate() 2 0.0001037 0.001669 0.009865 0.07% MLLinOp::makeSubCommunicator() 19 0.0009506 0.004531 0.008426 0.06% MLMG::getGradSolution() 18 0.007325 0.007598 0.007848 0.06% MLCellLinOp::compGrad() 18 0.007284 0.007555 0.007807 0.05% Castro::volWgtSum() 46 0.007354 0.007414 0.007447 0.05% FabArray::mult() 59 0.006847 0.006943 0.007204 0.05% FabArrayBase::CPC::define() 576 0.002229 0.003475 0.006979 0.05% amrex::Dot() 142 0 0.0004013 0.00642 0.05% Gravity::update_max_rhs() 2 0.001398 0.005473 0.006199 0.04% MLCellLinOp::defineBC() 19 0.005638 0.005916 0.006171 0.04% Castro::volInBoundary() 14 0.003452 0.00372 0.005576 0.04% MultiFab::max() 2 0.0001469 0.004235 0.004953 0.03% BndryData::define() 19 0.004049 0.004284 0.004489 0.03% FabArray::LinComb() 71 0 0.0002362 0.003779 0.03% MLPoisson::normalize() 79 0 0.0002105 0.003368 0.02% MLPoisson::prepareForSolve() 19 0.001782 0.00193 0.002315 0.02% MLCellLinOp::prepareForSolve() 19 0.001706 0.001853 0.002236 0.02% FabArray::setVal(val, thecmd, scomp, ncomp) 608 0.001325 0.001581 0.001876 0.01% AmrLevel::AmrLevel(dm) 1 0.001438 0.001566 0.001734 0.01% StateData::define() 6 0.001403 0.001531 0.001699 0.01% FabArrayBase::FB::FB() 75 0.0002836 0.0007091 0.001575 0.01% Gravity::swapTimeLevels() 8 0.001156 0.001238 0.001384 0.01% MLMG::MLResNormInf() 18 0.001255 0.001278 0.001327 0.01% MLMG::MLRhsNormInf() 18 0.00122 0.001242 0.001289 0.01% Castro::locWgtSum() 6 0.001142 0.001174 0.001203 0.01% MLMG::compResidual() 1 0.0009354 0.00107 0.001193 0.01% MLLinOp::makeAgglomeratedDMap 19 0.0009793 0.001026 0.001134 0.01% MLCellLinOp::setLevelBC() 19 0.0005487 0.0006784 0.0008542 0.01% makeSFC 95 0.0006255 0.0006603 0.0007463 0.01% Castro::swap_state_time_levels() 8 0.000594 0.0006572 0.0007174 0.01% Castro::buildMetrics() 1 0.0006323 0.0006407 0.0006739 0.00% Castro::errorEst() 2 0.0005037 0.0005674 0.000633 0.00% Castro::apply_tagging_restrictions() 2 0.0004045 0.0004676 0.0005327 0.00% Gravity::applyMetricTerms() 2 0.000477 0.0004845 0.0004958 0.00% Castro::volProductSum() 2 0.0003942 0.0004071 0.0004221 0.00% Castro::enforce_consistent_e() 1 0.0003556 0.0003657 0.0003958 0.00% Castro::finalize_advance() 1 0.0002566 0.0002707 0.0002921 0.00% DistributionMapping::Distribute() 95 0.0002195 0.0002336 0.0002664 0.00% Castro::finalize_do_advance() 17 0.0001705 0.0002017 0.0002434 0.00% OwnerMask() 2 0.0001356 0.0001507 0.000175 0.00% DistributionMapping::KnapSackDoIt() 1 9.492e-05 0.0001156 0.0001341 0.00% Castro::enforce_speed_limit() 79 0.0001105 0.0001176 0.0001294 0.00% Castro::apply_problem_tags() 2 9.151e-05 9.328e-05 9.567e-05 0.00% MultiFab::Swap() 5 7.027e-05 7.872e-05 9.416e-05 0.00% Amr::initSubcycle() 1 6.185e-05 6.392e-05 6.892e-05 0.00% knapsack() 1 5.969e-05 6.174e-05 6.467e-05 0.00% DistributionMapping::LeastUsedCPUs() 1 1.58e-05 3.649e-05 5.313e-05 0.00% Castro::construct_old_rotation_source() 20 2.958e-05 3.088e-05 3.458e-05 0.00% Castro::construct_new_rotation_source() 17 2.221e-05 2.296e-05 2.741e-05 0.00% Castro::estTimeStep() 10 1.22e-05 1.391e-05 1.558e-05 0.00% BoxList::parallelComplementIn() 4 1.334e-05 1.371e-05 1.428e-05 0.00% Castro::computeInitialDt() 2 1.046e-05 1.142e-05 1.371e-05 0.00% Gravity::set_mass_offset() 2 4.399e-06 6.097e-06 8.233e-06 0.00% Castro::gwstrain() 2 4.487e-06 5.058e-06 6.985e-06 0.00% BoxList::complementIn 4 6.051e-06 6.223e-06 6.541e-06 0.00% Castro::FluxRegCrseInit 1 1.647e-06 1.934e-06 2.309e-06 0.00% Castro::FluxRegFineAdd() 1 1.384e-06 1.578e-06 1.848e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::castro.dtnuc_mode(nvals = 1) :: [1] [TOP]::mg.v(nvals = 1) :: [0] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 16 16 12 B 14 B 20 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (23.06-29-g0236a3732cc8) finalized