Initializing AMReX (24.01-1-gf1ec8df75c56)... MPI initialized with 4 MPI processes MPI initialized with thread support level 3 OMP initialized with 4 OMP threads AMReX (24.01-1-gf1ec8df75c56) initialized Starting run at 01:38:49 UTC on 2024-01-06. Successfully read inputs file ... Castro git describe: 24.01-7-g0790f0902 AMReX git describe: 24.01-1-gf1ec8df75 Microphysics git describe: 24.01-7-gfc722142 reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Generated initial model for primary WD of mass 0.9 solar masses, central density 1.859e+07 g cm**-3, and radius 6.378e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.859e+07 g cm**-3, and radius 6.378e+08 cm. Castro::numpts_1d at level 0 is 118 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.1183193811 ...hydro-limited CFL timestep constrained at (i,j,k) = (28,31,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1183193811 Gravity::fill_multipole_BCs() time = 0.112925411 Gravity::multilevel_solve_for_new_phi() time = 0.580323427 Gravity::fill_multipole_BCs() time = 0.114490446 ... norm of composite residual at level 0 1.086122716e-10 INITIAL GRIDS Level 0 8 grids 262144 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_collision_plt00000 Write plotfile time = 0.166681924 seconds ...estimated hydro-limited timestep at level 0: 0.1183193811 ...hydro-limited CFL timestep constrained at (i,j,k) = (28,31,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1183193811 [Level 0 step 1] ADVANCE at time 0 with dt = 0.01183193811  Beginning subcycle 1 starting at time 0 with dt = 0.01183193811 Estimated number of subcycles remaining: 1 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 2231369728 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 85.77173129 on level 0 Castro::construct_old_gravity() time = 0.069744171 on level 0 Contributions to the state from old-time sources: mass added: 0 xmom added: 8.582360427e+23 ymom added: -1.813388729e+24 zmom added: -2.417851639e+24 eint added: 0 ener added: 9.696440457e+47 Castro::do_old_sources() time = 0.573645398 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.590295762e+48 ener added: 2.303659671e+42 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 5.874645388 on level 0 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.120261065 Gravity::solve_for_phi() time = 0.447620644 on level 0 Castro::construct_new_gravity() time = 0.483229952 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.51327785e+48 ener added: 2.625373387e+47 Contributions to the state from new-time sources: mass added: 0 xmom added: -3.472266263e+23 ymom added: 3.400103868e+23 zmom added: 2.172288582e+23 eint added: 0 ener added: -3.784296225e+48 Castro::do_new_sources() time = 0.559129792 on level 0 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 4.675697455e+11 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 43.8884634 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.391618989e+34 ener added: 1.998685181e+34 ...estimated hydro-limited timestep at level 0: 0.1187673879 ...hydro-limited CFL timestep constrained at (i,j,k) = (28,31,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1187673879  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.001877292917 [Level 0 step 1] Advanced 262144 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.74877907e+32 ener added: 2.750121247e+32 [STEP 1] Coarse TimeStep time: 140.1925358 [STEP 1] FAB kilobyte spread across MPI nodes: [351709 ... 351709] STEP = 1 TIME = 0.01183193811 DT = 0.01183193811 ...estimated hydro-limited timestep at level 0: 0.1187673879 ...hydro-limited CFL timestep constrained at (i,j,k) = (28,31,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1187673879 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.1187673879 ... but limiting to: 0.01301513192 = 1.1 * 0.01183193811 [Level 0 step 2] ADVANCE at time 0.01183193811 with dt = 0.01301513192  Beginning subcycle 1 starting at time 0.01183193811 with dt = 0.01301513192 Estimated number of subcycles remaining: 1 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 2.435495567e+11 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 85.64541059 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.198083749e+33 ener added: 7.397955396e+33 Castro::construct_old_gravity() time = 0.058284909 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -8.721164198e+46 ener added: 2.380398208e+46 Contributions to the state from old-time sources: mass added: 0 xmom added: 6.364070665e+23 ymom added: -1.088033238e+25 zmom added: -3.626777459e+24 eint added: 0 ener added: 1.095463814e+48 Castro::do_old_sources() time = 0.579838834 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.472161702e+48 ener added: 2.142787186e+42 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 5.990992379 on level 0 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.122858655 Gravity::solve_for_phi() time = 0.452500137 on level 0 Castro::construct_new_gravity() time = 0.48813448 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.290928202e+48 ener added: 4.386436932e+47 Contributions to the state from new-time sources: mass added: 0 xmom added: 1.18859139e+27 ymom added: 9.444732966e+22 zmom added: 3.211209208e+23 eint added: 0 ener added: -3.730425127e+48 Castro::do_new_sources() time = 0.551652792 on level 0 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: -3.125306976e+10 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 43.98137085 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.68592323e+34 ener added: 3.206658789e+34 ...estimated hydro-limited timestep at level 0: 0.1191998657 ...hydro-limited CFL timestep constrained at (i,j,k) = (27,32,32) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1191998657  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.001877349586 [Level 0 step 2] Advanced 262144 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.760936591e+32 ener added: 1.760936591e+32 [STEP 2] Coarse TimeStep time: 140.1729939 [STEP 2] FAB kilobyte spread across MPI nodes: [351709 ... 351709] STEP = 2 TIME = 0.02484707003 DT = 0.01301513192 ...estimated hydro-limited timestep at level 0: 0.1191998657 ...hydro-limited CFL timestep constrained at (i,j,k) = (27,32,32) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1191998657 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.1191998657 ... but limiting to: 0.01431664511 = 1.1 * 0.01301513192 [Level 0 step 3] ADVANCE at time 0.02484707003 with dt = 0.01431664511  Beginning subcycle 1 starting at time 0.02484707003 with dt = 0.01431664511 Estimated number of subcycles remaining: 1 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 1.436822819e+11 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 85.55648317 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 9.626078675e+33 ener added: 8.402041207e+33 Castro::construct_old_gravity() time = 0.057973142 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -9.442681417e+46 ener added: 3.66509217e+46 Contributions to the state from old-time sources: mass added: 0 xmom added: 2.614640467e+27 ymom added: -6.044629098e+23 zmom added: 2.176066475e+25 eint added: 0 ener added: 1.23707932e+48 Castro::do_old_sources() time = 0.585104962 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.67736191e+48 ener added: 4.437215404e+43 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 5.97525392 on level 0 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.130781486 Gravity::solve_for_phi() time = 0.462624782 on level 0 Castro::construct_new_gravity() time = 0.498465626 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.483376227e+48 ener added: 5.054832092e+47 Contributions to the state from new-time sources: mass added: 0 xmom added: -1.232511313e+28 ymom added: -2.502854236e+23 zmom added: 0 eint added: 0 ener added: -3.989127867e+48 Castro::do_new_sources() time = 0.583329606 on level 0 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 2.974447462e+11 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 43.53351261 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.075877425e+34 ener added: 3.081624282e+34 ...estimated hydro-limited timestep at level 0: 0.1195774577 ...hydro-limited CFL timestep constrained at (i,j,k) = (36,32,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1195774577  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.001884748542 [Level 0 step 3] Advanced 262144 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.370821947e+33 ener added: 2.370821947e+33 [STEP 3] Coarse TimeStep time: 139.6331276 [STEP 3] FAB kilobyte spread across MPI nodes: [351709 ... 351709] STEP = 3 TIME = 0.03916371514 DT = 0.01431664511 ...estimated hydro-limited timestep at level 0: 0.1195774577 ...hydro-limited CFL timestep constrained at (i,j,k) = (36,32,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1195774577 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.1195774577 ... but limiting to: 0.01574830962 = 1.1 * 0.01431664511 [Level 0 step 4] ADVANCE at time 0.03916371514 with dt = 0.01574830962  Beginning subcycle 1 starting at time 0.03916371514 with dt = 0.01574830962 Estimated number of subcycles remaining: 1 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 1.950230384e+11 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 85.51857652 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.527808664e+34 ener added: 2.306654062e+34 Castro::construct_old_gravity() time = 0.049399145 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -9.229313421e+46 ener added: 6.221775188e+46 Contributions to the state from old-time sources: mass added: 0 xmom added: -2.423724241e+28 ymom added: 5.440166188e+24 zmom added: 6.044629098e+24 eint added: 0 ener added: 1.399793572e+48 Castro::do_old_sources() time = 0.583051054 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.470034181e+48 ener added: 1.56937575e+44 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 5.981551193 on level 0 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.12464767 Gravity::solve_for_phi() time = 0.455526006 on level 0 Castro::construct_new_gravity() time = 0.495993024 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.024748253e+48 ener added: 6.067345122e+47 Contributions to the state from new-time sources: mass added: 0 xmom added: -2.461609549e+28 ymom added: -6.091852763e+23 zmom added: -7.177997054e+23 eint added: 0 ener added: -3.630738409e+48 Castro::do_new_sources() time = 0.553978579 on level 0 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: -3.576845258e+11 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 43.80970531 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.336639962e+34 ener added: 1.438259217e+34 ...estimated hydro-limited timestep at level 0: 0.1199391335 ...hydro-limited CFL timestep constrained at (i,j,k) = (36,32,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1199391335  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.00188188122 [Level 0 step 4] Advanced 262144 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 1.06901736e+34 ener added: 1.069024071e+34 [STEP 4] Coarse TimeStep time: 139.8328396 [STEP 4] FAB kilobyte spread across MPI nodes: [351709 ... 351709] STEP = 4 TIME = 0.05491202477 DT = 0.01574830962 ...estimated hydro-limited timestep at level 0: 0.1199391335 ...hydro-limited CFL timestep constrained at (i,j,k) = (36,32,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1199391335 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.1199391335 ... but limiting to: 0.01732314059 = 1.1 * 0.01574830962 [Level 0 step 5] ADVANCE at time 0.05491202477 with dt = 0.01732314059  Beginning subcycle 1 starting at time 0.05491202477 with dt = 0.01732314059 Estimated number of subcycles remaining: 1 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: 8.933916355e+10 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 85.37447235 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.30779465e+34 ener added: 3.133677461e+34 Castro::construct_old_gravity() time = 0.049221526 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -8.636206377e+46 ener added: 9.624862409e+46 Contributions to the state from old-time sources: mass added: 0 xmom added: -8.082045652e+28 ymom added: 2.417851639e+24 zmom added: -3.626777459e+24 eint added: 0 ener added: 1.57722099e+48 Castro::do_old_sources() time = 0.585209799 on level 0 ... Entering construct_ctu_hydro_source() on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 2.382578306e+48 ener added: 2.85075556e+44 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 5.976977485 on level 0 ... new-time level Poisson gravity solve at level 0 Gravity::fill_multipole_BCs() time = 0.125920221 Gravity::solve_for_phi() time = 0.457353391 on level 0 Castro::construct_new_gravity() time = 0.492886667 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: -3.095274302e+48 ener added: 6.211305833e+47 Contributions to the state from new-time sources: mass added: 0 xmom added: -4.3105426e+28 ymom added: 8.641930664e+23 zmom added: 1.086144291e+24 eint added: 0 ener added: -3.71484768e+48 Castro::do_new_sources() time = 0.55942008 on level 0 ... Entering burner on level 0 and doing half-timestep of burning. ... (rho e) added from burning: -6.829762775e+10 ... Leaving burner on level 0 after completing half-timestep of burning. Castro::react_state() time = 43.83409358 on level 0 Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 3.310920269e+34 ener added: 3.315874267e+34 ...estimated hydro-limited timestep at level 0: 0.1202880158 ...hydro-limited CFL timestep constrained at (i,j,k) = (36,32,31) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.1202880158  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.001883703442 [Level 0 step 5] Advanced 262144 cells Contributions to the state from negative energy resets: mass added: 0 xmom added: 0 ymom added: 0 zmom added: 0 eint added: 4.849018077e+33 ener added: 4.299799134e+33 [STEP 5] Coarse TimeStep time: 139.7024106 [STEP 5] FAB kilobyte spread across MPI nodes: [351709 ... 351709] STEP = 5 TIME = 0.07223516535 DT = 0.01732314059 PLOTFILE: file = wdmerger_collision_plt00005 Write plotfile time = 0.145483433 seconds Ending run at 01:50:34 UTC on 2024-01-06. Run time = 704.5161703 Run time without initialization = 699.6807442 Average number of zones advanced per microsecond: 0.002 Average number of zones advanced per microsecond per rank: 0.000 TinyProfiler total time across processes [min...avg...max]: 704.5 ... 704.5 ... 704.5 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::react_state() 10 646.6871 646.7437 646.9060 91.82% Castro::construct_ctu_hydro_source() 5 27.31 27.43 27.55 3.91% Castro::computeTemp() 43 3.866 3.878 3.888 0.55% MultiFab::AddProduct() 2055 2.75 2.901 3.341 0.47% amrex::Copy() 351 3.217 3.229 3.242 0.46% Castro::normalize_species() 42 2.662 2.692 2.755 0.39% Castro::reset_internal_energy(Fab) 344 2.371 2.376 2.379 0.34% MultiFab::Subtract() 85 1.778 1.782 1.785 0.25% FabArray::setVal() 2518 1.538 1.565 1.585 0.22% Castro::initMFs() 1 1.171 1.193 1.22 0.17% Amr::InitAmr() 1 1.203 1.205 1.205 0.17% Castro::reset_internal_energy(MultiFab) 43 0.8061 0.9837 1.161 0.16% Castro::estTimeStep() 11 0.8454 0.8965 1.001 0.14% FabArray::sum() 2065 0.9162 0.9196 0.9233 0.13% MLPoisson::Fsmooth() 1520 0.8476 0.8648 0.8843 0.13% Gravity::fill_multipole_BCs() 7 0.7676 0.7871 0.8443 0.12% Castro::enforce_min_density() 42 0.4188 0.5665 0.7476 0.11% FabArray::ParallelCopy_nowait() 249 0.496 0.4979 0.5001 0.07% FabArray::ParallelCopy_finish() 249 0.2111 0.2999 0.4723 0.07% MLPoisson::Fapply() 260 0.3099 0.3119 0.3177 0.05% check_for_negative_density() 5 0.09994 0.2037 0.3134 0.04% MLCellLinOp::applyBC() 1786 0.2409 0.2489 0.2673 0.04% Castro::construct_new_gravity_source() 5 0.2397 0.2459 0.2522 0.04% StateData::FillBoundary(geom) 21 0.245 0.2461 0.2478 0.04% FabArray::Saxpy() 82 0.2175 0.2184 0.2199 0.03% Castro::volInBoundary() 42 0.1843 0.1937 0.2072 0.03% Castro::wd_update() 6 0.1267 0.1518 0.1953 0.03% VisMF::Write(FabArray) 2 0.1853 0.1882 0.1944 0.03% FillPatchIterator::Initialize 21 0.1842 0.1846 0.1849 0.03% Castro::construct_old_gravity_source() 5 0.1344 0.1351 0.1358 0.02% FabArray::Xpay() 241 0.1194 0.1205 0.1226 0.02% MultiFab::contains_nan() 10 0.1175 0.1178 0.1182 0.02% Castro::initialize_do_advance() 5 0.1163 0.1169 0.1182 0.02% FillBoundary_finish() 1596 0.07509 0.1008 0.1133 0.02% FabArray::setDomainBndry() 21 0.1074 0.1095 0.1103 0.02% FillBoundary_nowait() 1596 0.107 0.1078 0.1089 0.02% Castro::fill_hybrid_hydro_source() 15 0.05148 0.05196 0.05248 0.01% Amr::writePlotFile() 2 0.03923 0.04727 0.05089 0.01% MLMG::addInterpCorrection() 190 0.04485 0.0455 0.0466 0.01% amrex::average_down 190 0.04009 0.04088 0.04233 0.01% Castro::initData() 1 0.03651 0.03782 0.03848 0.01% amrex::Add() 38 0.03248 0.03271 0.03312 0.00% Castro::evaluate_source_change() 95 0.03187 0.03266 0.03306 0.00% Gravity::get_new_grav_vector() 6 0.02702 0.02985 0.03228 0.00% StateData::define() 5 0.03078 0.03123 0.03141 0.00% Castro::swap_state_time_levels() 5 0.02955 0.03031 0.03069 0.00% FabArray::mult() 24 0.03031 0.03041 0.03051 0.00% FabArray::norminf() 102 0.02701 0.02756 0.0291 0.00% Castro::do_new_sources() 5 0.0002463 0.0127 0.02905 0.00% Gravity::get_old_grav_vector() 5 0.02571 0.02776 0.02889 0.00% Castro::do_old_sources() 5 0.0002922 0.01081 0.02815 0.00% MLCellLinOp::compGrad() 6 0.02257 0.02273 0.02288 0.00% MultiFab::max() 6 0.002933 0.007578 0.02038 0.00% Castro::initialize_advance() 5 0.01893 0.01938 0.01989 0.00% MLMG::prepareForSolve() 6 0.01462 0.01604 0.01926 0.00% Castro::construct_new_source() 35 0.01236 0.01374 0.01673 0.00% Castro::construct_old_source() 35 0.01122 0.01268 0.01479 0.00% AmrLevel::FillPatch() 21 0.013 0.01364 0.01429 0.00% MLCellLinOp::defineAuxData() 7 0.01309 0.01368 0.0142 0.00% Castro::finalize_do_advance() 5 0.01133 0.01169 0.01201 0.00% BndryData::define() 7 0.008786 0.008989 0.009183 0.00% MLMG::ResNormInf() 44 0.003913 0.006048 0.008352 0.00% Castro::linear_to_hybrid_momentum() 1 0.004739 0.005287 0.005494 0.00% Castro::Castro() 1 0.004521 0.004684 0.004845 0.00% Castro::construct_new_gravity() 5 0.0001997 0.001842 0.004419 0.00% Castro::buildMetrics() 1 0.004255 0.004298 0.004321 0.00% FabArray::FillBoundary() 1596 0.003852 0.003973 0.004185 0.00% FabArray::setVal(val, thecmd, scomp, ncomp) 294 0.003637 0.003731 0.003837 0.00% MLCellLinOp::smooth() 760 0.003463 0.003617 0.003789 0.00% MLCellLinOp::prepareForSolve() 7 0.003393 0.003476 0.00367 0.00% Castro::enforce_consistent_e() 1 0.003017 0.003292 0.003555 0.00% StateDataPhysBCFunct::() 21 0.0007447 0.001728 0.003555 0.00% Gravity::actual_solve_with_mlmg() 7 0.003149 0.003271 0.003469 0.00% MLCGSolver::bicgstab 9 0 0.0008278 0.003311 0.00% FabArrayBase::getFB() 1596 0.002911 0.003039 0.003228 0.00% FabArrayBase::getCPC() 543 0.002706 0.002766 0.002797 0.00% FabArrayBase::CPC::define() 188 0.001478 0.001831 0.002768 0.00% MLCellLinOp::setLevelBC() 7 0.002112 0.002137 0.002152 0.00% DistributionMapping::LeastUsedCPUs() 1 1.794e-05 0.000785 0.002128 0.00% MLLinOp::makeSubCommunicator() 7 0.0002484 0.001297 0.001902 0.00% Gravity::update_max_rhs() 6 0.001762 0.001792 0.001848 0.00% Gravity::solve_for_phi() 5 0.001345 0.001491 0.00174 0.00% amrex::Dot() 51 0 0.0003244 0.001298 0.00% MLMG::solve() 6 0.0004403 0.0008548 0.00128 0.00% Castro::construct_old_gravity() 5 0.0001832 0.0004421 0.0012 0.00% main() 1 0.0009424 0.0009906 0.00106 0.00% MLCellLinOp::apply() 260 0.0006476 0.00073 0.0009424 0.00% Gravity::multilevel_solve_for_new_phi() 1 0.0008707 0.0008947 0.0009266 0.00% MLLinOp::defineGrids() 7 0.0007084 0.0007665 0.0008465 0.00% Gravity::test_composite_phi() 1 0.0007488 0.0007857 0.0008342 0.00% FabArray::ParallelCopy() 249 0.0007171 0.0007512 0.0007958 0.00% Amr::coarseTimeStep() 5 0.0006981 0.0007035 0.000712 0.00% FabArrayBase::FB::FB() 16 0.0004199 0.0004693 0.000602 0.00% MLCGSolver::ParallelAllReduce 92 0 0.0001495 0.000598 0.00% MLCellLinOp::correctionResidual() 190 0.0005497 0.0005671 0.0005943 0.00% MLCellLinOp::defineBC() 7 0.0005325 0.000554 0.0005834 0.00% MLMG::mgVcycle() 38 0.0005131 0.0005279 0.0005399 0.00% Castro::clean_state() 42 0.0004712 0.0004889 0.0005005 0.00% Castro::subcycle_advance_ctu() 5 8.513e-05 0.000187 0.0004672 0.00% MLMG:computeResOfCorrection() 190 0.0004115 0.0004238 0.0004373 0.00% MLMG::actualBottomSolve() 38 4.271e-05 0.0001202 0.0003493 0.00% Castro::do_advance_ctu() 5 0.0003071 0.0003137 0.0003252 0.00% Gravity::actual_multilevel_solve() 1 0.0002567 0.0002695 0.0002847 0.00% MLMG::mgVcycle_down::0 38 0.0002514 0.0002607 0.0002729 0.00% FillPatchIterator::FillFromLevel0() 21 0.0002156 0.0002396 0.0002625 0.00% MLMG::mgVcycle_down::1 38 0.0002337 0.0002413 0.0002602 0.00% Amr::defBaseLevel() 1 0.0001898 0.0002056 0.0002515 0.00% MLMG::mgVcycle_down::2 38 0.0002208 0.0002263 0.0002382 0.00% Castro::finalize_advance() 5 0.000199 0.0002156 0.0002275 0.00% MLMG::mgVcycle_down::3 38 0.0001956 0.0002097 0.0002224 0.00% MLMG::mgVcycle_down::4 38 0.0001823 0.0001933 0.0002093 0.00% MLCellLinOp::solutionResidual() 45 0.0001736 0.0001776 0.0001843 0.00% MLMG::mgVcycle_up::0 38 0.0001679 0.0001718 0.0001749 0.00% MLMG::mgVcycle_up::1 38 0.0001562 0.0001597 0.0001654 0.00% MLMG::mgVcycle_up::4 38 0.0001282 0.0001398 0.0001563 0.00% MLMG::mgVcycle_up::2 38 0.0001496 0.0001522 0.0001556 0.00% Castro::computeNewDt() 4 2.869e-05 6.158e-05 0.0001532 0.00% MLMG::oneIter() 38 0.0001342 0.0001395 0.000146 0.00% FillPatchSingleLevel 21 0.0001323 0.0001372 0.0001458 0.00% MLMG::mgVcycle_up::3 38 0.000125 0.0001324 0.0001444 0.00% Amr::timeStep() 5 8.777e-05 0.0001106 0.0001413 0.00% Gravity::solve_phi_with_mlmg() 7 0.0001182 0.0001261 0.0001301 0.00% Amr::FinalizeInit() 1 2.05e-05 4.843e-05 0.0001181 0.00% Amr::writeSmallPlotFile() 1 9.11e-05 0.0001013 0.0001168 0.00% Castro::advance() 5 8.419e-05 8.994e-05 0.0001042 0.00% Castro::enforce_speed_limit() 42 8.433e-05 9.031e-05 9.843e-05 0.00% makeSFC 21 8.483e-05 8.97e-05 9.756e-05 0.00% MLMG::computeResidual() 38 8.472e-05 8.749e-05 9.435e-05 0.00% Castro::problem_post_timestep() 6 5.407e-05 6.462e-05 7.978e-05 0.00% MLMG::mgVcycle_bottom 38 6.991e-05 7.274e-05 7.533e-05 0.00% Castro::post_timestep() 5 3.976e-05 5.055e-05 7.455e-05 0.00% MLLinOp::makeAgglomeratedDMap 7 6.427e-05 6.611e-05 6.844e-05 0.00% MLPoisson::define() 7 5.714e-05 6.177e-05 6.81e-05 0.00% Castro::check_for_nan() 10 6.142e-05 6.334e-05 6.56e-05 0.00% Castro::expand_state() 5 3.916e-05 4.547e-05 5.386e-05 0.00% AmrLevel::AmrLevel(dm) 1 4.165e-05 4.794e-05 5.099e-05 0.00% Castro::apply_source_to_state() 10 4.298e-05 4.755e-05 5.041e-05 0.00% Gravity::set_mass_offset() 6 2.811e-05 3.852e-05 4.926e-05 0.00% Amr::initSubcycle() 1 4.364e-05 4.635e-05 4.924e-05 0.00% DistributionMapping::Distribute() 22 4.567e-05 4.676e-05 4.798e-05 0.00% Gravity::swapTimeLevels() 5 3.965e-05 4.22e-05 4.77e-05 0.00% MLLinOp::define() 7 3.298e-05 3.654e-05 4.444e-05 0.00% Castro::create_source_corrector() 5 2.705e-05 3.259e-05 3.763e-05 0.00% Castro::post_init() 1 2.324e-05 2.933e-05 3.564e-05 0.00% MLPoisson::prepareForSolve() 7 2.878e-05 3.008e-05 3.161e-05 0.00% Castro::retry_advance_ctu() 5 2.533e-05 2.791e-05 3.061e-05 0.00% MLMG::compResidual() 1 2.37e-05 2.404e-05 2.443e-05 0.00% Castro::construct_new_hybrid_source() 5 2.071e-05 2.28e-05 2.412e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 1.553e-05 1.918e-05 2.157e-05 0.00% MLMG::computeMLResidual() 6 1.726e-05 1.848e-05 1.983e-05 0.00% Castro::construct_old_hybrid_source() 5 1.596e-05 1.694e-05 1.864e-05 0.00% MLMG::getGradSolution() 6 1.736e-05 1.784e-05 1.851e-05 0.00% Castro::post_regrid() 1 1.497e-05 1.651e-05 1.766e-05 0.00% MLMG::MLResNormInf() 6 1.413e-05 1.477e-05 1.536e-05 0.00% Castro::construct_old_rotation_source() 5 1.186e-05 1.257e-05 1.382e-05 0.00% Castro::construct_new_rotation_source() 5 1.177e-05 1.282e-05 1.369e-05 0.00% Castro::computeInitialDt() 2 1.191e-05 1.277e-05 1.364e-05 0.00% MLMG::MLRhsNormInf() 6 1.216e-05 1.263e-05 1.35e-05 0.00% Amr::InitializeInit() 1 7.744e-06 9.994e-06 1.243e-05 0.00% Castro::FluxRegCrseInit 5 1.049e-05 1.107e-05 1.217e-05 0.00% Castro::FluxRegFineAdd() 5 6.313e-06 7.093e-06 8.084e-06 0.00% Amr::init() 1 5.648e-06 5.877e-06 6.133e-06 0.00% Amr::initialInit() 1 3.412e-06 3.976e-06 4.916e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 704.5 704.5 704.5 100.00% Amr::coarseTimeStep() 5 699.5 699.5 699.5 99.29% Amr::timeStep() 5 699.1 699.1 699.1 99.24% Castro::advance() 5 696.8 696.8 696.8 98.91% Castro::subcycle_advance_ctu() 5 694.4 694.4 694.5 98.58% Castro::do_advance_ctu() 5 694.4 694.4 694.5 98.58% Castro::react_state() 10 646.7 646.8 646.9 91.82% Castro::construct_ctu_hydro_source() 5 29.75 29.79 29.8 4.23% Castro::clean_state() 42 18.65 19.16 19.35 2.75% Castro::computeTemp() 43 11.95 12.2 12.31 1.75% Castro::reset_internal_energy(MultiFab) 43 8.068 8.324 8.446 1.20% Castro::evaluate_source_change() 95 4.682 4.835 5.272 0.75% Castro::enforce_min_density() 42 4.333 4.566 4.68 0.66% Castro::initialize_do_advance() 5 4.034 4.051 4.059 0.58% Amr::init() 1 3.63 3.63 3.632 0.52% Amr::initialInit() 1 3.462 3.463 3.465 0.49% MultiFab::AddProduct() 2055 2.75 2.901 3.341 0.47% amrex::Copy() 351 3.217 3.229 3.242 0.46% Gravity::solve_phi_with_mlmg() 7 2.897 2.917 2.975 0.42% Castro::do_old_sources() 5 2.88 2.898 2.907 0.41% Castro::do_new_sources() 5 2.673 2.769 2.808 0.40% Castro::normalize_species() 42 2.662 2.692 2.755 0.39% Castro::construct_new_gravity() 5 2.38 2.404 2.459 0.35% Castro::initialize_advance() 5 2.314 2.361 2.381 0.34% Castro::reset_internal_energy(Fab) 344 2.371 2.376 2.379 0.34% Castro::post_timestep() 5 2.32 2.323 2.324 0.33% Gravity::solve_for_phi() 5 2.198 2.218 2.276 0.32% Amr::InitializeInit() 1 2.094 2.118 2.127 0.30% Amr::defBaseLevel() 1 2.094 2.118 2.127 0.30% Gravity::actual_solve_with_mlmg() 7 2.119 2.12 2.121 0.30% MLMG::solve() 6 2.047 2.048 2.049 0.29% MultiFab::Subtract() 85 1.778 1.782 1.785 0.25% AmrLevel::FillPatch() 21 1.541 1.605 1.757 0.25% MLMG::oneIter() 38 1.712 1.714 1.718 0.24% MLMG::mgVcycle() 38 1.679 1.681 1.686 0.24% FabArray::setVal() 2518 1.538 1.565 1.585 0.22% FillPatchIterator::Initialize 21 1.213 1.276 1.427 0.20% Amr::FinalizeInit() 1 1.337 1.345 1.369 0.19% Castro::Castro() 1 1.265 1.272 1.289 0.18% MLCellLinOp::smooth() 760 1.214 1.23 1.265 0.18% Castro::initMFs() 1 1.173 1.195 1.221 0.17% Amr::InitAmr() 1 1.203 1.205 1.205 0.17% FillPatchIterator::FillFromLevel0() 21 0.918 0.9818 1.135 0.16% FillPatchSingleLevel 21 0.9178 0.9815 1.135 0.16% Castro::expand_state() 5 0.968 0.9885 1.007 0.14% Castro::estTimeStep() 11 0.8454 0.8965 1.001 0.14% FabArray::ParallelCopy() 249 0.7143 0.8004 0.9733 0.14% FabArray::sum() 2065 0.9162 0.9196 0.9233 0.13% MLPoisson::Fsmooth() 1520 0.8476 0.8648 0.8843 0.13% Gravity::fill_multipole_BCs() 7 0.7737 0.7932 0.8505 0.12% Castro::post_init() 1 0.8264 0.8309 0.8436 0.12% Castro::initData() 1 0.7965 0.8134 0.8286 0.12% MLMG::mgVcycle_down::0 38 0.7452 0.7471 0.7496 0.11% Gravity::multilevel_solve_for_new_phi() 1 0.5802 0.5803 0.5804 0.08% Gravity::actual_multilevel_solve() 1 0.5793 0.5794 0.5795 0.08% MLMG::mgVcycle_up::0 38 0.5272 0.5287 0.5321 0.08% Castro::finalize_do_advance() 5 0.3918 0.435 0.5147 0.07% FabArray::ParallelCopy_nowait() 249 0.4977 0.4998 0.5024 0.07% MLCellLinOp::applyBC() 1786 0.4345 0.465 0.4921 0.07% FabArray::ParallelCopy_finish() 249 0.2111 0.2999 0.4723 0.07% Castro::construct_new_source() 35 0.42 0.4284 0.4396 0.06% Castro::post_regrid() 1 0.4143 0.427 0.4315 0.06% MLCellLinOp::apply() 260 0.4027 0.4104 0.4214 0.06% Castro::problem_post_timestep() 6 0.3247 0.3456 0.4026 0.06% Castro::wd_update() 6 0.3246 0.3455 0.4025 0.06% MLPoisson::Fapply() 260 0.3099 0.3119 0.3177 0.05% check_for_negative_density() 5 0.09994 0.2037 0.3134 0.04% Amr::writePlotFile() 2 0.3126 0.3127 0.3127 0.04% Castro::computeNewDt() 4 0.3053 0.3053 0.3054 0.04% Castro::construct_old_gravity() 5 0.1742 0.2049 0.2848 0.04% Gravity::get_old_grav_vector() 5 0.174 0.2044 0.2846 0.04% MLMG:computeResOfCorrection() 190 0.2591 0.2644 0.273 0.04% MLCellLinOp::correctionResidual() 190 0.2587 0.264 0.2726 0.04% MLCellLinOp::solutionResidual() 45 0.2648 0.2666 0.2684 0.04% Castro::construct_new_gravity_source() 5 0.2397 0.2459 0.2522 0.04% StateDataPhysBCFunct::() 21 0.2469 0.2478 0.2486 0.04% StateData::FillBoundary(geom) 21 0.245 0.2461 0.2478 0.04% Castro::construct_old_source() 35 0.2305 0.2324 0.2345 0.03% FabArray::FillBoundary() 1596 0.19 0.2161 0.2274 0.03% MLMG::computeResidual() 38 0.2198 0.2233 0.2255 0.03% Gravity::get_new_grav_vector() 6 0.2127 0.2179 0.2213 0.03% FabArray::Saxpy() 82 0.2175 0.2184 0.2199 0.03% Castro::volInBoundary() 42 0.1843 0.1937 0.2072 0.03% VisMF::Write(FabArray) 2 0.1853 0.1882 0.1944 0.03% Castro::computeInitialDt() 2 0.1598 0.1679 0.1919 0.03% Castro::construct_old_gravity_source() 5 0.1344 0.1351 0.1358 0.02% MLMG::mgVcycle_down::1 38 0.1279 0.1298 0.1319 0.02% Gravity::test_composite_phi() 1 0.131 0.131 0.1311 0.02% MLMG::addInterpCorrection() 190 0.04998 0.1098 0.1302 0.02% FabArray::Xpay() 241 0.1194 0.1205 0.1226 0.02% Castro::check_for_nan() 10 0.1176 0.1178 0.1183 0.02% MultiFab::contains_nan() 10 0.1175 0.1178 0.1182 0.02% MLMG::mgVcycle_up::2 38 0.03415 0.09568 0.1167 0.02% FillBoundary_finish() 1596 0.07509 0.1008 0.1133 0.02% FillBoundary_nowait() 1596 0.1103 0.1113 0.1127 0.02% FabArray::setDomainBndry() 21 0.1074 0.1095 0.1103 0.02% MLMG::mgVcycle_up::1 38 0.09067 0.09092 0.09158 0.01% Castro::apply_source_to_state() 10 0.08708 0.08756 0.08824 0.01% Castro::fill_hybrid_hydro_source() 15 0.05148 0.05196 0.05248 0.01% amrex::average_down 190 0.04212 0.04347 0.04651 0.01% MLMG::mgVcycle_down::2 38 0.04375 0.04436 0.0455 0.01% MLMG::prepareForSolve() 6 0.03858 0.04013 0.04306 0.01% MLMG::computeMLResidual() 6 0.03547 0.03722 0.03879 0.01% MLMG::mgVcycle_down::3 38 0.006459 0.01456 0.03837 0.01% Castro::construct_new_hybrid_source() 5 0.03432 0.0347 0.03546 0.01% amrex::Add() 38 0.03248 0.03271 0.03312 0.00% MLPoisson::define() 7 0.03177 0.032 0.03242 0.00% AmrLevel::AmrLevel(dm) 1 0.03083 0.03128 0.03145 0.00% MLMG::ResNormInf() 44 0.02815 0.02947 0.03141 0.00% StateData::define() 5 0.03078 0.03123 0.03141 0.00% Castro::swap_state_time_levels() 5 0.02955 0.03031 0.03069 0.00% FabArray::mult() 24 0.03031 0.03041 0.03051 0.00% Gravity::update_max_rhs() 6 0.01297 0.01747 0.03019 0.00% FabArray::norminf() 102 0.02701 0.02756 0.0291 0.00% MLMG::getGradSolution() 6 0.02815 0.02831 0.02846 0.00% MLCellLinOp::compGrad() 6 0.02813 0.02829 0.02844 0.00% MLMG::mgVcycle_up::3 38 0.005203 0.01073 0.02697 0.00% MultiFab::max() 6 0.002933 0.007578 0.02038 0.00% MLCellLinOp::defineAuxData() 7 0.01666 0.01754 0.0187 0.00% Castro::finalize_advance() 5 0.01445 0.01543 0.01754 0.00% Castro::construct_old_hybrid_source() 5 0.01706 0.0173 0.01753 0.00% MLMG::mgVcycle_down::4 38 0.00628 0.008306 0.01404 0.00% MLMG::mgVcycle_bottom 38 0.0001126 0.003585 0.01399 0.00% MLMG::actualBottomSolve() 38 4.271e-05 0.003512 0.01392 0.00% MLCGSolver::bicgstab 9 0 0.003309 0.01323 0.00% MLCellLinOp::defineBC() 7 0.01193 0.0121 0.0123 0.00% BndryData::define() 7 0.01135 0.01155 0.01176 0.00% MLMG::mgVcycle_up::4 38 0.005231 0.006789 0.01112 0.00% Gravity::swapTimeLevels() 5 0.009461 0.009773 0.01053 0.00% Castro::buildMetrics() 1 0.009147 0.009777 0.01034 0.00% MLMG::compResidual() 1 0.0066 0.006778 0.006903 0.00% FabArrayBase::getCPC() 543 0.004232 0.004597 0.005553 0.00% Castro::linear_to_hybrid_momentum() 1 0.004739 0.005287 0.005494 0.00% FabArray::setVal(val, thecmd, scomp, ncomp) 294 0.003637 0.003731 0.003837 0.00% FabArrayBase::getFB() 1596 0.003331 0.003508 0.00383 0.00% MLPoisson::prepareForSolve() 7 0.003424 0.003506 0.003698 0.00% MLCellLinOp::prepareForSolve() 7 0.003393 0.003476 0.00367 0.00% Castro::enforce_consistent_e() 1 0.003017 0.003292 0.003555 0.00% MLMG::MLResNormInf() 6 0.003163 0.003178 0.003197 0.00% MLMG::MLRhsNormInf() 6 0.00314 0.003162 0.003186 0.00% MLLinOp::define() 7 0.001191 0.002297 0.002992 0.00% MLLinOp::defineGrids() 7 0.001147 0.002261 0.002957 0.00% FabArrayBase::CPC::define() 188 0.001478 0.001831 0.002768 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 3.803e-05 0.0008092 0.002154 0.00% MLCellLinOp::setLevelBC() 7 0.002112 0.002137 0.002152 0.00% DistributionMapping::LeastUsedCPUs() 1 1.794e-05 0.000785 0.002128 0.00% MLLinOp::makeSubCommunicator() 7 0.0002484 0.001297 0.001902 0.00% MLCGSolver::ParallelAllReduce 92 0 0.0003558 0.001423 0.00% amrex::Dot() 51 0 0.0003244 0.001298 0.00% FabArrayBase::FB::FB() 16 0.0004199 0.0004693 0.000602 0.00% MLLinOp::makeAgglomeratedDMap 7 0.0001901 0.0001975 0.0002084 0.00% makeSFC 21 0.0001259 0.0001314 0.0001399 0.00% Amr::writeSmallPlotFile() 1 9.11e-05 0.0001013 0.0001168 0.00% Castro::enforce_speed_limit() 42 8.433e-05 9.031e-05 9.843e-05 0.00% Gravity::set_mass_offset() 6 2.811e-05 3.852e-05 4.926e-05 0.00% Amr::initSubcycle() 1 4.364e-05 4.635e-05 4.924e-05 0.00% DistributionMapping::Distribute() 22 4.567e-05 4.676e-05 4.798e-05 0.00% Castro::create_source_corrector() 5 2.705e-05 3.259e-05 3.763e-05 0.00% Castro::retry_advance_ctu() 5 2.533e-05 2.791e-05 3.061e-05 0.00% Castro::construct_old_rotation_source() 5 1.186e-05 1.257e-05 1.382e-05 0.00% Castro::construct_new_rotation_source() 5 1.177e-05 1.282e-05 1.369e-05 0.00% Castro::FluxRegCrseInit 5 1.049e-05 1.107e-05 1.217e-05 0.00% Castro::FluxRegFineAdd() 5 6.313e-06 7.093e-06 8.084e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 4 4 0 B 0 B 0 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (24.01-1-gf1ec8df75c56) finalized