Initializing AMReX (v2024-33-g9e71c2cb9714)... Initializing CUDA... CUDA initialized with 1 device. AMReX (v2024-33-g9e71c2cb9714) initialized Starting run at 08:52:24 UTC on 2024-03-25. Successfully read inputs file ... Castro git describe: 24.03-26-g5707c0c2f AMReX git describe: v2024-33-g9e71c2cb9 Microphysics git describe: 24.03-21-g81de2adb reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.003332832 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.001812464 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.008921913 Gravity::multilevel_solve_for_new_phi() time = 0.053016309 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.149856957 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003284028 on level 0 Castro::do_old_sources() time = 0.008842821 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.134305618 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.006382974 Gravity::multilevel_solve_for_new_phi() time = 0.040377171 Castro::construct_new_gravity() time = 0.043936885 on level 0 Castro::do_new_sources() time = 0.007337609 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 4.024731856 [Level 0 step 1] Advanced 884736 cells Updating the rotational period from 30.54368673 s to 30.46017574 s. [STEP 1] Coarse TimeStep time: 0.229122762 [STEP 1] FAB kilobyte spread across MPI nodes: [2005477 ... 2005477] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.002176152 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.00717449 Gravity::multilevel_solve_for_new_phi() time = 0.060286613 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135647e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (104,88,99) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135647e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135647e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003427183 on level 0 Castro::do_old_sources() time = 0.006637953 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.161953138 on level 0 Castro::construct_old_gravity() time = 0.000410575 on level 1 Castro::do_old_sources() time = 0.001718896 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.010981084 on level 1 Castro::reflux() at level 0 : time = 0.018599812 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006954867 Gravity::multilevel_solve_for_new_phi() time = 0.041217431 Castro::construct_new_gravity() time = 0.044829191 on level 0 Castro::do_new_sources() time = 0.006251885 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461716053 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716053 Castro::construct_new_gravity() time = 0.000440815 on level 1 Castro::do_new_sources() time = 0.002711946 on level 1 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.349012694 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Updating the rotational period from 30.46017574 s to 30.52759479 s. [STEP 2] Coarse TimeStep time: 0.368294863 [STEP 2] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 2 TIME = 0.005426075163 DT = 2.526135647e-09 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008044208529 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.697001619e-07 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Now regridding at level lbase = 0 grid_places() time: 0.00228955 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.006959252 Gravity::multilevel_solve_for_new_phi() time = 0.015040553 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778749212e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003412498 on level 0 Castro::do_old_sources() time = 0.006742662 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.142110916 on level 0 Castro::construct_old_gravity() time = 0.00042799 on level 1 Castro::do_old_sources() time = 0.001441044 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.011779883 on level 1 Castro::reflux() at level 0 : time = 0.019732253 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006972221 Gravity::multilevel_solve_for_new_phi() time = 0.028794619 Castro::construct_new_gravity() time = 0.032289364 on level 0 Castro::do_new_sources() time = 0.01052055 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008044237461 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044237461 Castro::construct_new_gravity() time = 0.000446164 on level 1 Castro::do_new_sources() time = 0.002351772 on level 1 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.649468538 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Updating the rotational period from 30.52759479 s to 30.52761298 s. [STEP 3] Coarse TimeStep time: 0.302859499 [STEP 3] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 3 TIME = 0.005426077942 DT = 2.778749212e-09 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.000804308035 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.965184638e-07 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Now regridding at level lbase = 0 grid_places() time: 0.003492655 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.006983101 Gravity::multilevel_solve_for_new_phi() time = 0.015119705 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056624133e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003516681 on level 0 Castro::do_old_sources() time = 0.006786626 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.118237903 on level 0 Castro::construct_old_gravity() time = 0.000418228 on level 1 Castro::do_old_sources() time = 0.001485374 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.010609453 on level 1 Castro::reflux() at level 0 : time = 0.017821337 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.00694381 Gravity::multilevel_solve_for_new_phi() time = 0.028867859 Castro::construct_new_gravity() time = 0.032283293 on level 0 Castro::do_new_sources() time = 0.006163036 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008043112215 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008043112215 Castro::construct_new_gravity() time = 0.00045104 on level 1 Castro::do_new_sources() time = 0.002617417 on level 1 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.981764048 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Updating the rotational period from 30.52761298 s to 30.52761297 s. [STEP 4] Coarse TimeStep time: 0.28693671 [STEP 4] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 4 TIME = 0.005426080998 DT = 3.056624133e-09 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002497928903 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 7.255166705e-07 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Now regridding at level lbase = 0 grid_places() time: 0.002171325 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.006907012 Gravity::multilevel_solve_for_new_phi() time = 0.015059376 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286546e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003447745 on level 0 Castro::do_old_sources() time = 0.00655256 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.168115099 on level 0 Castro::construct_old_gravity() time = 0.0003863 on level 1 Castro::do_old_sources() time = 0.001770849 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.0079893 on level 1 Castro::reflux() at level 0 : time = 0.018520319 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006948438 Gravity::multilevel_solve_for_new_phi() time = 0.029022558 Castro::construct_new_gravity() time = 0.032500358 on level 0 Castro::do_new_sources() time = 0.006486539 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006805523364 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006805523364 Castro::construct_new_gravity() time = 0.000432259 on level 1 Castro::do_new_sources() time = 0.003303883 on level 1 ...estimated hydro-limited timestep at level 1: 7.553385904e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.553385904e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.4682424 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Updating the rotational period from 30.52761297 s to 30.52761296 s. [STEP 5] Coarse TimeStep time: 0.317622142 [STEP 5] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 5 TIME = 0.005426084361 DT = 3.362286546e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.161419154 seconds Ending run at 08:52:28 UTC on 2024-03-25. Run time = 3.119860071 Run time without initialization = 1.66705555 Average number of zones advanced per microsecond: 2.968 Average number of zones advanced per microsecond per rank: 2.968 CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9476456448 TinyProfiler total time across processes [min...avg...max]: 3.12 ... 3.12 ... 3.12 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Amr::InitAmr() 1 1.0642 1.0642 1.0642 34.11% Castro::construct_ctu_hydro_source() 9 0.732 0.732 0.732 23.46% VisMF::Write(FabArray) 3 0.3041 0.3041 0.3041 9.75% Castro::initMFs() 5 0.1526 0.1526 0.1526 4.89% Castro::computeTemp() 88 0.07557 0.07557 0.07557 2.42% Castro::normalize_species() 87 0.07478 0.07478 0.07478 2.40% Gravity::fill_multipole_BCs() 10 0.06913 0.06913 0.06913 2.22% MLCellLinOp::applyBC() 1917 0.04755 0.04755 0.04755 1.52% FillBoundary_nowait() 1728 0.04315 0.04315 0.04315 1.38% FabArray::setVal() 920 0.03828 0.03828 0.03828 1.23% StateData::FillBoundary(geom) 650 0.03511 0.03511 0.03511 1.13% FabArray::ParallelCopy_nowait() 954 0.03047 0.03047 0.03047 0.98% Castro::volInBoundary() 42 0.0299 0.0299 0.0299 0.96% amrex::Copy() 441 0.02669 0.02669 0.02669 0.86% Castro::reset_internal_energy(MultiFab) 88 0.02254 0.02254 0.02254 0.72% FluxRegister::Reflux() 24 0.02249 0.02249 0.02249 0.72% MLCellLinOp::defineAuxData() 10 0.02157 0.02157 0.02157 0.69% Castro::estTimeStep() 26 0.01748 0.01748 0.01748 0.56% Castro::enforce_min_density() 87 0.01737 0.01737 0.01737 0.56% amrex::Add() 136 0.01728 0.01728 0.01728 0.55% FabArray::Saxpy() 604 0.01593 0.01593 0.01593 0.51% StateDataPhysBCFunct::() 95 0.01451 0.01451 0.01451 0.46% Castro::reflux() 4 0.01381 0.01381 0.01381 0.44% BndryData::define() 26 0.0134 0.0134 0.0134 0.43% MLPoisson::Fsmooth() 1376 0.01258 0.01258 0.01258 0.40% Castro::problem_post_timestep() 10 0.01139 0.01139 0.01139 0.36% Castro::construct_new_source() 63 0.01139 0.01139 0.01139 0.36% MLCellLinOp::reflux() 30 0.009705 0.009705 0.009705 0.31% amrex::Dot() 466 0.009153 0.009153 0.009153 0.29% Castro::construct_old_gravity_source() 18 0.008784 0.008784 0.008784 0.28% FabArray::norminf() 439 0.008722 0.008722 0.008722 0.28% Castro::FluxRegCrseInit 9 0.007945 0.007945 0.007945 0.25% MLPoisson::Fapply() 493 0.00685 0.00685 0.00685 0.22% FabArray::Xpay() 353 0.006458 0.006458 0.006458 0.21% Castro::initData() 1 0.006362 0.006362 0.006362 0.20% amrex::average_down 233 0.006112 0.006112 0.006112 0.20% Castro::reset_internal_energy(Fab) 1476 0.00585 0.00585 0.00585 0.19% Castro::construct_new_gravity_source() 9 0.005748 0.005748 0.005748 0.18% AmrMesh::MakeNewGrids() 6 0.00553 0.00553 0.00553 0.18% Amr::writePlotFile() 2 0.004824 0.004824 0.004824 0.15% MLMG::addInterpCorrection() 155 0.004464 0.004464 0.004464 0.14% Castro::wd_update() 6 0.004181 0.004181 0.004181 0.13% Gravity::actual_solve_with_mlmg() 10 0.003928 0.003928 0.003928 0.13% FabArrayBase::CPC::define() 534 0.003763 0.003763 0.003763 0.12% FillPatchInterp(MF) 30 0.003663 0.003663 0.003663 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.003627 0.003627 0.003627 0.12% Castro::apply_tagging_restrictions() 6 0.003573 0.003573 0.003573 0.11% FabArray::setDomainBndry() 92 0.003316 0.003316 0.003316 0.11% MultiFab::contains_nan() 18 0.003252 0.003252 0.003252 0.10% Castro::construct_old_source() 63 0.003212 0.003212 0.003212 0.10% MLCellLinOp::compGrad() 18 0.002888 0.002888 0.002888 0.09% MLCellLinOp::defineBC() 10 0.002744 0.002744 0.002744 0.09% FabArray::mult() 55 0.002578 0.002578 0.002578 0.08% MLCellLinOp::setLevelBC() 18 0.002512 0.002512 0.002512 0.08% Castro::enforce_speed_limit() 87 0.002476 0.002476 0.002476 0.08% check_for_negative_density() 9 0.00238 0.00238 0.00238 0.08% MLCellLinOp::prepareForSolve() 10 0.002 0.002 0.002 0.06% MLMG::prepareForSolve() 10 0.001868 0.001868 0.001868 0.06% Castro::fill_hybrid_hydro_source() 27 0.001691 0.001691 0.001691 0.05% MLPoisson::FFlux() 150 0.001488 0.001488 0.001488 0.05% Gravity::get_new_grav_vector() 10 0.001472 0.001472 0.001472 0.05% TagBoxArray::mapPRD 6 0.001395 0.001395 0.001395 0.04% MLCellLinOp::updateSolBC() 25 0.001394 0.001394 0.001394 0.04% FabArray::norminf(mask) 21 0.001229 0.001229 0.001229 0.04% Gravity::get_old_grav_vector() 9 0.001212 0.001212 0.001212 0.04% Castro::clean_state() 87 0.001136 0.001136 0.001136 0.04% FabArrayBase::getCPC() 1522 0.001113 0.001113 0.001113 0.04% Castro::do_advance_ctu() 5 0.001055 0.001055 0.001055 0.03% OwnerMask() 6 0.001045 0.001045 0.001045 0.03% Castro::build_fine_mask() 44 0.001036 0.001036 0.001036 0.03% MLCellLinOp::updateCorBC() 17 0.001028 0.001028 0.001028 0.03% average_down_faces 48 0.0008729 0.0008729 0.0008729 0.03% TagBoxArray::collate() 6 0.0008073 0.0008073 0.0008073 0.03% Gravity::multilevel_solve_for_new_phi() 10 0.0007991 0.0007991 0.0007991 0.03% FillPatchTwoLevels 31 0.0007649 0.0007649 0.0007649 0.02% main() 1 0.0007129 0.0007129 0.0007129 0.02% MLCGSolver::bicgstab 31 0.0007046 0.0007046 0.0007046 0.02% Castro::FluxRegFineAdd() 9 0.0006633 0.0006633 0.0006633 0.02% AmrLevel::FillPatch() 62 0.0006457 0.0006457 0.0006457 0.02% MultiFab::max() 18 0.000619 0.000619 0.000619 0.02% FillPatchIterator::Initialize 62 0.000574 0.000574 0.000574 0.02% Castro::apply_problem_tags() 6 0.0005227 0.0005227 0.0005227 0.02% MultiFab::Multiply() 8 0.0005063 0.0005063 0.0005063 0.02% MLCellLinOp::smooth() 688 0.0004994 0.0004994 0.0004994 0.02% FabArrayBase::FB::FB() 37 0.0004364 0.0004364 0.0004364 0.01% Gravity::actual_multilevel_solve() 10 0.0004206 0.0004206 0.0004206 0.01% FabArray::FillBoundary() 1728 0.0004106 0.0004106 0.0004106 0.01% MLMG::interpCorrection_1 17 0.0004045 0.0004045 0.0004045 0.01% Castro::initialize_advance() 9 0.0003833 0.0003833 0.0003833 0.01% Amr::regrid() 5 0.00038 0.00038 0.00038 0.01% Castro::linear_to_hybrid_momentum() 1 0.0003694 0.0003694 0.0003694 0.01% Castro::Castro() 5 0.0003426 0.0003426 0.0003426 0.01% FabArray::ParallelCopy() 954 0.000328 0.000328 0.000328 0.01% Castro::subcycle_advance_ctu() 5 0.0003102 0.0003102 0.0003102 0.01% FabArrayBase::getFB() 1728 0.0003005 0.0003005 0.0003005 0.01% Castro::initialize_do_advance() 9 0.0002989 0.0002989 0.0002989 0.01% Castro::do_new_sources() 9 0.0002792 0.0002792 0.0002792 0.01% Castro::finalize_do_advance() 9 0.0002714 0.0002714 0.0002714 0.01% Gravity::update_max_rhs() 10 0.0002628 0.0002628 0.0002628 0.01% Castro::do_old_sources() 9 0.0002599 0.0002599 0.0002599 0.01% Castro::post_timestep() 9 0.0002412 0.0002412 0.0002412 0.01% Amr::coarseTimeStep() 5 0.0002363 0.0002363 0.0002363 0.01% AMRErrorTag::operator() 6 0.0002318 0.0002318 0.0002318 0.01% MLCellLinOp::apply() 493 0.0002212 0.0002212 0.0002212 0.01% Castro::construct_old_rotation_source() 9 0.0002181 0.0002181 0.0002181 0.01% Castro::construct_old_gravity() 9 0.00021 0.00021 0.00021 0.01% Castro::construct_new_rotation_source() 9 0.0002076 0.0002076 0.0002076 0.01% Castro::construct_new_gravity() 9 0.0002014 0.0002014 0.0002014 0.01% MLLinOp::defineGrids() 10 0.000199 0.000199 0.000199 0.01% Castro::react_state() 18 0.0001882 0.0001882 0.0001882 0.01% Castro::post_regrid() 10 0.0001669 0.0001669 0.0001669 0.01% Castro::create_source_corrector() 9 0.0001558 0.0001558 0.0001558 0.00% Castro::computeNewDt() 8 0.0001544 0.0001544 0.0001544 0.00% StateData::define() 25 0.0001539 0.0001539 0.0001539 0.00% Castro::enforce_consistent_e() 1 0.0001443 0.0001443 0.0001443 0.00% Castro::advance() 5 0.0001382 0.0001382 0.0001382 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0001363 0.0001363 0.0001363 0.00% MLCGSolver::ParallelAllReduce 745 0.0001346 0.0001346 0.0001346 0.00% Castro::finalize_advance() 9 0.000131 0.000131 0.000131 0.00% Castro::swap_state_time_levels() 9 0.0001288 0.0001288 0.0001288 0.00% MLMG::mgVcycle() 65 0.0001043 0.0001043 0.0001043 0.00% MLCellLinOp::correctionResidual() 189 0.0001023 0.0001023 0.0001023 0.00% Castro::buildMetrics() 5 9.966e-05 9.966e-05 9.966e-05 0.00% Amr::grid_places() 6 9.291e-05 9.291e-05 9.291e-05 0.00% Amr::timeStep() 9 8.256e-05 8.256e-05 8.256e-05 0.00% AmrLevel::derive() 6 8.05e-05 8.05e-05 8.05e-05 0.00% FillPatchSingleLevel 92 6.071e-05 6.071e-05 6.071e-05 0.00% MLMG::oneIter() 31 5.664e-05 5.664e-05 5.664e-05 0.00% AmrLevel::AmrLevel(dm) 5 5.658e-05 5.658e-05 5.658e-05 0.00% MLMG:computeResOfCorrection() 155 5.451e-05 5.451e-05 5.451e-05 0.00% Castro::errorEst() 6 5.057e-05 5.057e-05 5.057e-05 0.00% FabArray::RecvLayoutMask() 8 4.996e-05 4.996e-05 4.996e-05 0.00% MLMG::solve() 10 4.813e-05 4.813e-05 4.813e-05 0.00% MLCellLinOp::solutionResidual() 71 4.686e-05 4.686e-05 4.686e-05 0.00% Amr::defBaseLevel() 1 4.287e-05 4.287e-05 4.287e-05 0.00% MLMG::mgVcycle_bottom 65 4.085e-05 4.085e-05 4.085e-05 0.00% AmrLevel::FillCoarsePatch() 5 4.071e-05 4.071e-05 4.071e-05 0.00% MLMG::actualBottomSolve() 31 3.912e-05 3.912e-05 3.912e-05 0.00% Amr::writeSmallPlotFile() 1 3.67e-05 3.67e-05 3.67e-05 0.00% FillPatchIterator::FillFromLevel0() 31 3.621e-05 3.621e-05 3.621e-05 0.00% MLMG::mgVcycle_down::0 31 3.49e-05 3.49e-05 3.49e-05 0.00% MLMG::mgVcycle_down::1 31 3.191e-05 3.191e-05 3.191e-05 0.00% MLMG::mgVcycle_down::3 31 3.183e-05 3.183e-05 3.183e-05 0.00% FillPatchIterator::FillFromTwoLevels() 31 3.134e-05 3.134e-05 3.134e-05 0.00% MLMG::mgVcycle_down::2 31 3.055e-05 3.055e-05 3.055e-05 0.00% MLMG::mgVcycle_down::4 31 2.758e-05 2.758e-05 2.758e-05 0.00% MLMG::computeResWithCrseSolFineCor() 17 2.663e-05 2.663e-05 2.663e-05 0.00% MLPoisson::define() 10 2.618e-05 2.618e-05 2.618e-05 0.00% Gravity::solve_phi_with_mlmg() 10 2.576e-05 2.576e-05 2.576e-05 0.00% makeSFC 50 2.553e-05 2.553e-05 2.553e-05 0.00% MLMG::mgVcycle_up::4 31 2.546e-05 2.546e-05 2.546e-05 0.00% MLMG::ResNormInf() 54 2.308e-05 2.308e-05 2.308e-05 0.00% MLMG::mgVcycle_up::3 31 2.252e-05 2.252e-05 2.252e-05 0.00% DistributionMapping::Distribute() 52 2.073e-05 2.073e-05 2.073e-05 0.00% MLMG::mgVcycle_up::2 31 2.064e-05 2.064e-05 2.064e-05 0.00% MLMG::mgVcycle_up::0 31 2.064e-05 2.064e-05 2.064e-05 0.00% MLMG::mgVcycle_up::1 31 2.008e-05 2.008e-05 2.008e-05 0.00% FabArrayBase::TheFPinfo() 25 1.963e-05 1.963e-05 1.963e-05 0.00% FPinfo::FPinfo() 3 1.846e-05 1.846e-05 1.846e-05 0.00% MLLinOp::makeAgglomeratedDMap 10 1.423e-05 1.423e-05 1.423e-05 0.00% MLMG::computeMLResidual() 15 1.384e-05 1.384e-05 1.384e-05 0.00% MLMG::miniCycle() 34 1.383e-05 1.383e-05 1.383e-05 0.00% MLMG::computeResidual() 31 1.35e-05 1.35e-05 1.35e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 1.339e-05 1.339e-05 1.339e-05 0.00% Amr::FinalizeInit() 1 1.329e-05 1.329e-05 1.329e-05 0.00% Castro::check_for_nan() 18 1.325e-05 1.325e-05 1.325e-05 0.00% Amr::initSubcycle() 1 1.322e-05 1.322e-05 1.322e-05 0.00% ClusterList::intersect() 4 1.29e-05 1.29e-05 1.29e-05 0.00% Castro::apply_source_to_state() 18 1.049e-05 1.049e-05 1.049e-05 0.00% Castro::post_init() 1 1.016e-05 1.016e-05 1.016e-05 0.00% MLLinOp::define() 10 8.361e-06 8.361e-06 8.361e-06 0.00% Castro::expand_state() 13 8.309e-06 8.309e-06 8.309e-06 0.00% MLMG::getGradSolution() 10 7.821e-06 7.821e-06 7.821e-06 0.00% MLMG::MLResNormInf() 15 7.714e-06 7.714e-06 7.714e-06 0.00% Gravity::swapTimeLevels() 9 7.636e-06 7.636e-06 7.636e-06 0.00% Amr::InitializeInit() 1 7.634e-06 7.634e-06 7.634e-06 0.00% Castro::construct_new_hybrid_source() 9 7.625e-06 7.625e-06 7.625e-06 0.00% MLPoisson::prepareForSolve() 10 7.464e-06 7.464e-06 7.464e-06 0.00% Castro::avgDown(state_indx) 20 7.241e-06 7.241e-06 7.241e-06 0.00% AmrMesh-cluster 4 6.988e-06 6.988e-06 6.988e-06 0.00% amrex::average_down_w_geom 20 6.901e-06 6.901e-06 6.901e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 6.203e-06 6.203e-06 6.203e-06 0.00% Castro::construct_old_hybrid_source() 9 6.154e-06 6.154e-06 6.154e-06 0.00% Castro::init(old) 3 5.274e-06 5.274e-06 5.274e-06 0.00% MLMG::MLRhsNormInf() 10 5.076e-06 5.076e-06 5.076e-06 0.00% Castro::avgDown() 4 4.533e-06 4.533e-06 4.533e-06 0.00% BoxList::parallelComplementIn() 12 4.354e-06 4.354e-06 4.354e-06 0.00% Castro::computeInitialDt() 2 3.992e-06 3.992e-06 3.992e-06 0.00% Castro::derive() 6 3.915e-06 3.915e-06 3.915e-06 0.00% Gravity::set_mass_offset() 6 3.539e-06 3.539e-06 3.539e-06 0.00% BoxList::complementIn 12 3.471e-06 3.471e-06 3.471e-06 0.00% Castro::retry_advance_ctu() 5 2.822e-06 2.822e-06 2.822e-06 0.00% Amr::bldFineLevels() 1 2.607e-06 2.607e-06 2.607e-06 0.00% Amr::init() 1 2.583e-06 2.583e-06 2.583e-06 0.00% Castro::init() 1 2.432e-06 2.432e-06 2.432e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.875e-06 1.875e-06 1.875e-06 0.00% ClusterList::chop() 4 1.515e-06 1.515e-06 1.515e-06 0.00% Amr::initialInit() 1 1.476e-06 1.476e-06 1.476e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 3.12 3.12 3.12 100.00% Amr::coarseTimeStep() 5 1.505 1.505 1.505 48.24% Amr::timeStep() 9 1.5 1.5 1.5 48.07% Castro::advance() 5 1.277 1.277 1.277 40.92% Castro::subcycle_advance_ctu() 5 1.25 1.25 1.25 40.08% Castro::do_advance_ctu() 5 1.25 1.25 1.25 40.07% Amr::InitAmr() 1 1.064 1.064 1.064 34.11% Castro::construct_ctu_hydro_source() 9 0.7661 0.7661 0.7661 24.56% Amr::init() 1 0.3884 0.3884 0.3884 12.45% Gravity::multilevel_solve_for_new_phi() 10 0.3269 0.3269 0.3269 10.48% Gravity::actual_multilevel_solve() 10 0.3261 0.3261 0.3261 10.45% Gravity::solve_phi_with_mlmg() 10 0.3228 0.3228 0.3228 10.35% Amr::writePlotFile() 2 0.3114 0.3114 0.3114 9.98% VisMF::Write(FabArray) 3 0.3041 0.3041 0.3041 9.75% Gravity::actual_solve_with_mlmg() 10 0.251 0.251 0.251 8.05% Amr::initialInit() 1 0.2385 0.2385 0.2385 7.64% Castro::clean_state() 87 0.1984 0.1984 0.1984 6.36% MLMG::solve() 10 0.1967 0.1967 0.1967 6.30% Castro::construct_new_gravity() 9 0.1877 0.1877 0.1877 6.02% MLMG::oneIter() 31 0.1662 0.1662 0.1662 5.33% Amr::InitializeInit() 1 0.1659 0.1659 0.1659 5.32% Amr::defBaseLevel() 1 0.1659 0.1659 0.1659 5.32% Amr::regrid() 5 0.1571 0.1571 0.1571 5.04% Castro::Castro() 5 0.1567 0.1567 0.1567 5.02% Castro::initMFs() 5 0.1526 0.1526 0.1526 4.89% MLMG::mgVcycle() 65 0.1427 0.1427 0.1427 4.57% Castro::post_regrid() 10 0.1408 0.1408 0.1408 4.51% Castro::computeTemp() 88 0.104 0.104 0.104 3.33% MLCellLinOp::applyBC() 1917 0.09185 0.09185 0.09185 2.94% AmrLevel::FillPatch() 62 0.09171 0.09171 0.09171 2.94% MLCellLinOp::smooth() 688 0.08153 0.08153 0.08153 2.61% FillPatchIterator::Initialize 62 0.07725 0.07725 0.07725 2.48% Castro::normalize_species() 87 0.07478 0.07478 0.07478 2.40% Castro::reflux() 4 0.07471 0.07471 0.07471 2.39% Amr::FinalizeInit() 1 0.07252 0.07252 0.07252 2.32% Gravity::fill_multipole_BCs() 10 0.07126 0.07126 0.07126 2.28% FillPatchSingleLevel 92 0.06873 0.06873 0.06873 2.20% FillPatchIterator::FillFromLevel0() 31 0.06612 0.06612 0.06612 2.12% Castro::post_timestep() 9 0.06556 0.06556 0.06556 2.10% Castro::post_init() 1 0.06173 0.06173 0.06173 1.98% Castro::initialize_do_advance() 9 0.05625 0.05625 0.05625 1.80% Castro::problem_post_timestep() 10 0.05131 0.05131 0.05131 1.64% StateDataPhysBCFunct::() 95 0.04961 0.04961 0.04961 1.59% Castro::do_new_sources() 9 0.04782 0.04782 0.04782 1.53% FabArray::FillBoundary() 1728 0.0443 0.0443 0.0443 1.42% FillBoundary_nowait() 1728 0.04389 0.04389 0.04389 1.41% MLPoisson::define() 10 0.04354 0.04354 0.04354 1.40% Castro::expand_state() 13 0.0421 0.0421 0.0421 1.35% Castro::do_old_sources() 9 0.04206 0.04206 0.04206 1.35% FabArray::setVal() 920 0.03828 0.03828 0.03828 1.23% MLMG::mgVcycle_bottom 65 0.03827 0.03827 0.03827 1.23% StateData::FillBoundary(geom) 650 0.03511 0.03511 0.03511 1.13% FabArray::ParallelCopy() 954 0.03479 0.03479 0.03479 1.12% FabArray::ParallelCopy_nowait() 954 0.03447 0.03447 0.03447 1.10% Castro::wd_update() 6 0.03409 0.03409 0.03409 1.09% MLMG::actualBottomSolve() 31 0.03115 0.03115 0.03115 1.00% MLCGSolver::bicgstab 31 0.03086 0.03086 0.03086 0.99% Castro::volInBoundary() 42 0.02991 0.02991 0.02991 0.96% FluxRegister::Reflux() 24 0.0298 0.0298 0.0298 0.96% MLMG::mgVcycle_down::0 31 0.02909 0.02909 0.02909 0.93% Castro::reset_internal_energy(MultiFab) 88 0.02839 0.02839 0.02839 0.91% MLCellLinOp::apply() 493 0.02788 0.02788 0.02788 0.89% amrex::Copy() 441 0.02669 0.02669 0.02669 0.86% Castro::initialize_advance() 9 0.02589 0.02589 0.02589 0.83% MLCellLinOp::defineAuxData() 10 0.02492 0.02492 0.02492 0.80% Castro::construct_new_source() 63 0.0244 0.0244 0.0244 0.78% Gravity::get_new_grav_vector() 10 0.02307 0.02307 0.02307 0.74% MLMG::mgVcycle_up::0 31 0.02179 0.02179 0.02179 0.70% Castro::construct_old_gravity() 9 0.01882 0.01882 0.01882 0.60% Gravity::get_old_grav_vector() 9 0.01861 0.01861 0.01861 0.60% MLCellLinOp::defineBC() 10 0.01833 0.01833 0.01833 0.59% MLCellLinOp::correctionResidual() 189 0.01825 0.01825 0.01825 0.59% Castro::estTimeStep() 26 0.01748 0.01748 0.01748 0.56% Castro::enforce_min_density() 87 0.01737 0.01737 0.01737 0.56% amrex::Add() 136 0.01728 0.01728 0.01728 0.55% MLCellLinOp::reflux() 30 0.01593 0.01593 0.01593 0.51% FabArray::Saxpy() 604 0.01593 0.01593 0.01593 0.51% BndryData::define() 26 0.01545 0.01545 0.01545 0.50% Amr::grid_places() 6 0.01533 0.01533 0.01533 0.49% AmrMesh::MakeNewGrids() 6 0.01524 0.01524 0.01524 0.49% MLCellLinOp::solutionResidual() 71 0.01459 0.01459 0.01459 0.47% MLMG:computeResOfCorrection() 155 0.01395 0.01395 0.01395 0.45% MLMG::computeResWithCrseSolFineCor() 17 0.01357 0.01357 0.01357 0.43% MLMG::computeMLResidual() 15 0.01292 0.01292 0.01292 0.41% MLPoisson::Fsmooth() 1376 0.01258 0.01258 0.01258 0.40% Castro::construct_old_source() 63 0.01165 0.01165 0.01165 0.37% MLMG::mgVcycle_down::1 31 0.01164 0.01164 0.01164 0.37% Castro::initData() 1 0.01114 0.01114 0.01114 0.36% amrex::average_down 233 0.009828 0.009828 0.009828 0.31% amrex::Dot() 466 0.009153 0.009153 0.009153 0.29% Castro::FluxRegCrseInit 9 0.009 0.009 0.009 0.29% MLMG::prepareForSolve() 10 0.008883 0.008883 0.008883 0.28% Castro::construct_old_gravity_source() 18 0.008784 0.008784 0.008784 0.28% FabArray::norminf() 439 0.008722 0.008722 0.008722 0.28% MLMG::mgVcycle_up::1 31 0.008452 0.008452 0.008452 0.27% Castro::computeNewDt() 8 0.007881 0.007881 0.007881 0.25% FillPatchIterator::FillFromTwoLevels() 31 0.007423 0.007423 0.007423 0.24% FillPatchTwoLevels 31 0.007391 0.007391 0.007391 0.24% Amr::bldFineLevels() 1 0.007306 0.007306 0.007306 0.23% Castro::finalize_do_advance() 9 0.0073 0.0073 0.0073 0.23% MLMG::miniCycle() 34 0.007124 0.007124 0.007124 0.23% MLMG::addInterpCorrection() 155 0.006882 0.006882 0.006882 0.22% MLPoisson::Fapply() 493 0.00685 0.00685 0.00685 0.22% MLMG::mgVcycle_down::2 31 0.006778 0.006778 0.006778 0.22% FabArray::Xpay() 353 0.006458 0.006458 0.006458 0.21% MLMG::mgVcycle_down::3 31 0.006179 0.006179 0.006179 0.20% MLMG::mgVcycle_down::4 31 0.006043 0.006043 0.006043 0.19% MLMG::computeResidual() 31 0.005929 0.005929 0.005929 0.19% Castro::reset_internal_energy(Fab) 1476 0.00585 0.00585 0.00585 0.19% Castro::construct_new_gravity_source() 9 0.005748 0.005748 0.005748 0.18% Castro::errorEst() 6 0.005313 0.005313 0.005313 0.17% MLCGSolver::ParallelAllReduce 745 0.005206 0.005206 0.005206 0.17% MLMG::mgVcycle_up::4 31 0.004896 0.004896 0.004896 0.16% FabArrayBase::getCPC() 1522 0.004875 0.004875 0.004875 0.16% MLMG::mgVcycle_up::2 31 0.004743 0.004743 0.004743 0.15% MLMG::mgVcycle_up::3 31 0.004711 0.004711 0.004711 0.15% MLMG::getGradSolution() 10 0.004359 0.004359 0.004359 0.14% MLCellLinOp::compGrad() 18 0.004352 0.004352 0.004352 0.14% Castro::apply_source_to_state() 18 0.004227 0.004227 0.004227 0.14% FabArrayBase::CPC::define() 534 0.003763 0.003763 0.003763 0.12% FillPatchInterp(MF) 30 0.003663 0.003663 0.003663 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.003627 0.003627 0.003627 0.12% MLMG::computeResWithCrseCorFineCor() 17 0.003576 0.003576 0.003576 0.11% Castro::apply_tagging_restrictions() 6 0.003573 0.003573 0.003573 0.11% MLCellLinOp::updateSolBC() 25 0.003372 0.003372 0.003372 0.11% FabArray::setDomainBndry() 92 0.003316 0.003316 0.003316 0.11% TagBoxArray::mapPRD 6 0.003275 0.003275 0.003275 0.10% Castro::check_for_nan() 18 0.003265 0.003265 0.003265 0.10% MultiFab::contains_nan() 18 0.003252 0.003252 0.003252 0.10% Castro::init(old) 3 0.003042 0.003042 0.003042 0.10% Castro::computeInitialDt() 2 0.002726 0.002726 0.002726 0.09% FabArray::mult() 55 0.002578 0.002578 0.002578 0.08% Castro::buildMetrics() 5 0.002576 0.002576 0.002576 0.08% MLCellLinOp::setLevelBC() 18 0.002512 0.002512 0.002512 0.08% Castro::enforce_speed_limit() 87 0.002476 0.002476 0.002476 0.08% check_for_negative_density() 9 0.00238 0.00238 0.00238 0.08% MLCellLinOp::updateCorBC() 17 0.002342 0.002342 0.002342 0.08% MLMG::ResNormInf() 54 0.002079 0.002079 0.002079 0.07% Gravity::update_max_rhs() 10 0.002062 0.002062 0.002062 0.07% MLPoisson::prepareForSolve() 10 0.002008 0.002008 0.002008 0.06% MLCellLinOp::prepareForSolve() 10 0.002 0.002 0.002 0.06% Gravity::average_fine_ec_onto_crse_ec() 8 0.001983 0.001983 0.001983 0.06% Castro::fill_hybrid_hydro_source() 27 0.001691 0.001691 0.001691 0.05% MLPoisson::FFlux() 150 0.001488 0.001488 0.001488 0.05% average_down_faces 48 0.001304 0.001304 0.001304 0.04% FabArray::norminf(mask) 21 0.001229 0.001229 0.001229 0.04% Castro::avgDown() 4 0.001199 0.001199 0.001199 0.04% Castro::avgDown(state_indx) 20 0.001194 0.001194 0.001194 0.04% amrex::average_down_w_geom 20 0.001187 0.001187 0.001187 0.04% MLMG::MLResNormInf() 15 0.001143 0.001143 0.001143 0.04% OwnerMask() 6 0.001045 0.001045 0.001045 0.03% Castro::construct_new_hybrid_source() 9 0.001044 0.001044 0.001044 0.03% Castro::build_fine_mask() 44 0.001036 0.001036 0.001036 0.03% MLMG::interpCorrection_1 17 0.0009751 0.0009751 0.0009751 0.03% Castro::derive() 6 0.0009346 0.0009346 0.0009346 0.03% AmrLevel::derive() 6 0.0009307 0.0009307 0.0009307 0.03% Castro::init() 1 0.0009024 0.0009024 0.0009024 0.03% AmrLevel::FillCoarsePatch() 5 0.0009 0.0009 0.0009 0.03% TagBoxArray::collate() 6 0.0008073 0.0008073 0.0008073 0.03% FabArrayBase::getFB() 1728 0.0007369 0.0007369 0.0007369 0.02% Gravity::swapTimeLevels() 9 0.0006917 0.0006917 0.0006917 0.02% Castro::FluxRegFineAdd() 9 0.0006633 0.0006633 0.0006633 0.02% Castro::construct_old_hybrid_source() 9 0.0006609 0.0006609 0.0006609 0.02% MultiFab::max() 18 0.000619 0.000619 0.000619 0.02% MLMG::MLRhsNormInf() 10 0.000598 0.000598 0.000598 0.02% Castro::react_state() 18 0.0005643 0.0005643 0.0005643 0.02% Castro::apply_problem_tags() 6 0.0005227 0.0005227 0.0005227 0.02% MultiFab::Multiply() 8 0.0005063 0.0005063 0.0005063 0.02% FabArrayBase::FB::FB() 37 0.0004364 0.0004364 0.0004364 0.01% Castro::linear_to_hybrid_momentum() 1 0.0003694 0.0003694 0.0003694 0.01% MLLinOp::define() 10 0.000268 0.000268 0.000268 0.01% MLLinOp::defineGrids() 10 0.0002597 0.0002597 0.0002597 0.01% AMRErrorTag::operator() 6 0.0002318 0.0002318 0.0002318 0.01% Castro::construct_old_rotation_source() 9 0.0002181 0.0002181 0.0002181 0.01% AmrLevel::AmrLevel(dm) 5 0.0002105 0.0002105 0.0002105 0.01% Castro::construct_new_rotation_source() 9 0.0002076 0.0002076 0.0002076 0.01% Castro::create_source_corrector() 9 0.0001558 0.0001558 0.0001558 0.00% StateData::define() 25 0.0001539 0.0001539 0.0001539 0.00% Castro::enforce_consistent_e() 1 0.0001443 0.0001443 0.0001443 0.00% Castro::finalize_advance() 9 0.000131 0.000131 0.000131 0.00% Castro::swap_state_time_levels() 9 0.0001288 0.0001288 0.0001288 0.00% MLLinOp::makeAgglomeratedDMap 10 5.874e-05 5.874e-05 5.874e-05 0.00% FabArray::RecvLayoutMask() 8 4.996e-05 4.996e-05 4.996e-05 0.00% makeSFC 50 4.45e-05 4.45e-05 4.45e-05 0.00% FabArrayBase::TheFPinfo() 25 3.809e-05 3.809e-05 3.809e-05 0.00% Amr::writeSmallPlotFile() 1 3.67e-05 3.67e-05 3.67e-05 0.00% AmrMesh-cluster 4 2.141e-05 2.141e-05 2.141e-05 0.00% DistributionMapping::Distribute() 52 2.073e-05 2.073e-05 2.073e-05 0.00% FPinfo::FPinfo() 3 1.846e-05 1.846e-05 1.846e-05 0.00% Amr::initSubcycle() 1 1.322e-05 1.322e-05 1.322e-05 0.00% ClusterList::intersect() 4 1.29e-05 1.29e-05 1.29e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 7.953e-06 7.953e-06 7.953e-06 0.00% BoxList::parallelComplementIn() 12 7.825e-06 7.825e-06 7.825e-06 0.00% Gravity::set_mass_offset() 6 3.539e-06 3.539e-06 3.539e-06 0.00% BoxList::complementIn 12 3.471e-06 3.471e-06 3.471e-06 0.00% Castro::retry_advance_ctu() 5 2.822e-06 2.822e-06 2.822e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.875e-06 1.875e-06 1.875e-06 0.00% ClusterList::chop() 4 1.515e-06 1.515e-06 1.515e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Device Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Arena::Initialize() 1 1 1685 KiB 9037 MiB Castro::reflux() 436 436 13 MiB 666 MiB Castro::initMFs() 258 258 347 MiB 576 MiB Castro::swap_state_time_levels() 175 175 178 MiB 340 MiB StateData::define() 175 175 219 MiB 340 MiB Castro::initialize_do_advance() 143 143 122 MiB 325 MiB FillPatchIterator::Initialize 899 899 4089 KiB 303 MiB Amr::writePlotFile() 56 56 18 MiB 189 MiB Castro::problem_post_timestep() 162 162 804 KiB 167 MiB Castro::FluxRegCrseInit 660 660 451 KiB 167 MiB FluxRegister::Reflux() 648 648 1550 KiB 160 MiB Castro::initialize_advance() 143 143 48 MiB 121 MiB Castro::construct_new_source() 143 143 186 KiB 67 MiB Castro::construct_old_source() 143 143 114 KiB 67 MiB Castro::buildMetrics() 140 140 37 MiB 61 MiB Gravity::multilevel_solve_for_new_phi() 858 858 28 MiB 60 MiB Castro::Castro() 210 210 4072 KiB 50 MiB MLMG::prepareForSolve() 1902 1902 2714 KiB 42 MiB Gravity::get_old_grav_vector() 155 155 195 KiB 33 MiB Gravity::get_new_grav_vector() 188 188 241 KiB 33 MiB ResizeRandomSeed 1 1 30 MiB 30 MiB MLCellLinOp::defineAuxData() 5608 5608 893 KiB 16 MiB FillPatchTwoLevels 300 300 7694 B 11 MiB FillPatchInterp(MF) 330 330 4716 B 7955 KiB Gravity::fill_multipole_BCs() 346 346 127 KiB 7469 KiB Gravity::update_max_rhs() 286 286 4546 B 7424 KiB Gravity::actual_multilevel_solve() 286 286 760 KiB 7424 KiB Castro::build_fine_mask() 224 224 1265 KiB 6912 KiB AmrLevel::derive() 162 162 2582 B 6912 KiB BndryData::define() 3624 3624 284 KiB 4243 KiB MLCellLinOp::reflux() 450 450 5673 B 2376 KiB Castro::wd_update() 6 6 24 KiB 2240 KiB AmrLevel::FillCoarsePatch() 10 10 410 B 2095 KiB amrex::average_down 1483 1483 2356 B 1474 KiB AmrMesh::MakeNewGrids() 324 324 5138 B 1041 KiB MLMG::addInterpCorrection() 1240 1240 795 B 867 KiB MLCellLinOp::defineBC() 276 276 3854 B 689 KiB Castro::estTimeStep() 26 26 2604 B 480 KiB VisMF::Write(FabArray) 1568 1568 2336 B 320 KiB Castro::normalize_species() 87 87 7652 B 320 KiB Castro::volInBoundary() 70 70 3110 B 320 KiB Gravity::average_fine_ec_onto_crse_ec() 48 48 127 B 204 KiB FabArray::norminf(mask) 29 29 189 B 163 KiB amrex::Dot() 559 559 436 B 160 KiB FabArray::norminf() 470 470 432 B 160 KiB MultiFab::max() 18 18 31 B 160 KiB check_for_negative_density() 9 9 123 B 160 KiB Castro::initData() 1 1 45 B 160 KiB FabArray::setVal() 419 419 54 KiB 120 KiB MLMG::interpCorrection_1 34 34 27 B 91 KiB average_down_faces 99 99 641 B 70 KiB FabArray::ParallelCopy_nowait() 942 942 281 B 57 KiB MLPoisson::Fsmooth() 228 228 3101 B 55 KiB FillBoundary_nowait() 751 751 450 B 53 KiB TagBoxArray::mapPRD 324 324 42 B 45 KiB OwnerMask() 168 168 12 B 38 KiB MLCellLinOp::applyBC() 3834 3834 143 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9530 B 25 KiB MultiFab::contains_nan() 18 18 21 B 20 KiB StateData::FillBoundary(geom) 2538 2538 43 B 14 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B MultiFab::Multiply() 8 8 631 B 3456 B MLCGSolver::bicgstab 155 155 26 B 2688 B TagBoxArray::collate() 14 14 0 B 640 B ------------------------------------------------------------------------------ Managed Memory Usage: ----------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem ----------------------------------------------------------------- The_Managed_Arena::Initialize() 1 1 160 B 8192 KiB ----------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Pinned_Arena::Initialize() 1 1 11 KiB 8192 KiB VisMF::Write(FabArray) 1624 1624 624 KiB 7168 KiB Gravity::fill_multipole_BCs() 30 30 1242 B 278 KiB FabArray::setVal() 419 419 54 KiB 120 KiB FabArray::ParallelCopy_nowait() 942 942 281 B 57 KiB MLPoisson::Fsmooth() 228 228 3101 B 55 KiB FillBoundary_nowait() 751 751 450 B 53 KiB MLCellLinOp::applyBC() 1917 1917 143 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9530 B 25 KiB Gravity::get_new_grav_vector() 18 18 3408 B 20 KiB Gravity::get_old_grav_vector() 12 12 6341 B 20 KiB OwnerMask() 6 6 0 B 15 KiB StateData::FillBoundary(geom) 2538 2538 43 B 14 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B amrex::average_down 149 149 1340 B 6304 B MLMG::prepareForSolve() 18 18 231 B 3712 B FabArray::norminf(mask) 29 29 129 B 3472 B MLMG::addInterpCorrection() 62 62 2 B 3456 B MultiFab::Multiply() 8 8 631 B 3456 B Castro::FluxRegCrseInit 12 12 9 B 3456 B FillPatchInterp(MF) 140 140 3 B 3232 B MLCellLinOp::setLevelBC() 68 68 0 B 2592 B average_down_faces 51 51 613 B 2272 B TagBoxArray::collate() 16 16 0 B 640 B amrex::Dot() 559 559 3 B 400 B Castro::build_fine_mask() 8 8 0 B 208 B MLCellLinOp::defineAuxData() 8 8 0 B 208 B Castro::reflux() 4 4 0 B 208 B FabArray::norminf() 470 470 1 B 144 B Castro::wd_update() 6 6 1 B 112 B Castro::problem_post_timestep() 19 19 0 B 48 B Castro::estTimeStep() 26 26 0 B 32 B MultiFab::contains_nan() 18 18 0 B 16 B MultiFab::max() 18 18 0 B 16 B Castro::volInBoundary() 70 70 0 B 16 B Castro::normalize_species() 87 87 0 B 16 B Castro::initData() 1 1 0 B 16 B check_for_negative_density() 9 9 0 B 16 B ------------------------------------------------------------------------------ Total GPU global memory (MB): 12049 Free GPU global memory (MB): 1856 [The Arena] space allocated (MB): 9037 [The Arena] space used (MB): 0 [The Managed Arena] space allocated (MB): 8 [The Managed Arena] space used (MB): 0 [The Pinned Arena] space allocated (MB): 8 [The Pinned Arena] space used (MB): 0 AMReX (v2024-33-g9e71c2cb9714) finalized