Initializing AMReX (24.04-18-ge293ff685199)... Initializing CUDA... CUDA initialized with 1 device. AMReX (24.04-18-ge293ff685199) initialized Starting run at 09:11:23 UTC on 2024-04-16. Successfully read inputs file ... Castro git describe: 24.04-14-g3d76cd5a0 AMReX git describe: 24.04-18-ge293ff685 Microphysics git describe: 24.04-7-gd4312a6c reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.004090329 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.001749894 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.009071511 Gravity::multilevel_solve_for_new_phi() time = 0.053552032 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.149837747 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003383797 on level 0 Castro::do_old_sources() time = 0.009306808 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.137919798 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.006761665 Gravity::multilevel_solve_for_new_phi() time = 0.040980692 Castro::construct_new_gravity() time = 0.044606768 on level 0 Castro::do_new_sources() time = 0.00868243 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 3.892627599 [Level 0 step 1] Advanced 884736 cells Updating the rotational period from 30.54368673 s to 30.46017574 s. [STEP 1] Coarse TimeStep time: 0.2371964 [STEP 1] FAB kilobyte spread across MPI nodes: [2005477 ... 2005477] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.002306964 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007434873 Gravity::multilevel_solve_for_new_phi() time = 0.06095147 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135647e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (104,88,99) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135647e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135647e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003482216 on level 0 Castro::do_old_sources() time = 0.011384466 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.186493026 on level 0 Castro::construct_old_gravity() time = 0.000356225 on level 1 Castro::do_old_sources() time = 0.002156391 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.008019628 on level 1 Castro::reflux() at level 0 : time = 0.018074412 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007322353 Gravity::multilevel_solve_for_new_phi() time = 0.041949364 Castro::construct_new_gravity() time = 0.045526171 on level 0 Castro::do_new_sources() time = 0.006582432 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461716053 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716053 Castro::construct_new_gravity() time = 0.000379351 on level 1 Castro::do_new_sources() time = 0.002589284 on level 1 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 2.953887304 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Updating the rotational period from 30.46017574 s to 30.52759479 s. [STEP 2] Coarse TimeStep time: 0.406999978 [STEP 2] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 2 TIME = 0.005426075163 DT = 2.526135647e-09 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008044208529 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.697001619e-07 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Now regridding at level lbase = 0 grid_places() time: 0.002096897 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007262078 Gravity::multilevel_solve_for_new_phi() time = 0.015413379 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778749212e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003525301 on level 0 Castro::do_old_sources() time = 0.00710047 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.169363553 on level 0 Castro::construct_old_gravity() time = 0.000361413 on level 1 Castro::do_old_sources() time = 0.002038842 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.008023002 on level 1 Castro::reflux() at level 0 : time = 0.01784308 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007250685 Gravity::multilevel_solve_for_new_phi() time = 0.029413311 Castro::construct_new_gravity() time = 0.033125558 on level 0 Castro::do_new_sources() time = 0.006837353 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008044237461 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044237461 Castro::construct_new_gravity() time = 0.000401852 on level 1 Castro::do_new_sources() time = 0.002578153 on level 1 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.421644328 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Updating the rotational period from 30.52759479 s to 30.52761298 s. [STEP 3] Coarse TimeStep time: 0.322096876 [STEP 3] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 3 TIME = 0.005426077942 DT = 2.778749212e-09 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.000804308035 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.965184638e-07 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Now regridding at level lbase = 0 grid_places() time: 0.002190773 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007227611 Gravity::multilevel_solve_for_new_phi() time = 0.015472918 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056624133e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.00347047 on level 0 Castro::do_old_sources() time = 0.006716981 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.182612418 on level 0 Castro::construct_old_gravity() time = 0.000364033 on level 1 Castro::do_old_sources() time = 0.002595034 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.00738953 on level 1 Castro::reflux() at level 0 : time = 0.017448061 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007260184 Gravity::multilevel_solve_for_new_phi() time = 0.029380932 Castro::construct_new_gravity() time = 0.032999782 on level 0 Castro::do_new_sources() time = 0.007067441 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008043112215 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008043112215 Castro::construct_new_gravity() time = 0.000426395 on level 1 Castro::do_new_sources() time = 0.002459649 on level 1 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.268604796 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Updating the rotational period from 30.52761298 s to 30.52761297 s. [STEP 4] Coarse TimeStep time: 0.330456563 [STEP 4] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 4 TIME = 0.005426080998 DT = 3.056624133e-09 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002497928903 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 7.255166705e-07 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Now regridding at level lbase = 0 grid_places() time: 0.002180919 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.00725957 Gravity::multilevel_solve_for_new_phi() time = 0.015471137 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286546e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003296112 on level 0 Castro::do_old_sources() time = 0.00692807 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.172180298 on level 0 Castro::construct_old_gravity() time = 0.000341317 on level 1 Castro::do_old_sources() time = 0.001976297 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.007647218 on level 1 Castro::reflux() at level 0 : time = 0.018524456 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007253631 Gravity::multilevel_solve_for_new_phi() time = 0.029509757 Castro::construct_new_gravity() time = 0.033098046 on level 0 Castro::do_new_sources() time = 0.006816829 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006805523364 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006805523364 Castro::construct_new_gravity() time = 0.00041622 on level 1 Castro::do_new_sources() time = 0.001633657 on level 1 ...estimated hydro-limited timestep at level 1: 7.553385904e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.553385904e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.433548101 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Updating the rotational period from 30.52761297 s to 30.52761296 s. [STEP 5] Coarse TimeStep time: 0.316308322 [STEP 5] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 5 TIME = 0.005426084361 DT = 3.362286546e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.158319602 seconds Ending run at 09:11:27 UTC on 2024-04-16. Run time = 3.273919151 Run time without initialization = 1.77227712 Average number of zones advanced per microsecond: 2.792 Average number of zones advanced per microsecond per rank: 2.792 CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9476456448 TinyProfiler total time across processes [min...avg...max]: 3.274 ... 3.274 ... 3.274 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Amr::InitAmr() 1 1.1136 1.1136 1.1136 34.01% Castro::construct_ctu_hydro_source() 9 0.8433 0.8433 0.8433 25.76% VisMF::Write(FabArray) 3 0.3013 0.3013 0.3013 9.20% Castro::initMFs() 5 0.149 0.149 0.149 4.55% Gravity::fill_multipole_BCs() 10 0.07211 0.07211 0.07211 2.20% Castro::normalize_species() 87 0.07211 0.07211 0.07211 2.20% Castro::computeTemp() 88 0.06737 0.06737 0.06737 2.06% MLCellLinOp::applyBC() 1917 0.04834 0.04834 0.04834 1.48% FillBoundary_nowait() 1728 0.04322 0.04322 0.04322 1.32% FabArray::setVal() 920 0.03838 0.03838 0.03838 1.17% StateData::FillBoundary(geom) 650 0.03572 0.03572 0.03572 1.09% FabArray::ParallelCopy_nowait() 954 0.03062 0.03062 0.03062 0.94% Castro::volInBoundary() 42 0.03025 0.03025 0.03025 0.92% amrex::Copy() 441 0.02671 0.02671 0.02671 0.82% Castro::reset_internal_energy(MultiFab) 88 0.02387 0.02387 0.02387 0.73% FluxRegister::Reflux() 24 0.0224 0.0224 0.0224 0.68% MLCellLinOp::defineAuxData() 10 0.02194 0.02194 0.02194 0.67% amrex::Add() 136 0.01727 0.01727 0.01727 0.53% Castro::estTimeStep() 26 0.01664 0.01664 0.01664 0.51% Castro::enforce_min_density() 87 0.0162 0.0162 0.0162 0.49% FabArray::Saxpy() 604 0.01598 0.01598 0.01598 0.49% StateDataPhysBCFunct::() 95 0.01592 0.01592 0.01592 0.49% Castro::construct_new_source() 63 0.01423 0.01423 0.01423 0.43% BndryData::define() 26 0.01361 0.01361 0.01361 0.42% MLPoisson::Fsmooth() 1376 0.01261 0.01261 0.01261 0.39% Castro::problem_post_timestep() 10 0.01101 0.01101 0.01101 0.34% Castro::reflux() 4 0.01032 0.01032 0.01032 0.32% MLCellLinOp::reflux() 30 0.009775 0.009775 0.009775 0.30% amrex::Dot() 466 0.00934 0.00934 0.00934 0.29% FabArray::norminf() 439 0.008827 0.008827 0.008827 0.27% Castro::construct_old_gravity_source() 18 0.008624 0.008624 0.008624 0.26% Castro::FluxRegCrseInit 9 0.007964 0.007964 0.007964 0.24% MLPoisson::Fapply() 493 0.006885 0.006885 0.006885 0.21% Castro::initData() 1 0.006807 0.006807 0.006807 0.21% FabArray::Xpay() 353 0.006503 0.006503 0.006503 0.20% AmrMesh::MakeNewGrids() 6 0.006226 0.006226 0.006226 0.19% amrex::average_down 233 0.00617 0.00617 0.00617 0.19% Castro::construct_new_gravity_source() 9 0.005453 0.005453 0.005453 0.17% Castro::construct_old_source() 63 0.005036 0.005036 0.005036 0.15% Castro::wd_update() 6 0.004953 0.004953 0.004953 0.15% Castro::reset_internal_energy(Fab) 1476 0.004666 0.004666 0.004666 0.14% Castro::enforce_speed_limit() 87 0.004542 0.004542 0.004542 0.14% Amr::writePlotFile() 2 0.00451 0.00451 0.00451 0.14% MLMG::addInterpCorrection() 155 0.004473 0.004473 0.004473 0.14% FabArrayBase::CPC::define() 534 0.003827 0.003827 0.003827 0.12% Gravity::actual_solve_with_mlmg() 10 0.003764 0.003764 0.003764 0.11% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.003736 0.003736 0.003736 0.11% FillPatchInterp(MF) 30 0.003707 0.003707 0.003707 0.11% FabArray::setDomainBndry() 92 0.003396 0.003396 0.003396 0.10% MultiFab::contains_nan() 18 0.003247 0.003247 0.003247 0.10% check_for_negative_density() 9 0.003223 0.003223 0.003223 0.10% MLCellLinOp::compGrad() 18 0.002925 0.002925 0.002925 0.09% MLCellLinOp::defineBC() 10 0.002787 0.002787 0.002787 0.09% Castro::clean_state() 87 0.002763 0.002763 0.002763 0.08% FabArray::mult() 55 0.002591 0.002591 0.002591 0.08% MLCellLinOp::setLevelBC() 18 0.002583 0.002583 0.002583 0.08% Castro::apply_tagging_restrictions() 6 0.002166 0.002166 0.002166 0.07% MLCellLinOp::prepareForSolve() 10 0.002001 0.002001 0.002001 0.06% MLMG::prepareForSolve() 10 0.001847 0.001847 0.001847 0.06% Castro::fill_hybrid_hydro_source() 27 0.001665 0.001665 0.001665 0.05% Castro::advance() 5 0.001601 0.001601 0.001601 0.05% MLPoisson::FFlux() 150 0.001529 0.001529 0.001529 0.05% TagBoxArray::mapPRD 6 0.001412 0.001412 0.001412 0.04% MLCellLinOp::updateSolBC() 25 0.001407 0.001407 0.001407 0.04% Gravity::get_new_grav_vector() 10 0.001281 0.001281 0.001281 0.04% FabArray::norminf(mask) 21 0.001225 0.001225 0.001225 0.04% FabArrayBase::getCPC() 1522 0.001144 0.001144 0.001144 0.03% MLCellLinOp::updateCorBC() 17 0.00104 0.00104 0.00104 0.03% OwnerMask() 6 0.001026 0.001026 0.001026 0.03% Castro::build_fine_mask() 44 0.001025 0.001025 0.001025 0.03% Gravity::get_old_grav_vector() 9 0.0009819 0.0009819 0.0009819 0.03% Castro::post_timestep() 9 0.0008631 0.0008631 0.0008631 0.03% average_down_faces 48 0.0008605 0.0008605 0.0008605 0.03% TagBoxArray::collate() 6 0.0008248 0.0008248 0.0008248 0.03% Gravity::multilevel_solve_for_new_phi() 10 0.0008192 0.0008192 0.0008192 0.03% FillPatchTwoLevels 31 0.0007581 0.0007581 0.0007581 0.02% MLCGSolver::bicgstab 31 0.0007158 0.0007158 0.0007158 0.02% main() 1 0.0006997 0.0006997 0.0006997 0.02% Castro::FluxRegFineAdd() 9 0.0006598 0.0006598 0.0006598 0.02% AmrLevel::FillPatch() 62 0.0006401 0.0006401 0.0006401 0.02% MultiFab::max() 18 0.0006109 0.0006109 0.0006109 0.02% FillPatchIterator::Initialize 62 0.0005882 0.0005882 0.0005882 0.02% Castro::apply_problem_tags() 6 0.0005396 0.0005396 0.0005396 0.02% MLCellLinOp::smooth() 688 0.0005277 0.0005277 0.0005277 0.02% MultiFab::Multiply() 8 0.0005015 0.0005015 0.0005015 0.02% Castro::linear_to_hybrid_momentum() 1 0.0004539 0.0004539 0.0004539 0.01% FabArrayBase::FB::FB() 37 0.0004394 0.0004394 0.0004394 0.01% MLMG::interpCorrection_1 17 0.0004173 0.0004173 0.0004173 0.01% Amr::regrid() 5 0.0004062 0.0004062 0.0004062 0.01% Castro::Castro() 5 0.0003959 0.0003959 0.0003959 0.01% FabArray::FillBoundary() 1728 0.0003941 0.0003941 0.0003941 0.01% Castro::initialize_do_advance() 9 0.000361 0.000361 0.000361 0.01% FabArray::ParallelCopy() 954 0.0003193 0.0003193 0.0003193 0.01% FabArrayBase::getFB() 1728 0.0003154 0.0003154 0.0003154 0.01% Amr::coarseTimeStep() 5 0.0002798 0.0002798 0.0002798 0.01% Castro::subcycle_advance_ctu() 5 0.0002782 0.0002782 0.0002782 0.01% Gravity::actual_multilevel_solve() 10 0.0002445 0.0002445 0.0002445 0.01% Gravity::update_max_rhs() 10 0.0002433 0.0002433 0.0002433 0.01% MLCellLinOp::apply() 493 0.0002414 0.0002414 0.0002414 0.01% AMRErrorTag::operator() 6 0.0002348 0.0002348 0.0002348 0.01% Castro::construct_new_rotation_source() 9 0.0002206 0.0002206 0.0002206 0.01% Castro::do_advance_ctu() 5 0.0002161 0.0002161 0.0002161 0.01% MLLinOp::defineGrids() 10 0.0002 0.0002 0.0002 0.01% Castro::construct_old_rotation_source() 9 0.0001949 0.0001949 0.0001949 0.01% Castro::finalize_do_advance() 9 0.0001891 0.0001891 0.0001891 0.01% StateData::define() 25 0.0001588 0.0001588 0.0001588 0.00% Castro::initialize_advance() 9 0.0001493 0.0001493 0.0001493 0.00% Castro::enforce_consistent_e() 1 0.0001451 0.0001451 0.0001451 0.00% Castro::construct_new_gravity() 9 0.0001414 0.0001414 0.0001414 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0001357 0.0001357 0.0001357 0.00% MLCGSolver::ParallelAllReduce 745 0.0001338 0.0001338 0.0001338 0.00% Castro::swap_state_time_levels() 9 0.000127 0.000127 0.000127 0.00% Castro::do_old_sources() 9 0.000125 0.000125 0.000125 0.00% Castro::buildMetrics() 5 0.0001133 0.0001133 0.0001133 0.00% MLCellLinOp::correctionResidual() 189 0.0001 0.0001 0.0001 0.00% Amr::grid_places() 6 9.992e-05 9.992e-05 9.992e-05 0.00% Castro::construct_old_gravity() 9 9.956e-05 9.956e-05 9.956e-05 0.00% Castro::create_source_corrector() 9 9.525e-05 9.525e-05 9.525e-05 0.00% MLMG::mgVcycle() 65 9.517e-05 9.517e-05 9.517e-05 0.00% Castro::finalize_advance() 9 9.089e-05 9.089e-05 9.089e-05 0.00% Castro::do_new_sources() 9 8.794e-05 8.794e-05 8.794e-05 0.00% AmrLevel::derive() 6 7.756e-05 7.756e-05 7.756e-05 0.00% Amr::timeStep() 9 7.637e-05 7.637e-05 7.637e-05 0.00% FillPatchSingleLevel 92 6.311e-05 6.311e-05 6.311e-05 0.00% Castro::post_regrid() 10 6.302e-05 6.302e-05 6.302e-05 0.00% MLMG::oneIter() 31 5.866e-05 5.866e-05 5.866e-05 0.00% AmrLevel::AmrLevel(dm) 5 5.32e-05 5.32e-05 5.32e-05 0.00% Castro::errorEst() 6 5.005e-05 5.005e-05 5.005e-05 0.00% FabArray::RecvLayoutMask() 8 4.951e-05 4.951e-05 4.951e-05 0.00% Amr::defBaseLevel() 1 4.92e-05 4.92e-05 4.92e-05 0.00% MLMG:computeResOfCorrection() 155 4.904e-05 4.904e-05 4.904e-05 0.00% MLMG::solve() 10 4.844e-05 4.844e-05 4.844e-05 0.00% MLCellLinOp::solutionResidual() 71 4.38e-05 4.38e-05 4.38e-05 0.00% AmrLevel::FillCoarsePatch() 5 4.04e-05 4.04e-05 4.04e-05 0.00% MLMG::mgVcycle_bottom 65 3.936e-05 3.936e-05 3.936e-05 0.00% Amr::writeSmallPlotFile() 1 3.844e-05 3.844e-05 3.844e-05 0.00% FillPatchIterator::FillFromLevel0() 31 3.811e-05 3.811e-05 3.811e-05 0.00% MLMG::actualBottomSolve() 31 3.809e-05 3.809e-05 3.809e-05 0.00% MLMG::mgVcycle_down::0 31 3.515e-05 3.515e-05 3.515e-05 0.00% FillPatchIterator::FillFromTwoLevels() 31 3.448e-05 3.448e-05 3.448e-05 0.00% MLMG::mgVcycle_down::1 31 3.168e-05 3.168e-05 3.168e-05 0.00% Gravity::solve_phi_with_mlmg() 10 3.138e-05 3.138e-05 3.138e-05 0.00% MLMG::mgVcycle_down::2 31 2.854e-05 2.854e-05 2.854e-05 0.00% MLMG::ResNormInf() 54 2.833e-05 2.833e-05 2.833e-05 0.00% MLMG::mgVcycle_down::3 31 2.827e-05 2.827e-05 2.827e-05 0.00% Castro::react_state() 18 2.788e-05 2.788e-05 2.788e-05 0.00% MLMG::computeResWithCrseSolFineCor() 17 2.71e-05 2.71e-05 2.71e-05 0.00% MLPoisson::define() 10 2.6e-05 2.6e-05 2.6e-05 0.00% makeSFC 50 2.597e-05 2.597e-05 2.597e-05 0.00% MLMG::mgVcycle_down::4 31 2.512e-05 2.512e-05 2.512e-05 0.00% Castro::computeNewDt() 8 2.496e-05 2.496e-05 2.496e-05 0.00% FabArrayBase::TheFPinfo() 25 2.363e-05 2.363e-05 2.363e-05 0.00% DistributionMapping::Distribute() 52 2.29e-05 2.29e-05 2.29e-05 0.00% MLMG::mgVcycle_up::4 31 2.272e-05 2.272e-05 2.272e-05 0.00% MLMG::mgVcycle_up::0 31 2.061e-05 2.061e-05 2.061e-05 0.00% MLMG::mgVcycle_up::3 31 2.045e-05 2.045e-05 2.045e-05 0.00% MLMG::mgVcycle_up::1 31 1.922e-05 1.922e-05 1.922e-05 0.00% FPinfo::FPinfo() 3 1.893e-05 1.893e-05 1.893e-05 0.00% MLMG::mgVcycle_up::2 31 1.802e-05 1.802e-05 1.802e-05 0.00% MLMG::computeMLResidual() 15 1.778e-05 1.778e-05 1.778e-05 0.00% Amr::initSubcycle() 1 1.559e-05 1.559e-05 1.559e-05 0.00% MLLinOp::makeAgglomeratedDMap 10 1.412e-05 1.412e-05 1.412e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 1.393e-05 1.393e-05 1.393e-05 0.00% MLMG::miniCycle() 34 1.384e-05 1.384e-05 1.384e-05 0.00% MLMG::computeResidual() 31 1.349e-05 1.349e-05 1.349e-05 0.00% MLLinOp::define() 10 1.293e-05 1.293e-05 1.293e-05 0.00% Amr::FinalizeInit() 1 1.287e-05 1.287e-05 1.287e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 1.201e-05 1.201e-05 1.201e-05 0.00% ClusterList::intersect() 4 1.191e-05 1.191e-05 1.191e-05 0.00% Castro::check_for_nan() 18 1.085e-05 1.085e-05 1.085e-05 0.00% Castro::post_init() 1 9.968e-06 9.968e-06 9.968e-06 0.00% Castro::apply_source_to_state() 18 9.827e-06 9.827e-06 9.827e-06 0.00% Castro::expand_state() 13 9.227e-06 9.227e-06 9.227e-06 0.00% Castro::construct_new_hybrid_source() 9 8.733e-06 8.733e-06 8.733e-06 0.00% MLMG::getGradSolution() 10 8.523e-06 8.523e-06 8.523e-06 0.00% Amr::InitializeInit() 1 8.054e-06 8.054e-06 8.054e-06 0.00% Gravity::swapTimeLevels() 9 7.737e-06 7.737e-06 7.737e-06 0.00% MLMG::MLResNormInf() 15 7.364e-06 7.364e-06 7.364e-06 0.00% amrex::average_down_w_geom 20 7.049e-06 7.049e-06 7.049e-06 0.00% Castro::avgDown(state_indx) 20 7.019e-06 7.019e-06 7.019e-06 0.00% AmrMesh-cluster 4 6.947e-06 6.947e-06 6.947e-06 0.00% MLPoisson::prepareForSolve() 10 6.936e-06 6.936e-06 6.936e-06 0.00% Castro::construct_old_hybrid_source() 9 6.755e-06 6.755e-06 6.755e-06 0.00% BoxList::parallelComplementIn() 12 5.575e-06 5.575e-06 5.575e-06 0.00% MLMG::MLRhsNormInf() 10 5.293e-06 5.293e-06 5.293e-06 0.00% BoxList::complementIn 12 4.98e-06 4.98e-06 4.98e-06 0.00% Castro::init(old) 3 4.829e-06 4.829e-06 4.829e-06 0.00% Gravity::set_mass_offset() 6 4.702e-06 4.702e-06 4.702e-06 0.00% Castro::derive() 6 4.026e-06 4.026e-06 4.026e-06 0.00% Castro::avgDown() 4 3.682e-06 3.682e-06 3.682e-06 0.00% Castro::retry_advance_ctu() 5 2.79e-06 2.79e-06 2.79e-06 0.00% Castro::computeInitialDt() 2 2.783e-06 2.783e-06 2.783e-06 0.00% Castro::init() 1 2.21e-06 2.21e-06 2.21e-06 0.00% Amr::bldFineLevels() 1 2.199e-06 2.199e-06 2.199e-06 0.00% ClusterList::chop() 4 1.959e-06 1.959e-06 1.959e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.944e-06 1.944e-06 1.944e-06 0.00% Amr::init() 1 1.846e-06 1.846e-06 1.846e-06 0.00% Amr::initialInit() 1 1.172e-06 1.172e-06 1.172e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 3.274 3.274 3.274 100.00% Amr::coarseTimeStep() 5 1.613 1.613 1.613 49.28% Amr::timeStep() 9 1.608 1.608 1.608 49.11% Castro::advance() 5 1.394 1.394 1.394 42.59% Castro::subcycle_advance_ctu() 5 1.37 1.37 1.37 41.86% Castro::do_advance_ctu() 5 1.37 1.37 1.37 41.85% Amr::InitAmr() 1 1.114 1.114 1.114 34.01% Castro::construct_ctu_hydro_source() 9 0.8797 0.8797 0.8797 26.87% Amr::init() 1 0.3879 0.3879 0.3879 11.85% Gravity::multilevel_solve_for_new_phi() 10 0.3323 0.3323 0.3323 10.15% Gravity::actual_multilevel_solve() 10 0.3315 0.3315 0.3315 10.12% Gravity::solve_phi_with_mlmg() 10 0.3282 0.3282 0.3282 10.03% Amr::writePlotFile() 2 0.3083 0.3083 0.3083 9.42% VisMF::Write(FabArray) 3 0.3013 0.3013 0.3013 9.20% Gravity::actual_solve_with_mlmg() 10 0.2535 0.2535 0.2535 7.74% Amr::initialInit() 1 0.238 0.238 0.238 7.27% MLMG::solve() 10 0.1984 0.1984 0.1984 6.06% Castro::construct_new_gravity() 9 0.1911 0.1911 0.1911 5.84% Castro::clean_state() 87 0.1902 0.1902 0.1902 5.81% MLMG::oneIter() 31 0.1677 0.1677 0.1677 5.12% Amr::InitializeInit() 1 0.1634 0.1634 0.1634 4.99% Amr::defBaseLevel() 1 0.1634 0.1634 0.1634 4.99% Castro::Castro() 5 0.1532 0.1532 0.1532 4.68% Castro::initMFs() 5 0.149 0.149 0.149 4.55% Amr::regrid() 5 0.1488 0.1488 0.1488 4.55% MLMG::mgVcycle() 65 0.1439 0.1439 0.1439 4.40% Castro::post_regrid() 10 0.1342 0.1342 0.1342 4.10% Castro::computeTemp() 88 0.0959 0.0959 0.0959 2.93% AmrLevel::FillPatch() 62 0.09393 0.09393 0.09393 2.87% MLCellLinOp::applyBC() 1917 0.09271 0.09271 0.09271 2.83% MLCellLinOp::smooth() 688 0.08205 0.08205 0.08205 2.51% FillPatchIterator::Initialize 62 0.07944 0.07944 0.07944 2.43% Amr::FinalizeInit() 1 0.07461 0.07461 0.07461 2.28% Gravity::fill_multipole_BCs() 10 0.07422 0.07422 0.07422 2.27% Castro::normalize_species() 87 0.07211 0.07211 0.07211 2.20% Castro::reflux() 4 0.07193 0.07193 0.07193 2.20% FillPatchSingleLevel 92 0.0708 0.0708 0.0708 2.16% FillPatchIterator::FillFromLevel0() 31 0.06814 0.06814 0.06814 2.08% Castro::post_timestep() 9 0.06558 0.06558 0.06558 2.00% Castro::post_init() 1 0.06268 0.06268 0.06268 1.91% Castro::initialize_do_advance() 9 0.05785 0.05785 0.05785 1.77% Castro::problem_post_timestep() 10 0.05214 0.05214 0.05214 1.59% StateDataPhysBCFunct::() 95 0.05164 0.05164 0.05164 1.58% Castro::do_old_sources() 9 0.05029 0.05029 0.05029 1.54% Castro::do_new_sources() 9 0.04532 0.04532 0.04532 1.38% FabArray::FillBoundary() 1728 0.04437 0.04437 0.04437 1.36% MLPoisson::define() 10 0.04432 0.04432 0.04432 1.35% FillBoundary_nowait() 1728 0.04397 0.04397 0.04397 1.34% Castro::expand_state() 13 0.04266 0.04266 0.04266 1.30% MLMG::mgVcycle_bottom 65 0.03877 0.03877 0.03877 1.18% FabArray::setVal() 920 0.03838 0.03838 0.03838 1.17% StateData::FillBoundary(geom) 650 0.03572 0.03572 0.03572 1.09% Castro::wd_update() 6 0.03521 0.03521 0.03521 1.08% FabArray::ParallelCopy() 954 0.03499 0.03499 0.03499 1.07% FabArray::ParallelCopy_nowait() 954 0.03467 0.03467 0.03467 1.06% MLMG::actualBottomSolve() 31 0.03161 0.03161 0.03161 0.97% MLCGSolver::bicgstab 31 0.03132 0.03132 0.03132 0.96% Castro::volInBoundary() 42 0.03025 0.03025 0.03025 0.92% FluxRegister::Reflux() 24 0.02971 0.02971 0.02971 0.91% MLMG::mgVcycle_down::0 31 0.02922 0.02922 0.02922 0.89% Castro::reset_internal_energy(MultiFab) 88 0.02853 0.02853 0.02853 0.87% MLCellLinOp::apply() 493 0.02826 0.02826 0.02826 0.86% Castro::construct_new_source() 63 0.02697 0.02697 0.02697 0.82% amrex::Copy() 441 0.02671 0.02671 0.02671 0.82% MLCellLinOp::defineAuxData() 10 0.02536 0.02536 0.02536 0.77% Gravity::get_new_grav_vector() 10 0.02371 0.02371 0.02371 0.72% Castro::initialize_advance() 9 0.02221 0.02221 0.02221 0.68% MLMG::mgVcycle_up::0 31 0.0218 0.0218 0.0218 0.67% Castro::construct_old_gravity() 9 0.01867 0.01867 0.01867 0.57% MLCellLinOp::defineBC() 10 0.01866 0.01866 0.01866 0.57% Gravity::get_old_grav_vector() 9 0.01857 0.01857 0.01857 0.57% MLCellLinOp::correctionResidual() 189 0.01845 0.01845 0.01845 0.56% amrex::Add() 136 0.01727 0.01727 0.01727 0.53% Castro::estTimeStep() 26 0.01664 0.01664 0.01664 0.51% Castro::enforce_min_density() 87 0.0162 0.0162 0.0162 0.49% MLCellLinOp::reflux() 30 0.01608 0.01608 0.01608 0.49% FabArray::Saxpy() 604 0.01598 0.01598 0.01598 0.49% BndryData::define() 26 0.01573 0.01573 0.01573 0.48% MLCellLinOp::solutionResidual() 71 0.0148 0.0148 0.0148 0.45% Amr::grid_places() 6 0.01467 0.01467 0.01467 0.45% AmrMesh::MakeNewGrids() 6 0.01457 0.01457 0.01457 0.45% MLMG:computeResOfCorrection() 155 0.01408 0.01408 0.01408 0.43% MLMG::computeResWithCrseSolFineCor() 17 0.01373 0.01373 0.01373 0.42% Castro::construct_old_source() 63 0.01317 0.01317 0.01317 0.40% MLMG::computeMLResidual() 15 0.01304 0.01304 0.01304 0.40% MLPoisson::Fsmooth() 1376 0.01261 0.01261 0.01261 0.39% Castro::initData() 1 0.01187 0.01187 0.01187 0.36% MLMG::mgVcycle_down::1 31 0.01166 0.01166 0.01166 0.36% amrex::average_down 233 0.009939 0.009939 0.009939 0.30% amrex::Dot() 466 0.00934 0.00934 0.00934 0.29% Castro::FluxRegCrseInit 9 0.009024 0.009024 0.009024 0.28% MLMG::prepareForSolve() 10 0.008914 0.008914 0.008914 0.27% FabArray::norminf() 439 0.008827 0.008827 0.008827 0.27% Castro::construct_old_gravity_source() 18 0.008624 0.008624 0.008624 0.26% MLMG::mgVcycle_up::1 31 0.00852 0.00852 0.00852 0.26% Amr::bldFineLevels() 1 0.008299 0.008299 0.008299 0.25% FillPatchIterator::FillFromTwoLevels() 31 0.007493 0.007493 0.007493 0.23% FillPatchTwoLevels 31 0.007458 0.007458 0.007458 0.23% MLMG::miniCycle() 34 0.007171 0.007171 0.007171 0.22% Castro::computeNewDt() 8 0.007067 0.007067 0.007067 0.22% Castro::finalize_do_advance() 9 0.007051 0.007051 0.007051 0.22% MLMG::addInterpCorrection() 155 0.006929 0.006929 0.006929 0.21% MLPoisson::Fapply() 493 0.006885 0.006885 0.006885 0.21% MLMG::mgVcycle_down::2 31 0.006878 0.006878 0.006878 0.21% FabArray::Xpay() 353 0.006503 0.006503 0.006503 0.20% MLMG::mgVcycle_down::3 31 0.006267 0.006267 0.006267 0.19% MLMG::mgVcycle_down::4 31 0.006149 0.006149 0.006149 0.19% MLMG::computeResidual() 31 0.006028 0.006028 0.006028 0.18% Castro::construct_new_gravity_source() 9 0.005453 0.005453 0.005453 0.17% MLCGSolver::ParallelAllReduce 745 0.00529 0.00529 0.00529 0.16% FabArrayBase::getCPC() 1522 0.004971 0.004971 0.004971 0.15% MLMG::mgVcycle_up::4 31 0.00496 0.00496 0.00496 0.15% MLMG::mgVcycle_up::2 31 0.004794 0.004794 0.004794 0.15% MLMG::mgVcycle_up::3 31 0.004772 0.004772 0.004772 0.15% Castro::reset_internal_energy(Fab) 1476 0.004666 0.004666 0.004666 0.14% Castro::enforce_speed_limit() 87 0.004542 0.004542 0.004542 0.14% MLMG::getGradSolution() 10 0.004445 0.004445 0.004445 0.14% MLCellLinOp::compGrad() 18 0.004436 0.004436 0.004436 0.14% Castro::apply_source_to_state() 18 0.004219 0.004219 0.004219 0.13% Castro::errorEst() 6 0.003918 0.003918 0.003918 0.12% FabArrayBase::CPC::define() 534 0.003827 0.003827 0.003827 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.003736 0.003736 0.003736 0.11% FillPatchInterp(MF) 30 0.003707 0.003707 0.003707 0.11% MLMG::computeResWithCrseCorFineCor() 17 0.003632 0.003632 0.003632 0.11% MLCellLinOp::updateSolBC() 25 0.003416 0.003416 0.003416 0.10% FabArray::setDomainBndry() 92 0.003396 0.003396 0.003396 0.10% TagBoxArray::mapPRD 6 0.003285 0.003285 0.003285 0.10% Castro::check_for_nan() 18 0.003258 0.003258 0.003258 0.10% MultiFab::contains_nan() 18 0.003247 0.003247 0.003247 0.10% check_for_negative_density() 9 0.003223 0.003223 0.003223 0.10% Castro::init(old) 3 0.003091 0.003091 0.003091 0.09% Castro::computeInitialDt() 2 0.002738 0.002738 0.002738 0.08% Castro::buildMetrics() 5 0.002603 0.002603 0.002603 0.08% FabArray::mult() 55 0.002591 0.002591 0.002591 0.08% MLCellLinOp::setLevelBC() 18 0.002583 0.002583 0.002583 0.08% MLCellLinOp::updateCorBC() 17 0.00238 0.00238 0.00238 0.07% Castro::apply_tagging_restrictions() 6 0.002166 0.002166 0.002166 0.07% MLMG::ResNormInf() 54 0.002085 0.002085 0.002085 0.06% Gravity::update_max_rhs() 10 0.00202 0.00202 0.00202 0.06% MLPoisson::prepareForSolve() 10 0.002008 0.002008 0.002008 0.06% MLCellLinOp::prepareForSolve() 10 0.002001 0.002001 0.002001 0.06% Gravity::average_fine_ec_onto_crse_ec() 8 0.001997 0.001997 0.001997 0.06% Castro::fill_hybrid_hydro_source() 27 0.001665 0.001665 0.001665 0.05% MLPoisson::FFlux() 150 0.001529 0.001529 0.001529 0.05% average_down_faces 48 0.001321 0.001321 0.001321 0.04% FabArray::norminf(mask) 21 0.001225 0.001225 0.001225 0.04% Castro::avgDown() 4 0.001182 0.001182 0.001182 0.04% Castro::avgDown(state_indx) 20 0.001178 0.001178 0.001178 0.04% amrex::average_down_w_geom 20 0.001171 0.001171 0.001171 0.04% MLMG::MLResNormInf() 15 0.001148 0.001148 0.001148 0.04% Castro::construct_new_hybrid_source() 9 0.001058 0.001058 0.001058 0.03% OwnerMask() 6 0.001026 0.001026 0.001026 0.03% Castro::build_fine_mask() 44 0.001025 0.001025 0.001025 0.03% MLMG::interpCorrection_1 17 0.0009946 0.0009946 0.0009946 0.03% Castro::derive() 6 0.0009276 0.0009276 0.0009276 0.03% AmrLevel::derive() 6 0.0009235 0.0009235 0.0009235 0.03% Castro::init() 1 0.000903 0.000903 0.000903 0.03% AmrLevel::FillCoarsePatch() 5 0.0009008 0.0009008 0.0009008 0.03% TagBoxArray::collate() 6 0.0008248 0.0008248 0.0008248 0.03% FabArrayBase::getFB() 1728 0.0007547 0.0007547 0.0007547 0.02% Gravity::swapTimeLevels() 9 0.0006943 0.0006943 0.0006943 0.02% Castro::FluxRegFineAdd() 9 0.0006598 0.0006598 0.0006598 0.02% Castro::construct_old_hybrid_source() 9 0.000623 0.000623 0.000623 0.02% MultiFab::max() 18 0.0006109 0.0006109 0.0006109 0.02% MLMG::MLRhsNormInf() 10 0.000604 0.000604 0.000604 0.02% Castro::apply_problem_tags() 6 0.0005396 0.0005396 0.0005396 0.02% MultiFab::Multiply() 8 0.0005015 0.0005015 0.0005015 0.02% Castro::linear_to_hybrid_momentum() 1 0.0004539 0.0004539 0.0004539 0.01% FabArrayBase::FB::FB() 37 0.0004394 0.0004394 0.0004394 0.01% Castro::react_state() 18 0.0003921 0.0003921 0.0003921 0.01% MLLinOp::define() 10 0.0002734 0.0002734 0.0002734 0.01% MLLinOp::defineGrids() 10 0.0002605 0.0002605 0.0002605 0.01% AMRErrorTag::operator() 6 0.0002348 0.0002348 0.0002348 0.01% Castro::construct_new_rotation_source() 9 0.0002206 0.0002206 0.0002206 0.01% AmrLevel::AmrLevel(dm) 5 0.000212 0.000212 0.000212 0.01% Castro::construct_old_rotation_source() 9 0.0001949 0.0001949 0.0001949 0.01% StateData::define() 25 0.0001588 0.0001588 0.0001588 0.00% Castro::enforce_consistent_e() 1 0.0001451 0.0001451 0.0001451 0.00% Castro::swap_state_time_levels() 9 0.000127 0.000127 0.000127 0.00% Castro::create_source_corrector() 9 9.525e-05 9.525e-05 9.525e-05 0.00% Castro::finalize_advance() 9 9.089e-05 9.089e-05 9.089e-05 0.00% MLLinOp::makeAgglomeratedDMap 10 5.85e-05 5.85e-05 5.85e-05 0.00% FabArray::RecvLayoutMask() 8 4.951e-05 4.951e-05 4.951e-05 0.00% makeSFC 50 4.438e-05 4.438e-05 4.438e-05 0.00% FabArrayBase::TheFPinfo() 25 4.256e-05 4.256e-05 4.256e-05 0.00% Amr::writeSmallPlotFile() 1 3.844e-05 3.844e-05 3.844e-05 0.00% DistributionMapping::Distribute() 52 2.29e-05 2.29e-05 2.29e-05 0.00% AmrMesh-cluster 4 2.082e-05 2.082e-05 2.082e-05 0.00% FPinfo::FPinfo() 3 1.893e-05 1.893e-05 1.893e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 1.649e-05 1.649e-05 1.649e-05 0.00% Amr::initSubcycle() 1 1.559e-05 1.559e-05 1.559e-05 0.00% ClusterList::intersect() 4 1.191e-05 1.191e-05 1.191e-05 0.00% BoxList::parallelComplementIn() 12 1.055e-05 1.055e-05 1.055e-05 0.00% BoxList::complementIn 12 4.98e-06 4.98e-06 4.98e-06 0.00% Gravity::set_mass_offset() 6 4.702e-06 4.702e-06 4.702e-06 0.00% Castro::retry_advance_ctu() 5 2.79e-06 2.79e-06 2.79e-06 0.00% ClusterList::chop() 4 1.959e-06 1.959e-06 1.959e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.944e-06 1.944e-06 1.944e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Device Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Arena::Initialize() 1 1 1571 KiB 9037 MiB Castro::reflux() 436 436 11 MiB 666 MiB Castro::initMFs() 258 258 347 MiB 576 MiB Castro::swap_state_time_levels() 175 175 181 MiB 340 MiB StateData::define() 175 175 220 MiB 340 MiB Castro::initialize_do_advance() 143 143 127 MiB 325 MiB FillPatchIterator::Initialize 899 899 3951 KiB 303 MiB Amr::writePlotFile() 56 56 17 MiB 189 MiB Castro::problem_post_timestep() 162 162 805 KiB 167 MiB Castro::FluxRegCrseInit 660 660 431 KiB 167 MiB FluxRegister::Reflux() 648 648 1473 KiB 160 MiB Castro::initialize_advance() 143 143 51 MiB 121 MiB Castro::construct_new_source() 143 143 239 KiB 67 MiB Castro::construct_old_source() 143 143 136 KiB 67 MiB Castro::buildMetrics() 140 140 37 MiB 61 MiB Gravity::multilevel_solve_for_new_phi() 858 858 28 MiB 60 MiB Castro::Castro() 210 210 4012 KiB 50 MiB MLMG::prepareForSolve() 1902 1902 2609 KiB 42 MiB Gravity::get_old_grav_vector() 155 155 188 KiB 33 MiB Gravity::get_new_grav_vector() 188 188 237 KiB 33 MiB ResizeRandomSeed 1 1 30 MiB 30 MiB MLCellLinOp::defineAuxData() 5608 5608 861 KiB 16 MiB FillPatchTwoLevels 300 300 7392 B 11 MiB FillPatchInterp(MF) 330 330 4560 B 7955 KiB Gravity::fill_multipole_BCs() 346 346 128 KiB 7469 KiB Gravity::update_max_rhs() 286 286 4268 B 7424 KiB Gravity::actual_multilevel_solve() 286 286 736 KiB 7424 KiB Castro::build_fine_mask() 224 224 1170 KiB 6912 KiB AmrLevel::derive() 162 162 2457 B 6912 KiB BndryData::define() 3624 3624 273 KiB 4243 KiB MLCellLinOp::reflux() 450 450 5426 B 2376 KiB Castro::wd_update() 6 6 23 KiB 2240 KiB AmrLevel::FillCoarsePatch() 10 10 392 B 2095 KiB amrex::average_down 1483 1483 2318 B 1474 KiB AmrMesh::MakeNewGrids() 324 324 4678 B 1042 KiB MLMG::addInterpCorrection() 1240 1240 763 B 867 KiB MLCellLinOp::defineBC() 276 276 3711 B 689 KiB Castro::estTimeStep() 26 26 2417 B 480 KiB VisMF::Write(FabArray) 1568 1568 2234 B 320 KiB Castro::normalize_species() 87 87 7171 B 320 KiB Castro::volInBoundary() 70 70 2998 B 320 KiB Gravity::average_fine_ec_onto_crse_ec() 48 48 122 B 204 KiB FabArray::norminf(mask) 29 29 180 B 163 KiB amrex::Dot() 559 559 423 B 160 KiB FabArray::norminf() 470 470 416 B 160 KiB MultiFab::max() 18 18 29 B 160 KiB check_for_negative_density() 9 9 159 B 160 KiB Castro::initData() 1 1 47 B 160 KiB FabArray::setVal() 419 419 55 KiB 120 KiB MLMG::interpCorrection_1 34 34 26 B 91 KiB average_down_faces 99 99 645 B 70 KiB FabArray::ParallelCopy_nowait() 942 942 266 B 57 KiB MLPoisson::Fsmooth() 228 228 2982 B 55 KiB FillBoundary_nowait() 751 751 422 B 53 KiB TagBoxArray::mapPRD 324 324 40 B 45 KiB OwnerMask() 168 168 11 B 38 KiB MLCellLinOp::applyBC() 3834 3834 137 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9640 B 25 KiB MultiFab::contains_nan() 18 18 20 B 20 KiB StateData::FillBoundary(geom) 2538 2538 45 B 13 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B MultiFab::Multiply() 8 8 584 B 3456 B MLCGSolver::bicgstab 155 155 25 B 2688 B TagBoxArray::collate() 14 14 0 B 640 B ------------------------------------------------------------------------------ Managed Memory Usage: ----------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem ----------------------------------------------------------------- The_Managed_Arena::Initialize() 1 1 163 B 8192 KiB ----------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Pinned_Arena::Initialize() 1 1 20 KiB 8192 KiB VisMF::Write(FabArray) 1624 1624 588 KiB 7168 KiB Gravity::fill_multipole_BCs() 30 30 1190 B 278 KiB FabArray::setVal() 419 419 55 KiB 120 KiB FabArray::ParallelCopy_nowait() 942 942 266 B 57 KiB MLPoisson::Fsmooth() 228 228 2982 B 55 KiB FillBoundary_nowait() 751 751 423 B 53 KiB MLCellLinOp::applyBC() 1917 1917 136 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9640 B 25 KiB Gravity::get_new_grav_vector() 18 18 3361 B 20 KiB Gravity::get_old_grav_vector() 12 12 6670 B 20 KiB OwnerMask() 6 6 0 B 15 KiB StateData::FillBoundary(geom) 2538 2538 46 B 13 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B amrex::average_down 149 149 1349 B 6304 B MLMG::prepareForSolve() 18 18 222 B 3712 B FabArray::norminf(mask) 29 29 124 B 3472 B MLMG::addInterpCorrection() 62 62 2 B 3456 B MultiFab::Multiply() 8 8 584 B 3456 B Castro::FluxRegCrseInit 12 12 8 B 3456 B FillPatchInterp(MF) 140 140 3 B 3232 B MLCellLinOp::setLevelBC() 68 68 0 B 2592 B average_down_faces 51 51 618 B 2272 B TagBoxArray::collate() 16 16 0 B 640 B amrex::Dot() 559 559 3 B 400 B Castro::build_fine_mask() 8 8 0 B 208 B MLCellLinOp::defineAuxData() 8 8 0 B 208 B Castro::reflux() 4 4 0 B 208 B FabArray::norminf() 470 470 1 B 144 B Castro::wd_update() 6 6 1 B 112 B Castro::problem_post_timestep() 19 19 0 B 48 B Castro::estTimeStep() 26 26 0 B 32 B MultiFab::contains_nan() 18 18 0 B 16 B MultiFab::max() 18 18 0 B 16 B Castro::volInBoundary() 70 70 0 B 16 B Castro::normalize_species() 87 87 0 B 16 B Castro::initData() 1 1 0 B 16 B check_for_negative_density() 9 9 0 B 16 B ------------------------------------------------------------------------------ Total GPU global memory (MB): 12049 Free GPU global memory (MB): 1781 [The Arena] space allocated (MB): 9037 [The Arena] space used (MB): 0 [The Managed Arena] space allocated (MB): 8 [The Managed Arena] space used (MB): 0 [The Pinned Arena] space allocated (MB): 8 [The Pinned Arena] space used (MB): 0 AMReX (24.04-18-ge293ff685199) finalized