Initializing AMReX (24.03-8-gae3af4339425)... Initializing CUDA... CUDA initialized with 1 device. AMReX (24.03-8-gae3af4339425) initialized Starting run at 08:49:20 UTC on 2024-03-13. Successfully read inputs file ... Castro git describe: 24.03-4-ge94d82665 AMReX git describe: 24.03-8-gae3af4339 Microphysics git describe: 24.03-7-g124b71be reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.003251376 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.001691218 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.009324605 Gravity::multilevel_solve_for_new_phi() time = 0.052596493 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.144054168 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003437097 on level 0 Castro::do_old_sources() time = 0.009236561 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.127520534 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.006932406 Gravity::multilevel_solve_for_new_phi() time = 0.048830283 Castro::construct_new_gravity() time = 0.052340186 on level 0 Castro::do_new_sources() time = 0.007702571 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 3.968488211 [Level 0 step 1] Advanced 884736 cells Updating the rotational period from 30.54368673 s to 30.46017574 s. [STEP 1] Coarse TimeStep time: 0.23214375 [STEP 1] FAB kilobyte spread across MPI nodes: [2005477 ... 2005477] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.002367295 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007817799 Gravity::multilevel_solve_for_new_phi() time = 0.060051883 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135647e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (104,88,99) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135647e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135647e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003414914 on level 0 Castro::do_old_sources() time = 0.006505052 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.120590355 on level 0 Castro::construct_old_gravity() time = 0.000414596 on level 1 Castro::do_old_sources() time = 0.001833267 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.011328098 on level 1 Castro::reflux() at level 0 : time = 0.021220683 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006885695 Gravity::multilevel_solve_for_new_phi() time = 0.040575822 Castro::construct_new_gravity() time = 0.044141861 on level 0 Castro::do_new_sources() time = 0.009711264 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461716053 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716053 Castro::construct_new_gravity() time = 0.000422416 on level 1 Castro::do_new_sources() time = 0.001456109 on level 1 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.801592167 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Updating the rotational period from 30.46017574 s to 30.52759479 s. [STEP 2] Coarse TimeStep time: 0.337116467 [STEP 2] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 2 TIME = 0.005426075163 DT = 2.526135647e-09 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008044208529 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.697001619e-07 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Now regridding at level lbase = 0 grid_places() time: 0.002890829 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.00691005 Gravity::multilevel_solve_for_new_phi() time = 0.014668377 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778749212e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003509654 on level 0 Castro::do_old_sources() time = 0.006794999 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.13411166 on level 0 Castro::construct_old_gravity() time = 0.000402587 on level 1 Castro::do_old_sources() time = 0.001456863 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.011198293 on level 1 Castro::reflux() at level 0 : time = 0.018185272 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006927059 Gravity::multilevel_solve_for_new_phi() time = 0.028358511 Castro::construct_new_gravity() time = 0.031890121 on level 0 Castro::do_new_sources() time = 0.006549389 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008044237461 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044237461 Castro::construct_new_gravity() time = 0.00043772 on level 1 Castro::do_new_sources() time = 0.001876298 on level 1 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.776982482 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Updating the rotational period from 30.52759479 s to 30.52761298 s. [STEP 3] Coarse TimeStep time: 0.300999007 [STEP 3] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 3 TIME = 0.005426077942 DT = 2.778749212e-09 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.000804308035 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.965184638e-07 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Now regridding at level lbase = 0 grid_places() time: 0.002131007 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.006924418 Gravity::multilevel_solve_for_new_phi() time = 0.014765687 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056624133e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003413482 on level 0 Castro::do_old_sources() time = 0.006555756 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.154023688 on level 0 Castro::construct_old_gravity() time = 0.000397847 on level 1 Castro::do_old_sources() time = 0.001642043 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.013105936 on level 1 Castro::reflux() at level 0 : time = 0.017384276 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006962794 Gravity::multilevel_solve_for_new_phi() time = 0.028392698 Castro::construct_new_gravity() time = 0.031905577 on level 0 Castro::do_new_sources() time = 0.006403264 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008043112215 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008043112215 Castro::construct_new_gravity() time = 0.000438682 on level 1 Castro::do_new_sources() time = 0.002955884 on level 1 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.609367577 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Updating the rotational period from 30.52761298 s to 30.52761297 s. [STEP 4] Coarse TimeStep time: 0.301414154 [STEP 4] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 4 TIME = 0.005426080998 DT = 3.056624133e-09 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002497928903 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 7.255166705e-07 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Now regridding at level lbase = 0 grid_places() time: 0.002188194 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.006944831 Gravity::multilevel_solve_for_new_phi() time = 0.014870151 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286546e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003448898 on level 0 Castro::do_old_sources() time = 0.006774198 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.109911407 on level 0 Castro::construct_old_gravity() time = 0.000392258 on level 1 Castro::do_old_sources() time = 0.001561304 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.012994764 on level 1 Castro::reflux() at level 0 : time = 0.021484703 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.006924438 Gravity::multilevel_solve_for_new_phi() time = 0.028560087 Castro::construct_new_gravity() time = 0.032002801 on level 0 Castro::do_new_sources() time = 0.009576421 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006805523364 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006805523364 Castro::construct_new_gravity() time = 0.000432303 on level 1 Castro::do_new_sources() time = 0.002315002 on level 1 ...estimated hydro-limited timestep at level 1: 7.553385904e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.553385904e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 4.146479021 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Updating the rotational period from 30.52761297 s to 30.52761296 s. [STEP 5] Coarse TimeStep time: 0.270569558 [STEP 5] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 5 TIME = 0.005426084361 DT = 3.362286546e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.154462388 seconds Ending run at 08:49:23 UTC on 2024-03-13. Run time = 3.022432586 Run time without initialization = 1.597569362 Average number of zones advanced per microsecond: 3.097 Average number of zones advanced per microsecond per rank: 3.097 CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9476456448 TinyProfiler total time across processes [min...avg...max]: 3.022 ... 3.022 ... 3.022 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Amr::InitAmr() 1 1.0411 1.0411 1.0411 34.45% Castro::construct_ctu_hydro_source() 9 0.662 0.662 0.662 21.90% VisMF::Write(FabArray) 3 0.2916 0.2916 0.2916 9.65% Castro::initMFs() 5 0.1533 0.1533 0.1533 5.07% Castro::computeTemp() 88 0.07537 0.07537 0.07537 2.49% Castro::normalize_species() 87 0.07161 0.07161 0.07161 2.37% Gravity::fill_multipole_BCs() 10 0.07061 0.07061 0.07061 2.34% MLCellLinOp::applyBC() 1917 0.0473 0.0473 0.0473 1.56% FillBoundary_nowait() 1728 0.04276 0.04276 0.04276 1.41% FabArray::setVal() 920 0.03786 0.03786 0.03786 1.25% StateData::FillBoundary(geom) 650 0.0351 0.0351 0.0351 1.16% Castro::volInBoundary() 42 0.03041 0.03041 0.03041 1.01% FabArray::ParallelCopy_nowait() 954 0.03005 0.03005 0.03005 0.99% amrex::Copy() 441 0.02653 0.02653 0.02653 0.88% Castro::reset_internal_energy(MultiFab) 88 0.0232 0.0232 0.0232 0.77% FluxRegister::Reflux() 24 0.02243 0.02243 0.02243 0.74% MLCellLinOp::defineAuxData() 10 0.02013 0.02013 0.02013 0.67% Castro::enforce_min_density() 87 0.02001 0.02001 0.02001 0.66% MLPoisson::Fsmooth() 1376 0.01988 0.01988 0.01988 0.66% Castro::estTimeStep() 26 0.01913 0.01913 0.01913 0.63% Castro::reflux() 4 0.01796 0.01796 0.01796 0.59% amrex::Add() 136 0.01729 0.01729 0.01729 0.57% FabArray::Saxpy() 604 0.01582 0.01582 0.01582 0.52% StateDataPhysBCFunct::() 95 0.01454 0.01454 0.01454 0.48% BndryData::define() 26 0.01243 0.01243 0.01243 0.41% Castro::construct_new_source() 63 0.01044 0.01044 0.01044 0.35% MLCellLinOp::reflux() 30 0.01006 0.01006 0.01006 0.33% Castro::problem_post_timestep() 10 0.00998 0.00998 0.00998 0.33% Castro::construct_old_gravity_source() 18 0.009348 0.009348 0.009348 0.31% amrex::Dot() 466 0.008931 0.008931 0.008931 0.30% FabArray::norminf() 439 0.00855 0.00855 0.00855 0.28% Castro::FluxRegCrseInit 9 0.007947 0.007947 0.007947 0.26% MLPoisson::Fapply() 493 0.006749 0.006749 0.006749 0.22% Castro::initData() 1 0.006672 0.006672 0.006672 0.22% FabArray::Xpay() 353 0.00637 0.00637 0.00637 0.21% amrex::average_down 233 0.006147 0.006147 0.006147 0.20% Castro::construct_new_gravity_source() 9 0.005883 0.005883 0.005883 0.19% AmrMesh::MakeNewGrids() 6 0.005444 0.005444 0.005444 0.18% Castro::reset_internal_energy(Fab) 1476 0.005104 0.005104 0.005104 0.17% Castro::wd_update() 6 0.004822 0.004822 0.004822 0.16% Amr::writePlotFile() 2 0.004627 0.004627 0.004627 0.15% MLMG::addInterpCorrection() 155 0.00443 0.00443 0.00443 0.15% Gravity::actual_solve_with_mlmg() 10 0.004093 0.004093 0.004093 0.14% FabArrayBase::CPC::define() 534 0.003733 0.003733 0.003733 0.12% FillPatchInterp(MF) 30 0.003646 0.003646 0.003646 0.12% Castro::construct_old_source() 63 0.00359 0.00359 0.00359 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.00358 0.00358 0.00358 0.12% MultiFab::contains_nan() 18 0.00324 0.00324 0.00324 0.11% FabArray::setDomainBndry() 92 0.003238 0.003238 0.003238 0.11% Castro::apply_tagging_restrictions() 6 0.003105 0.003105 0.003105 0.10% MLCellLinOp::compGrad() 18 0.002953 0.002953 0.002953 0.10% MLCellLinOp::defineBC() 10 0.002688 0.002688 0.002688 0.09% FabArray::mult() 55 0.002573 0.002573 0.002573 0.09% MLCellLinOp::setLevelBC() 18 0.002412 0.002412 0.002412 0.08% Castro::enforce_speed_limit() 87 0.002407 0.002407 0.002407 0.08% check_for_negative_density() 9 0.002281 0.002281 0.002281 0.08% MLCellLinOp::prepareForSolve() 10 0.00201 0.00201 0.00201 0.07% MLMG::prepareForSolve() 10 0.001876 0.001876 0.001876 0.06% Castro::fill_hybrid_hydro_source() 27 0.001676 0.001676 0.001676 0.06% MLPoisson::FFlux() 150 0.001445 0.001445 0.001445 0.05% Gravity::get_new_grav_vector() 10 0.00142 0.00142 0.00142 0.05% MLCellLinOp::updateSolBC() 25 0.001366 0.001366 0.001366 0.05% TagBoxArray::mapPRD 6 0.001324 0.001324 0.001324 0.04% FabArray::norminf(mask) 21 0.001225 0.001225 0.001225 0.04% Gravity::get_old_grav_vector() 9 0.001176 0.001176 0.001176 0.04% FabArrayBase::getCPC() 1522 0.001139 0.001139 0.001139 0.04% Castro::clean_state() 87 0.001052 0.001052 0.001052 0.03% OwnerMask() 6 0.001025 0.001025 0.001025 0.03% MLCellLinOp::updateCorBC() 17 0.001002 0.001002 0.001002 0.03% Castro::build_fine_mask() 44 0.0009743 0.0009743 0.0009743 0.03% average_down_faces 48 0.0008671 0.0008671 0.0008671 0.03% Gravity::multilevel_solve_for_new_phi() 10 0.0008001 0.0008001 0.0008001 0.03% TagBoxArray::collate() 6 0.0007768 0.0007768 0.0007768 0.03% FillPatchTwoLevels 31 0.0007495 0.0007495 0.0007495 0.02% main() 1 0.0007341 0.0007341 0.0007341 0.02% MLCGSolver::bicgstab 31 0.0007011 0.0007011 0.0007011 0.02% Castro::FluxRegFineAdd() 9 0.0006538 0.0006538 0.0006538 0.02% AmrLevel::FillPatch() 62 0.0006335 0.0006335 0.0006335 0.02% MultiFab::max() 18 0.0006173 0.0006173 0.0006173 0.02% FillPatchIterator::Initialize 62 0.000555 0.000555 0.000555 0.02% MultiFab::Multiply() 8 0.0005073 0.0005073 0.0005073 0.02% MLCellLinOp::smooth() 688 0.0004905 0.0004905 0.0004905 0.02% Castro::apply_problem_tags() 6 0.0004631 0.0004631 0.0004631 0.02% FabArrayBase::FB::FB() 37 0.0004501 0.0004501 0.0004501 0.01% Castro::initialize_advance() 9 0.0004294 0.0004294 0.0004294 0.01% MLMG::interpCorrection_1 17 0.0004186 0.0004186 0.0004186 0.01% Gravity::actual_multilevel_solve() 10 0.0004181 0.0004181 0.0004181 0.01% FabArray::FillBoundary() 1728 0.000406 0.000406 0.000406 0.01% Castro::subcycle_advance_ctu() 5 0.0003747 0.0003747 0.0003747 0.01% Castro::linear_to_hybrid_momentum() 1 0.0003727 0.0003727 0.0003727 0.01% Castro::react_state() 18 0.0003719 0.0003719 0.0003719 0.01% Castro::Castro() 5 0.0003543 0.0003543 0.0003543 0.01% Amr::regrid() 5 0.0003541 0.0003541 0.0003541 0.01% Castro::initialize_do_advance() 9 0.0003505 0.0003505 0.0003505 0.01% Castro::finalize_do_advance() 9 0.0003248 0.0003248 0.0003248 0.01% FabArray::ParallelCopy() 954 0.000321 0.000321 0.000321 0.01% FabArrayBase::getFB() 1728 0.0003073 0.0003073 0.0003073 0.01% Castro::do_new_sources() 9 0.0002927 0.0002927 0.0002927 0.01% Castro::post_regrid() 10 0.0002858 0.0002858 0.0002858 0.01% Amr::coarseTimeStep() 5 0.0002677 0.0002677 0.0002677 0.01% Castro::construct_old_gravity() 9 0.0002625 0.0002625 0.0002625 0.01% Gravity::update_max_rhs() 10 0.0002552 0.0002552 0.0002552 0.01% AMRErrorTag::operator() 6 0.0002379 0.0002379 0.0002379 0.01% Castro::do_old_sources() 9 0.0002317 0.0002317 0.0002317 0.01% Castro::post_timestep() 9 0.0002285 0.0002285 0.0002285 0.01% MLCellLinOp::apply() 493 0.0002176 0.0002176 0.0002176 0.01% Castro::construct_new_gravity() 9 0.0002155 0.0002155 0.0002155 0.01% MLLinOp::defineGrids() 10 0.0002006 0.0002006 0.0002006 0.01% Castro::construct_new_rotation_source() 9 0.000198 0.000198 0.000198 0.01% Castro::construct_old_rotation_source() 9 0.0001924 0.0001924 0.0001924 0.01% Castro::do_advance_ctu() 5 0.0001745 0.0001745 0.0001745 0.01% StateData::define() 25 0.0001562 0.0001562 0.0001562 0.01% Castro::enforce_consistent_e() 1 0.0001498 0.0001498 0.0001498 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0001415 0.0001415 0.0001415 0.00% Castro::create_source_corrector() 9 0.0001346 0.0001346 0.0001346 0.00% Castro::finalize_advance() 9 0.0001292 0.0001292 0.0001292 0.00% Castro::computeNewDt() 8 0.0001277 0.0001277 0.0001277 0.00% MLCGSolver::ParallelAllReduce 745 0.0001271 0.0001271 0.0001271 0.00% Castro::advance() 5 0.0001263 0.0001263 0.0001263 0.00% Castro::swap_state_time_levels() 9 0.0001249 0.0001249 0.0001249 0.00% MLMG::mgVcycle() 65 0.0001075 0.0001075 0.0001075 0.00% Castro::buildMetrics() 5 0.0001007 0.0001007 0.0001007 0.00% MLCellLinOp::correctionResidual() 189 9.971e-05 9.971e-05 9.971e-05 0.00% Amr::grid_places() 6 9.084e-05 9.084e-05 9.084e-05 0.00% Amr::timeStep() 9 8.166e-05 8.166e-05 8.166e-05 0.00% AmrLevel::derive() 6 7.912e-05 7.912e-05 7.912e-05 0.00% FillPatchSingleLevel 92 5.7e-05 5.7e-05 5.7e-05 0.00% MLMG::oneIter() 31 5.543e-05 5.543e-05 5.543e-05 0.00% MLMG:computeResOfCorrection() 155 5.232e-05 5.232e-05 5.232e-05 0.00% FabArray::RecvLayoutMask() 8 5.009e-05 5.009e-05 5.009e-05 0.00% Castro::errorEst() 6 4.885e-05 4.885e-05 4.885e-05 0.00% MLMG::solve() 10 4.764e-05 4.764e-05 4.764e-05 0.00% MLCellLinOp::solutionResidual() 71 4.575e-05 4.575e-05 4.575e-05 0.00% AmrLevel::AmrLevel(dm) 5 4.223e-05 4.223e-05 4.223e-05 0.00% MLMG::mgVcycle_bottom 65 4.122e-05 4.122e-05 4.122e-05 0.00% Amr::defBaseLevel() 1 4.084e-05 4.084e-05 4.084e-05 0.00% AmrLevel::FillCoarsePatch() 5 4.049e-05 4.049e-05 4.049e-05 0.00% MLMG::actualBottomSolve() 31 3.897e-05 3.897e-05 3.897e-05 0.00% FillPatchIterator::FillFromLevel0() 31 3.841e-05 3.841e-05 3.841e-05 0.00% Amr::writeSmallPlotFile() 1 3.684e-05 3.684e-05 3.684e-05 0.00% MLMG::mgVcycle_down::0 31 3.588e-05 3.588e-05 3.588e-05 0.00% FillPatchIterator::FillFromTwoLevels() 31 3.315e-05 3.315e-05 3.315e-05 0.00% Gravity::solve_phi_with_mlmg() 10 3.244e-05 3.244e-05 3.244e-05 0.00% MLMG::mgVcycle_down::1 31 3.212e-05 3.212e-05 3.212e-05 0.00% MLMG::mgVcycle_down::3 31 2.974e-05 2.974e-05 2.974e-05 0.00% MLMG::mgVcycle_down::2 31 2.917e-05 2.917e-05 2.917e-05 0.00% MLPoisson::define() 10 2.821e-05 2.821e-05 2.821e-05 0.00% MLMG::computeResWithCrseSolFineCor() 17 2.733e-05 2.733e-05 2.733e-05 0.00% MLMG::mgVcycle_down::4 31 2.65e-05 2.65e-05 2.65e-05 0.00% makeSFC 50 2.534e-05 2.534e-05 2.534e-05 0.00% MLMG::ResNormInf() 54 2.463e-05 2.463e-05 2.463e-05 0.00% MLMG::mgVcycle_up::4 31 2.395e-05 2.395e-05 2.395e-05 0.00% MLMG::mgVcycle_up::0 31 2.195e-05 2.195e-05 2.195e-05 0.00% MLMG::mgVcycle_up::3 31 2.1e-05 2.1e-05 2.1e-05 0.00% FPinfo::FPinfo() 3 2.067e-05 2.067e-05 2.067e-05 0.00% DistributionMapping::Distribute() 52 1.973e-05 1.973e-05 1.973e-05 0.00% MLMG::mgVcycle_up::1 31 1.959e-05 1.959e-05 1.959e-05 0.00% FabArrayBase::TheFPinfo() 25 1.952e-05 1.952e-05 1.952e-05 0.00% MLMG::mgVcycle_up::2 31 1.883e-05 1.883e-05 1.883e-05 0.00% MLLinOp::makeAgglomeratedDMap 10 1.42e-05 1.42e-05 1.42e-05 0.00% Amr::initSubcycle() 1 1.42e-05 1.42e-05 1.42e-05 0.00% MLMG::computeResidual() 31 1.387e-05 1.387e-05 1.387e-05 0.00% MLMG::computeMLResidual() 15 1.372e-05 1.372e-05 1.372e-05 0.00% Amr::FinalizeInit() 1 1.346e-05 1.346e-05 1.346e-05 0.00% MLMG::miniCycle() 34 1.337e-05 1.337e-05 1.337e-05 0.00% ClusterList::intersect() 4 1.325e-05 1.325e-05 1.325e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 1.325e-05 1.325e-05 1.325e-05 0.00% Castro::post_init() 1 1.14e-05 1.14e-05 1.14e-05 0.00% Castro::apply_source_to_state() 18 1.08e-05 1.08e-05 1.08e-05 0.00% Castro::check_for_nan() 18 1.065e-05 1.065e-05 1.065e-05 0.00% MLMG::getGradSolution() 10 9.062e-06 9.062e-06 9.062e-06 0.00% MLPoisson::prepareForSolve() 10 9.024e-06 9.024e-06 9.024e-06 0.00% MLLinOp::define() 10 8.133e-06 8.133e-06 8.133e-06 0.00% Castro::expand_state() 13 7.882e-06 7.882e-06 7.882e-06 0.00% MLMG::MLResNormInf() 15 7.48e-06 7.48e-06 7.48e-06 0.00% Castro::avgDown(state_indx) 20 7.405e-06 7.405e-06 7.405e-06 0.00% Castro::construct_new_hybrid_source() 9 7.282e-06 7.282e-06 7.282e-06 0.00% Gravity::swapTimeLevels() 9 6.976e-06 6.976e-06 6.976e-06 0.00% amrex::average_down_w_geom 20 6.887e-06 6.887e-06 6.887e-06 0.00% Castro::construct_old_hybrid_source() 9 6.769e-06 6.769e-06 6.769e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 6.475e-06 6.475e-06 6.475e-06 0.00% AmrMesh-cluster 4 6.013e-06 6.013e-06 6.013e-06 0.00% Amr::InitializeInit() 1 5.977e-06 5.977e-06 5.977e-06 0.00% MLMG::MLRhsNormInf() 10 5.496e-06 5.496e-06 5.496e-06 0.00% Castro::init(old) 3 5.048e-06 5.048e-06 5.048e-06 0.00% BoxList::parallelComplementIn() 12 4.215e-06 4.215e-06 4.215e-06 0.00% Castro::derive() 6 3.908e-06 3.908e-06 3.908e-06 0.00% Castro::avgDown() 4 3.805e-06 3.805e-06 3.805e-06 0.00% Gravity::set_mass_offset() 6 3.569e-06 3.569e-06 3.569e-06 0.00% BoxList::complementIn 12 3.524e-06 3.524e-06 3.524e-06 0.00% Castro::computeInitialDt() 2 3.007e-06 3.007e-06 3.007e-06 0.00% Castro::retry_advance_ctu() 5 2.974e-06 2.974e-06 2.974e-06 0.00% Castro::init() 1 2.431e-06 2.431e-06 2.431e-06 0.00% Amr::bldFineLevels() 1 2.427e-06 2.427e-06 2.427e-06 0.00% Amr::init() 1 2.185e-06 2.185e-06 2.185e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.936e-06 1.936e-06 1.936e-06 0.00% ClusterList::chop() 4 1.619e-06 1.619e-06 1.619e-06 0.00% Amr::initialInit() 1 1.263e-06 1.263e-06 1.263e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 3.022 3.022 3.022 100.00% Amr::coarseTimeStep() 5 1.442 1.442 1.442 47.72% Amr::timeStep() 9 1.437 1.437 1.437 47.55% Castro::advance() 5 1.217 1.217 1.217 40.27% Castro::subcycle_advance_ctu() 5 1.191 1.191 1.191 39.42% Castro::do_advance_ctu() 5 1.191 1.191 1.191 39.40% Amr::InitAmr() 1 1.041 1.041 1.041 34.45% Castro::construct_ctu_hydro_source() 9 0.6948 0.6948 0.6948 22.99% Amr::init() 1 0.3836 0.3836 0.3836 12.69% Gravity::multilevel_solve_for_new_phi() 10 0.3318 0.3318 0.3318 10.98% Gravity::actual_multilevel_solve() 10 0.331 0.331 0.331 10.95% Gravity::solve_phi_with_mlmg() 10 0.3277 0.3277 0.3277 10.84% Amr::writePlotFile() 2 0.2987 0.2987 0.2987 9.88% VisMF::Write(FabArray) 3 0.2916 0.2916 0.2916 9.65% Gravity::actual_solve_with_mlmg() 10 0.2545 0.2545 0.2545 8.42% Amr::initialInit() 1 0.2395 0.2395 0.2395 7.92% MLMG::solve() 10 0.2026 0.2026 0.2026 6.70% Castro::clean_state() 87 0.1973 0.1973 0.1973 6.53% Castro::construct_new_gravity() 9 0.1941 0.1941 0.1941 6.42% MLMG::oneIter() 31 0.1722 0.1722 0.1722 5.70% Amr::InitializeInit() 1 0.1672 0.1672 0.1672 5.53% Amr::defBaseLevel() 1 0.1672 0.1672 0.1672 5.53% Castro::Castro() 5 0.1573 0.1573 0.1573 5.20% Castro::initMFs() 5 0.1533 0.1533 0.1533 5.07% Amr::regrid() 5 0.1524 0.1524 0.1524 5.04% MLMG::mgVcycle() 65 0.1487 0.1487 0.1487 4.92% Castro::post_regrid() 10 0.137 0.137 0.137 4.53% Castro::computeTemp() 88 0.1037 0.1037 0.1037 3.43% AmrLevel::FillPatch() 62 0.09133 0.09133 0.09133 3.02% MLCellLinOp::applyBC() 1917 0.09123 0.09123 0.09123 3.02% MLCellLinOp::smooth() 688 0.08846 0.08846 0.08846 2.93% Castro::reflux() 4 0.07831 0.07831 0.07831 2.59% FillPatchIterator::Initialize 62 0.07692 0.07692 0.07692 2.54% Gravity::fill_multipole_BCs() 10 0.07267 0.07267 0.07267 2.40% Amr::FinalizeInit() 1 0.07225 0.07225 0.07225 2.39% Castro::normalize_species() 87 0.07161 0.07161 0.07161 2.37% FillPatchSingleLevel 92 0.0686 0.0686 0.0686 2.27% Castro::post_timestep() 9 0.06746 0.06746 0.06746 2.23% FillPatchIterator::FillFromLevel0() 31 0.06604 0.06604 0.06604 2.18% Castro::post_init() 1 0.06151 0.06151 0.06151 2.04% Castro::initialize_do_advance() 9 0.05416 0.05416 0.05416 1.79% Castro::problem_post_timestep() 10 0.05133 0.05133 0.05133 1.70% StateDataPhysBCFunct::() 95 0.04964 0.04964 0.04964 1.64% Castro::do_new_sources() 9 0.04862 0.04862 0.04862 1.61% FabArray::FillBoundary() 1728 0.04392 0.04392 0.04392 1.45% FillBoundary_nowait() 1728 0.04352 0.04352 0.04352 1.44% Castro::do_old_sources() 9 0.04244 0.04244 0.04244 1.40% Castro::expand_state() 13 0.04156 0.04156 0.04156 1.38% MLPoisson::define() 10 0.04103 0.04103 0.04103 1.36% FabArray::setVal() 920 0.03786 0.03786 0.03786 1.25% MLMG::mgVcycle_bottom 65 0.03738 0.03738 0.03738 1.24% Castro::wd_update() 6 0.03524 0.03524 0.03524 1.17% StateData::FillBoundary(geom) 650 0.0351 0.0351 0.0351 1.16% FabArray::ParallelCopy() 954 0.03436 0.03436 0.03436 1.14% FabArray::ParallelCopy_nowait() 954 0.03404 0.03404 0.03404 1.13% Castro::volInBoundary() 42 0.03042 0.03042 0.03042 1.01% MLMG::actualBottomSolve() 31 0.03039 0.03039 0.03039 1.01% MLCGSolver::bicgstab 31 0.0301 0.0301 0.0301 1.00% FluxRegister::Reflux() 24 0.02969 0.02969 0.02969 0.98% MLMG::mgVcycle_up::0 31 0.02929 0.02929 0.02929 0.97% MLMG::mgVcycle_down::0 31 0.02918 0.02918 0.02918 0.97% Castro::reset_internal_energy(MultiFab) 88 0.0283 0.0283 0.0283 0.94% MLCellLinOp::apply() 493 0.02753 0.02753 0.02753 0.91% amrex::Copy() 441 0.02653 0.02653 0.02653 0.88% Castro::initialize_advance() 9 0.02543 0.02543 0.02543 0.84% Castro::construct_new_source() 63 0.02361 0.02361 0.02361 0.78% MLCellLinOp::defineAuxData() 10 0.02345 0.02345 0.02345 0.78% Gravity::get_new_grav_vector() 10 0.02313 0.02313 0.02313 0.77% Castro::enforce_min_density() 87 0.02001 0.02001 0.02001 0.66% MLPoisson::Fsmooth() 1376 0.01988 0.01988 0.01988 0.66% Castro::estTimeStep() 26 0.01913 0.01913 0.01913 0.63% Castro::construct_old_gravity() 9 0.01892 0.01892 0.01892 0.63% Gravity::get_old_grav_vector() 9 0.01866 0.01866 0.01866 0.62% MLCellLinOp::correctionResidual() 189 0.01806 0.01806 0.01806 0.60% amrex::Add() 136 0.01729 0.01729 0.01729 0.57% MLCellLinOp::defineBC() 10 0.01728 0.01728 0.01728 0.57% MLCellLinOp::reflux() 30 0.01622 0.01622 0.01622 0.54% FabArray::Saxpy() 604 0.01582 0.01582 0.01582 0.52% Amr::grid_places() 6 0.01458 0.01458 0.01458 0.48% AmrMesh::MakeNewGrids() 6 0.01449 0.01449 0.01449 0.48% BndryData::define() 26 0.01446 0.01446 0.01446 0.48% MLCellLinOp::solutionResidual() 71 0.01436 0.01436 0.01436 0.48% MLMG:computeResOfCorrection() 155 0.01386 0.01386 0.01386 0.46% MLMG::computeResWithCrseSolFineCor() 17 0.01359 0.01359 0.01359 0.45% MLMG::computeMLResidual() 15 0.01309 0.01309 0.01309 0.43% Castro::construct_old_source() 63 0.01227 0.01227 0.01227 0.41% Castro::initData() 1 0.01173 0.01173 0.01173 0.39% MLMG::mgVcycle_down::1 31 0.01153 0.01153 0.01153 0.38% amrex::average_down 233 0.009848 0.009848 0.009848 0.33% Castro::construct_old_gravity_source() 18 0.009348 0.009348 0.009348 0.31% Castro::finalize_do_advance() 9 0.009178 0.009178 0.009178 0.30% Castro::FluxRegCrseInit 9 0.008982 0.008982 0.008982 0.30% amrex::Dot() 466 0.008931 0.008931 0.008931 0.30% MLMG::prepareForSolve() 10 0.008806 0.008806 0.008806 0.29% FabArray::norminf() 439 0.00855 0.00855 0.00855 0.28% MLMG::mgVcycle_up::1 31 0.008348 0.008348 0.008348 0.28% Castro::computeNewDt() 8 0.007746 0.007746 0.007746 0.26% FillPatchIterator::FillFromTwoLevels() 31 0.007261 0.007261 0.007261 0.24% Amr::bldFineLevels() 1 0.007235 0.007235 0.007235 0.24% FillPatchTwoLevels 31 0.007227 0.007227 0.007227 0.24% MLMG::miniCycle() 34 0.006999 0.006999 0.006999 0.23% MLMG::addInterpCorrection() 155 0.006811 0.006811 0.006811 0.23% MLPoisson::Fapply() 493 0.006749 0.006749 0.006749 0.22% MLMG::mgVcycle_down::2 31 0.006678 0.006678 0.006678 0.22% FabArray::Xpay() 353 0.00637 0.00637 0.00637 0.21% MLMG::mgVcycle_down::3 31 0.006086 0.006086 0.006086 0.20% MLMG::mgVcycle_down::4 31 0.005962 0.005962 0.005962 0.20% Castro::construct_new_gravity_source() 9 0.005883 0.005883 0.005883 0.19% MLMG::computeResidual() 31 0.005772 0.005772 0.005772 0.19% Castro::reset_internal_energy(Fab) 1476 0.005104 0.005104 0.005104 0.17% MLCGSolver::ParallelAllReduce 745 0.005075 0.005075 0.005075 0.17% FabArrayBase::getCPC() 1522 0.004872 0.004872 0.004872 0.16% MLMG::mgVcycle_up::4 31 0.00482 0.00482 0.00482 0.16% Castro::errorEst() 6 0.004785 0.004785 0.004785 0.16% MLMG::mgVcycle_up::2 31 0.004661 0.004661 0.004661 0.15% MLMG::mgVcycle_up::3 31 0.004637 0.004637 0.004637 0.15% MLMG::getGradSolution() 10 0.004415 0.004415 0.004415 0.15% MLCellLinOp::compGrad() 18 0.004406 0.004406 0.004406 0.15% Castro::apply_source_to_state() 18 0.004229 0.004229 0.004229 0.14% FabArrayBase::CPC::define() 534 0.003733 0.003733 0.003733 0.12% FillPatchInterp(MF) 30 0.003646 0.003646 0.003646 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.00358 0.00358 0.00358 0.12% MLMG::computeResWithCrseCorFineCor() 17 0.003488 0.003488 0.003488 0.12% MLCellLinOp::updateSolBC() 25 0.003281 0.003281 0.003281 0.11% Castro::check_for_nan() 18 0.00325 0.00325 0.00325 0.11% MultiFab::contains_nan() 18 0.00324 0.00324 0.00324 0.11% FabArray::setDomainBndry() 92 0.003238 0.003238 0.003238 0.11% TagBoxArray::mapPRD 6 0.003172 0.003172 0.003172 0.10% Castro::apply_tagging_restrictions() 6 0.003105 0.003105 0.003105 0.10% Castro::init(old) 3 0.003 0.003 0.003 0.10% Castro::computeInitialDt() 2 0.002663 0.002663 0.002663 0.09% FabArray::mult() 55 0.002573 0.002573 0.002573 0.09% Castro::buildMetrics() 5 0.002486 0.002486 0.002486 0.08% MLCellLinOp::setLevelBC() 18 0.002412 0.002412 0.002412 0.08% Castro::enforce_speed_limit() 87 0.002407 0.002407 0.002407 0.08% MLCellLinOp::updateCorBC() 17 0.002282 0.002282 0.002282 0.08% check_for_negative_density() 9 0.002281 0.002281 0.002281 0.08% MLMG::ResNormInf() 54 0.002095 0.002095 0.002095 0.07% Gravity::update_max_rhs() 10 0.002036 0.002036 0.002036 0.07% MLPoisson::prepareForSolve() 10 0.002019 0.002019 0.002019 0.07% MLCellLinOp::prepareForSolve() 10 0.00201 0.00201 0.00201 0.07% Gravity::average_fine_ec_onto_crse_ec() 8 0.001934 0.001934 0.001934 0.06% Castro::fill_hybrid_hydro_source() 27 0.001676 0.001676 0.001676 0.06% MLPoisson::FFlux() 150 0.001445 0.001445 0.001445 0.05% average_down_faces 48 0.001269 0.001269 0.001269 0.04% FabArray::norminf(mask) 21 0.001225 0.001225 0.001225 0.04% Castro::avgDown() 4 0.001179 0.001179 0.001179 0.04% Castro::avgDown(state_indx) 20 0.001175 0.001175 0.001175 0.04% amrex::average_down_w_geom 20 0.001167 0.001167 0.001167 0.04% MLMG::MLResNormInf() 15 0.001147 0.001147 0.001147 0.04% Castro::construct_new_hybrid_source() 9 0.00104 0.00104 0.00104 0.03% OwnerMask() 6 0.001025 0.001025 0.001025 0.03% Castro::build_fine_mask() 44 0.0009743 0.0009743 0.0009743 0.03% MLMG::interpCorrection_1 17 0.0009721 0.0009721 0.0009721 0.03% Castro::init() 1 0.0009484 0.0009484 0.0009484 0.03% AmrLevel::FillCoarsePatch() 5 0.000946 0.000946 0.000946 0.03% Castro::derive() 6 0.0009307 0.0009307 0.0009307 0.03% AmrLevel::derive() 6 0.0009268 0.0009268 0.0009268 0.03% TagBoxArray::collate() 6 0.0007768 0.0007768 0.0007768 0.03% FabArrayBase::getFB() 1728 0.0007573 0.0007573 0.0007573 0.03% Castro::react_state() 18 0.0007232 0.0007232 0.0007232 0.02% Gravity::swapTimeLevels() 9 0.0006789 0.0006789 0.0006789 0.02% Castro::FluxRegFineAdd() 9 0.0006538 0.0006538 0.0006538 0.02% Castro::construct_old_hybrid_source() 9 0.0006498 0.0006498 0.0006498 0.02% MultiFab::max() 18 0.0006173 0.0006173 0.0006173 0.02% MLMG::MLRhsNormInf() 10 0.0006019 0.0006019 0.0006019 0.02% MultiFab::Multiply() 8 0.0005073 0.0005073 0.0005073 0.02% Castro::apply_problem_tags() 6 0.0004631 0.0004631 0.0004631 0.02% FabArrayBase::FB::FB() 37 0.0004501 0.0004501 0.0004501 0.01% Castro::linear_to_hybrid_momentum() 1 0.0003727 0.0003727 0.0003727 0.01% MLLinOp::define() 10 0.000268 0.000268 0.000268 0.01% MLLinOp::defineGrids() 10 0.0002598 0.0002598 0.0002598 0.01% AMRErrorTag::operator() 6 0.0002379 0.0002379 0.0002379 0.01% AmrLevel::AmrLevel(dm) 5 0.0001985 0.0001985 0.0001985 0.01% Castro::construct_new_rotation_source() 9 0.000198 0.000198 0.000198 0.01% Castro::construct_old_rotation_source() 9 0.0001924 0.0001924 0.0001924 0.01% StateData::define() 25 0.0001562 0.0001562 0.0001562 0.01% Castro::enforce_consistent_e() 1 0.0001498 0.0001498 0.0001498 0.00% Castro::create_source_corrector() 9 0.0001346 0.0001346 0.0001346 0.00% Castro::finalize_advance() 9 0.0001292 0.0001292 0.0001292 0.00% Castro::swap_state_time_levels() 9 0.0001249 0.0001249 0.0001249 0.00% MLLinOp::makeAgglomeratedDMap 10 5.734e-05 5.734e-05 5.734e-05 0.00% FabArray::RecvLayoutMask() 8 5.009e-05 5.009e-05 5.009e-05 0.00% makeSFC 50 4.313e-05 4.313e-05 4.313e-05 0.00% FabArrayBase::TheFPinfo() 25 4.018e-05 4.018e-05 4.018e-05 0.00% Amr::writeSmallPlotFile() 1 3.684e-05 3.684e-05 3.684e-05 0.00% AmrMesh-cluster 4 2.088e-05 2.088e-05 2.088e-05 0.00% FPinfo::FPinfo() 3 2.067e-05 2.067e-05 2.067e-05 0.00% DistributionMapping::Distribute() 52 1.973e-05 1.973e-05 1.973e-05 0.00% Amr::initSubcycle() 1 1.42e-05 1.42e-05 1.42e-05 0.00% ClusterList::intersect() 4 1.325e-05 1.325e-05 1.325e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 8.414e-06 8.414e-06 8.414e-06 0.00% BoxList::parallelComplementIn() 12 7.739e-06 7.739e-06 7.739e-06 0.00% Gravity::set_mass_offset() 6 3.569e-06 3.569e-06 3.569e-06 0.00% BoxList::complementIn 12 3.524e-06 3.524e-06 3.524e-06 0.00% Castro::retry_advance_ctu() 5 2.974e-06 2.974e-06 2.974e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.936e-06 1.936e-06 1.936e-06 0.00% ClusterList::chop() 4 1.619e-06 1.619e-06 1.619e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Device Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Arena::Initialize() 1 1 1595 KiB 9037 MiB Castro::reflux() 436 436 14 MiB 666 MiB Castro::initMFs() 258 258 345 MiB 576 MiB Castro::swap_state_time_levels() 175 175 176 MiB 340 MiB StateData::define() 175 175 218 MiB 340 MiB Castro::initialize_do_advance() 143 143 120 MiB 325 MiB FillPatchIterator::Initialize 899 899 4192 KiB 303 MiB Amr::writePlotFile() 56 56 17 MiB 189 MiB Castro::problem_post_timestep() 162 162 817 KiB 167 MiB Castro::FluxRegCrseInit 660 660 465 KiB 167 MiB FluxRegister::Reflux() 648 648 1595 KiB 160 MiB Castro::initialize_advance() 143 143 48 MiB 121 MiB Castro::construct_new_source() 143 143 190 KiB 67 MiB Castro::construct_old_source() 143 143 128 KiB 67 MiB Castro::buildMetrics() 140 140 36 MiB 61 MiB Gravity::multilevel_solve_for_new_phi() 858 858 27 MiB 60 MiB Castro::Castro() 210 210 4066 KiB 50 MiB MLMG::prepareForSolve() 1902 1902 2882 KiB 42 MiB Gravity::get_old_grav_vector() 155 155 202 KiB 33 MiB Gravity::get_new_grav_vector() 188 188 250 KiB 33 MiB ResizeRandomSeed 1 1 30 MiB 30 MiB MLCellLinOp::defineAuxData() 5608 5608 913 KiB 16 MiB FillPatchTwoLevels 300 300 7819 B 11 MiB FillPatchInterp(MF) 330 330 4895 B 7955 KiB Gravity::fill_multipole_BCs() 346 346 134 KiB 7469 KiB Gravity::update_max_rhs() 286 286 4638 B 7424 KiB Gravity::actual_multilevel_solve() 286 286 795 KiB 7424 KiB Castro::build_fine_mask() 224 224 1288 KiB 6912 KiB AmrLevel::derive() 162 162 2669 B 6912 KiB BndryData::define() 3624 3624 299 KiB 4243 KiB MLCellLinOp::reflux() 450 450 6212 B 2376 KiB Castro::wd_update() 6 6 25 KiB 2240 KiB AmrLevel::FillCoarsePatch() 10 10 452 B 2095 KiB amrex::average_down 1483 1483 2382 B 1474 KiB AmrMesh::MakeNewGrids() 324 324 5045 B 1041 KiB MLMG::addInterpCorrection() 1240 1240 819 B 867 KiB MLCellLinOp::defineBC() 276 276 3944 B 689 KiB Castro::estTimeStep() 26 26 2767 B 480 KiB VisMF::Write(FabArray) 1568 1568 2367 B 320 KiB Castro::normalize_species() 87 87 7556 B 320 KiB Castro::volInBoundary() 70 70 3265 B 320 KiB Gravity::average_fine_ec_onto_crse_ec() 48 48 128 B 204 KiB FabArray::norminf(mask) 29 29 193 B 163 KiB amrex::Dot() 559 559 439 B 160 KiB FabArray::norminf() 470 470 437 B 160 KiB MultiFab::max() 18 18 32 B 160 KiB check_for_negative_density() 9 9 121 B 160 KiB Castro::initData() 1 1 46 B 160 KiB FabArray::setVal() 419 419 54 KiB 120 KiB MLMG::interpCorrection_1 34 34 28 B 91 KiB average_down_faces 99 99 630 B 70 KiB FabArray::ParallelCopy_nowait() 942 942 288 B 57 KiB MLPoisson::Fsmooth() 228 228 3306 B 55 KiB FillBoundary_nowait() 751 751 470 B 53 KiB TagBoxArray::mapPRD 324 324 42 B 45 KiB OwnerMask() 168 168 12 B 38 KiB MLCellLinOp::applyBC() 3834 3834 147 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9464 B 25 KiB MultiFab::contains_nan() 18 18 21 B 20 KiB StateData::FillBoundary(geom) 2538 2538 45 B 13 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B MultiFab::Multiply() 8 8 643 B 3456 B MLCGSolver::bicgstab 155 155 26 B 2688 B TagBoxArray::collate() 14 14 0 B 640 B ------------------------------------------------------------------------------ Managed Memory Usage: ----------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem ----------------------------------------------------------------- The_Managed_Arena::Initialize() 1 1 160 B 8192 KiB ----------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Pinned_Arena::Initialize() 1 1 19 KiB 8192 KiB VisMF::Write(FabArray) 1624 1624 614 KiB 7168 KiB Gravity::fill_multipole_BCs() 30 30 1299 B 278 KiB FabArray::setVal() 419 419 54 KiB 120 KiB FabArray::ParallelCopy_nowait() 942 942 289 B 57 KiB MLPoisson::Fsmooth() 228 228 3306 B 55 KiB FillBoundary_nowait() 751 751 471 B 53 KiB MLCellLinOp::applyBC() 1917 1917 147 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9464 B 25 KiB Gravity::get_new_grav_vector() 18 18 3368 B 20 KiB Gravity::get_old_grav_vector() 12 12 6221 B 20 KiB OwnerMask() 6 6 0 B 15 KiB StateData::FillBoundary(geom) 2538 2538 45 B 13 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B amrex::average_down 149 149 1329 B 6304 B MLMG::prepareForSolve() 18 18 245 B 3712 B FabArray::norminf(mask) 29 29 131 B 3472 B MLMG::addInterpCorrection() 62 62 2 B 3456 B MultiFab::Multiply() 8 8 643 B 3456 B Castro::FluxRegCrseInit 12 12 9 B 3456 B FillPatchInterp(MF) 140 140 3 B 3232 B MLCellLinOp::setLevelBC() 68 68 0 B 2592 B average_down_faces 51 51 603 B 2272 B TagBoxArray::collate() 16 16 0 B 640 B amrex::Dot() 559 559 3 B 400 B Castro::build_fine_mask() 8 8 0 B 208 B MLCellLinOp::defineAuxData() 8 8 0 B 208 B Castro::reflux() 4 4 0 B 208 B FabArray::norminf() 470 470 1 B 144 B Castro::wd_update() 6 6 1 B 112 B Castro::problem_post_timestep() 19 19 0 B 48 B Castro::estTimeStep() 26 26 0 B 32 B MultiFab::contains_nan() 18 18 0 B 16 B MultiFab::max() 18 18 0 B 16 B Castro::volInBoundary() 70 70 0 B 16 B Castro::normalize_species() 87 87 0 B 16 B Castro::initData() 1 1 0 B 16 B check_for_negative_density() 9 9 0 B 16 B ------------------------------------------------------------------------------ Total GPU global memory (MB): 12049 Free GPU global memory (MB): 1856 [The Arena] space allocated (MB): 9037 [The Arena] space used (MB): 0 [The Managed Arena] space allocated (MB): 8 [The Managed Arena] space used (MB): 0 [The Pinned Arena] space allocated (MB): 8 [The Pinned Arena] space used (MB): 0 AMReX (24.03-8-gae3af4339425) finalized