Initializing AMReX (24.04-14-g6c6247554f27)... Initializing CUDA... CUDA initialized with 1 device. AMReX (24.04-14-g6c6247554f27) initialized Starting run at 09:48:06 UTC on 2024-04-11. Successfully read inputs file ... Castro git describe: 24.04-13-g88d497647 AMReX git describe: 24.04-14-g6c6247554 Microphysics git describe: 24.04-7-gd4312a6c reading extern runtime parameters ... 13 Species: He4 C12 O16 Ne20 Mg24 Si28 S32 Ar36 Ca40 Ti44 Cr48 Fe52 Ni56 Successfully read inputs file ... Creating primary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Creating secondary with CO core with mass fractions C = 0.4and O = 0.6 and a He shell of solar mass =0. Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm. Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU. The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU. The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU. The initial orbital period is 30.543687 s. The initial orbital speed of the primary is -1.831e+08 cm/s. The initial orbital speed of the secondary is 1.831e+08 cm/s. Castro::numpts_1d at level 0 is 174 Setting the gravity type to PoissonGrav Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 grid_places() time: 0.003270026 new finest: 0 Now regridding at level lbase = 0 grid_places() time: 0.001685776 new finest: 0 STEP = 0 TIME = 0 : REGRID with lbase = 0 Gravity::fill_multipole_BCs() time = 0.009079554 Gravity::multilevel_solve_for_new_phi() time = 0.053522059 INITIAL GRIDS Level 0 27 grids 884736 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = wdmerger_3D_plt00000 Write plotfile time = 0.146760156 seconds ...estimated hydro-limited timestep at level 0: 0.05426072637 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05426072637 [Level 0 step 1] ADVANCE at time 0 with dt = 0.005426072637  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003352777 on level 0 Castro::do_old_sources() time = 0.009237218 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.136308626 on level 0 ... new-time composite Poisson gravity solve from level 0 to level 0 Gravity::fill_multipole_BCs() time = 0.006817636 Gravity::multilevel_solve_for_new_phi() time = 0.041144806 Castro::construct_new_gravity() time = 0.044798871 on level 0 Castro::do_new_sources() time = 0.008082432 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 3.95035957 [Level 0 step 1] Advanced 884736 cells Updating the rotational period from 30.54368673 s to 30.46017574 s. [STEP 1] Coarse TimeStep time: 0.233385045 [STEP 1] FAB kilobyte spread across MPI nodes: [2005477 ... 2005477] STEP = 1 TIME = 0.005426072637 DT = 0.005426072637 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.05461684804 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637 Now regridding at level lbase = 0 grid_places() time: 0.002268752 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007414228 Gravity::multilevel_solve_for_new_phi() time = 0.06084807 STEP = 1 TIME = 0.005426072637 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.05461684804 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461684804 ...estimated hydro-limited timestep at level 1: 2.526135647e-09 ...hydro-limited CFL timestep constrained at (i,j,k) = (104,88,99) Castro::estTimeStep (hydro-limited) at level 1: estdt = 2.526135647e-09 [Level 0 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135647e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003418797 on level 0 Castro::do_old_sources() time = 0.006850884 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.164092562 on level 0 Castro::construct_old_gravity() time = 0.000397601 on level 1 Castro::do_old_sources() time = 0.001852476 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.009788952 on level 1 Castro::reflux() at level 0 : time = 0.018408304 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007232281 Gravity::multilevel_solve_for_new_phi() time = 0.041815436 Castro::construct_new_gravity() time = 0.045385665 on level 0 Castro::do_new_sources() time = 0.006842404 on level 0 ...estimated hydro-limited timestep at level 0: 0.05461716053 ...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.05461716053 Castro::construct_new_gravity() time = 0.0003995 on level 1 Castro::do_new_sources() time = 0.002574928 on level 1 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.314875656 [Level 0 step 2] Advanced 884736 cells [Level 1 step 2] ADVANCE at time 0.005426072637 with dt = 2.526135647e-09 Advance at this level has already been completed. [Level 1 step 2] Advanced 65536 cells Updating the rotational period from 30.46017574 s to 30.52759479 s. [STEP 2] Coarse TimeStep time: 0.370664089 [STEP 2] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 2 TIME = 0.005426075163 DT = 2.526135647e-09 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0008044208529 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.697001619e-07 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09 Now regridding at level lbase = 0 grid_places() time: 0.002029769 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007261465 Gravity::multilevel_solve_for_new_phi() time = 0.015304789 STEP = 2 TIME = 0.005426075163 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0008044208529 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044208529 ...estimated hydro-limited timestep at level 1: 6.697001619e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.697001619e-07 [Level 0 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778749212e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003494165 on level 0 Castro::do_old_sources() time = 0.006782282 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.179058055 on level 0 Castro::construct_old_gravity() time = 0.000369572 on level 1 Castro::do_old_sources() time = 0.001496752 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.008251961 on level 1 Castro::reflux() at level 0 : time = 0.017942363 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007275954 Gravity::multilevel_solve_for_new_phi() time = 0.029258756 Castro::construct_new_gravity() time = 0.032897769 on level 0 Castro::do_new_sources() time = 0.007088943 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008044237461 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008044237461 Castro::construct_new_gravity() time = 0.000392388 on level 1 Castro::do_new_sources() time = 0.002562503 on level 1 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.294158454 [Level 0 step 3] Advanced 884736 cells [Level 1 step 3] ADVANCE at time 0.005426075163 with dt = 2.778749212e-09 Advance at this level has already been completed. [Level 1 step 3] Advanced 65536 cells Updating the rotational period from 30.52759479 s to 30.52761298 s. [STEP 3] Coarse TimeStep time: 0.331423718 [STEP 3] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 3 TIME = 0.005426077942 DT = 2.778749212e-09 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.000804308035 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 6.965184638e-07 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09 Now regridding at level lbase = 0 grid_places() time: 0.00211592 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007262101 Gravity::multilevel_solve_for_new_phi() time = 0.015378997 STEP = 3 TIME = 0.005426077942 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.000804308035 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.000804308035 ...estimated hydro-limited timestep at level 1: 6.965184638e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 6.965184638e-07 [Level 0 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056624133e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003401161 on level 0 Castro::do_old_sources() time = 0.006698226 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.167935339 on level 0 Castro::construct_old_gravity() time = 0.000361395 on level 1 Castro::do_old_sources() time = 0.001742287 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.007771007 on level 1 Castro::reflux() at level 0 : time = 0.017781032 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007242559 Gravity::multilevel_solve_for_new_phi() time = 0.029154772 Castro::construct_new_gravity() time = 0.032727186 on level 0 Castro::do_new_sources() time = 0.006807983 on level 0 ...estimated hydro-limited timestep at level 0: 0.0008043112215 ...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0008043112215 Castro::construct_new_gravity() time = 0.000430399 on level 1 Castro::do_new_sources() time = 0.002532542 on level 1 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.493411433 [Level 0 step 4] Advanced 884736 cells [Level 1 step 4] ADVANCE at time 0.005426077942 with dt = 3.056624133e-09 Advance at this level has already been completed. [Level 1 step 4] Advanced 65536 cells Updating the rotational period from 30.52761298 s to 30.52761297 s. [STEP 4] Coarse TimeStep time: 0.310990154 [STEP 4] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 4 TIME = 0.005426080998 DT = 3.056624133e-09 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.0002497928903 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Castro::compute_new_dt : limiting dt at level 1 ... new dt computed: 7.255166705e-07 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09 Now regridding at level lbase = 0 grid_places() time: 0.002120932 new finest: 1 Castro::numpts_1d at level 1 is 340 Gravity::fill_multipole_BCs() time = 0.007253005 Gravity::multilevel_solve_for_new_phi() time = 0.015420176 STEP = 4 TIME = 0.005426080998 : REGRID with lbase = 0 Level 1 2 grids 65536 cells 0.9259259259 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 ...estimated hydro-limited timestep at level 0: 0.0002497928903 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0002497928903 ...estimated hydro-limited timestep at level 1: 7.255166705e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.255166705e-07 [Level 0 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286546e-09 Estimated number of subcycles remaining: 1 Castro::construct_old_gravity() time = 0.003379189 on level 0 Castro::do_old_sources() time = 0.006775782 on level 0 ... Entering construct_ctu_hydro_source() on level 0 ... Leaving construct_ctu_hydro_source() on level 0 Castro::construct_ctu_hydro_source() time = 0.179938981 on level 0 Castro::construct_old_gravity() time = 0.000343644 on level 1 Castro::do_old_sources() time = 0.001744788 on level 1 ... Entering construct_ctu_hydro_source() on level 1 ... Leaving construct_ctu_hydro_source() on level 1 Castro::construct_ctu_hydro_source() time = 0.006653019 on level 1 Castro::reflux() at level 0 : time = 0.018162705 ... new-time composite Poisson gravity solve from level 0 to level 1 Gravity::fill_multipole_BCs() time = 0.007253281 Gravity::multilevel_solve_for_new_phi() time = 0.029453509 Castro::construct_new_gravity() time = 0.03292397 on level 0 Castro::do_new_sources() time = 0.006603377 on level 0 ...estimated hydro-limited timestep at level 0: 0.0006805523364 ...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46) Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.0006805523364 Castro::construct_new_gravity() time = 0.000398391 on level 1 Castro::do_new_sources() time = 0.002342957 on level 1 ...estimated hydro-limited timestep at level 1: 7.553385904e-07 ...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90) Castro::estTimeStep (hydro-limited) at level 1: estdt = 7.553385904e-07  Subcycle completed Subcycling complete Zones advanced per microsecond from level 0 to level 1: 3.349759601 [Level 0 step 5] Advanced 884736 cells [Level 1 step 5] ADVANCE at time 0.005426080998 with dt = 3.362286546e-09 Advance at this level has already been completed. [Level 1 step 5] Advanced 65536 cells Updating the rotational period from 30.52761297 s to 30.52761296 s. [STEP 5] Coarse TimeStep time: 0.327023062 [STEP 5] FAB kilobyte spread across MPI nodes: [2667545 ... 2667545] STEP = 5 TIME = 0.005426084361 DT = 3.362286546e-09 PLOTFILE: file = wdmerger_3D_plt00005 Write plotfile time = 0.155274987 seconds Ending run at 09:48:09 UTC on 2024-04-11. Run time = 3.224523169 Run time without initialization = 1.729649026 Average number of zones advanced per microsecond: 2.861 Average number of zones advanced per microsecond per rank: 2.861 CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9476456448 TinyProfiler total time across processes [min...avg...max]: 3.225 ... 3.225 ... 3.225 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Amr::InitAmr() 1 1.1103 1.1103 1.1103 34.43% Castro::construct_ctu_hydro_source() 9 0.8232 0.8232 0.8232 25.53% VisMF::Write(FabArray) 3 0.2953 0.2953 0.2953 9.16% Castro::initMFs() 5 0.151 0.151 0.151 4.68% Gravity::fill_multipole_BCs() 10 0.07213 0.07213 0.07213 2.24% Castro::normalize_species() 87 0.06504 0.06504 0.06504 2.02% Castro::computeTemp() 88 0.05998 0.05998 0.05998 1.86% MLCellLinOp::applyBC() 1917 0.04813 0.04813 0.04813 1.49% FillBoundary_nowait() 1728 0.04339 0.04339 0.04339 1.35% FabArray::setVal() 920 0.03837 0.03837 0.03837 1.19% StateData::FillBoundary(geom) 650 0.03485 0.03485 0.03485 1.08% FabArray::ParallelCopy_nowait() 954 0.03045 0.03045 0.03045 0.94% Castro::volInBoundary() 42 0.02962 0.02962 0.02962 0.92% amrex::Copy() 441 0.02674 0.02674 0.02674 0.83% Castro::reset_internal_energy(MultiFab) 88 0.02284 0.02284 0.02284 0.71% FluxRegister::Reflux() 24 0.02242 0.02242 0.02242 0.70% MLCellLinOp::defineAuxData() 10 0.02168 0.02168 0.02168 0.67% amrex::Add() 136 0.01727 0.01727 0.01727 0.54% Castro::estTimeStep() 26 0.01653 0.01653 0.01653 0.51% FabArray::Saxpy() 604 0.016 0.016 0.016 0.50% StateDataPhysBCFunct::() 95 0.01563 0.01563 0.01563 0.48% Castro::construct_new_source() 63 0.01556 0.01556 0.01556 0.48% Castro::problem_post_timestep() 10 0.01376 0.01376 0.01376 0.43% BndryData::define() 26 0.01335 0.01335 0.01335 0.41% MLPoisson::Fsmooth() 1376 0.0127 0.0127 0.0127 0.39% Castro::enforce_min_density() 87 0.01225 0.01225 0.01225 0.38% Castro::reflux() 4 0.01153 0.01153 0.01153 0.36% MLCellLinOp::reflux() 30 0.00969 0.00969 0.00969 0.30% amrex::Dot() 466 0.009276 0.009276 0.009276 0.29% FabArray::norminf() 439 0.00878 0.00878 0.00878 0.27% Castro::FluxRegCrseInit 9 0.007961 0.007961 0.007961 0.25% Castro::construct_old_gravity_source() 18 0.007934 0.007934 0.007934 0.25% MLPoisson::Fapply() 493 0.0069 0.0069 0.0069 0.21% FabArray::Xpay() 353 0.006496 0.006496 0.006496 0.20% Castro::initData() 1 0.006314 0.006314 0.006314 0.20% amrex::average_down 233 0.006183 0.006183 0.006183 0.19% Castro::construct_new_gravity_source() 9 0.005445 0.005445 0.005445 0.17% AmrMesh::MakeNewGrids() 6 0.005367 0.005367 0.005367 0.17% Castro::reset_internal_energy(Fab) 1476 0.004851 0.004851 0.004851 0.15% Castro::wd_update() 6 0.0048 0.0048 0.0048 0.15% Castro::construct_old_source() 63 0.004717 0.004717 0.004717 0.15% MLMG::addInterpCorrection() 155 0.0045 0.0045 0.0045 0.14% Amr::writePlotFile() 2 0.004441 0.004441 0.004441 0.14% Castro::enforce_speed_limit() 87 0.004385 0.004385 0.004385 0.14% FabArrayBase::CPC::define() 534 0.003755 0.003755 0.003755 0.12% Gravity::actual_solve_with_mlmg() 10 0.003744 0.003744 0.003744 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.003689 0.003689 0.003689 0.11% FillPatchInterp(MF) 30 0.003679 0.003679 0.003679 0.11% FabArray::setDomainBndry() 92 0.003346 0.003346 0.003346 0.10% MultiFab::contains_nan() 18 0.003248 0.003248 0.003248 0.10% check_for_negative_density() 9 0.003245 0.003245 0.003245 0.10% MLCellLinOp::compGrad() 18 0.002919 0.002919 0.002919 0.09% MLCellLinOp::defineBC() 10 0.002763 0.002763 0.002763 0.09% FabArray::mult() 55 0.002595 0.002595 0.002595 0.08% MLCellLinOp::setLevelBC() 18 0.002527 0.002527 0.002527 0.08% Castro::apply_tagging_restrictions() 6 0.002016 0.002016 0.002016 0.06% MLCellLinOp::prepareForSolve() 10 0.001999 0.001999 0.001999 0.06% MLMG::prepareForSolve() 10 0.001872 0.001872 0.001872 0.06% Castro::fill_hybrid_hydro_source() 27 0.001658 0.001658 0.001658 0.05% MLPoisson::FFlux() 150 0.001492 0.001492 0.001492 0.05% MLCellLinOp::updateSolBC() 25 0.001406 0.001406 0.001406 0.04% TagBoxArray::mapPRD 6 0.001364 0.001364 0.001364 0.04% Gravity::get_new_grav_vector() 10 0.001243 0.001243 0.001243 0.04% FabArray::norminf(mask) 21 0.001225 0.001225 0.001225 0.04% FabArrayBase::getCPC() 1522 0.001163 0.001163 0.001163 0.04% Gravity::get_old_grav_vector() 9 0.001055 0.001055 0.001055 0.03% MLCellLinOp::updateCorBC() 17 0.001028 0.001028 0.001028 0.03% OwnerMask() 6 0.001012 0.001012 0.001012 0.03% Castro::clean_state() 87 0.001006 0.001006 0.001006 0.03% Castro::build_fine_mask() 44 0.0009823 0.0009823 0.0009823 0.03% Castro::post_timestep() 9 0.0009317 0.0009317 0.0009317 0.03% Castro::finalize_advance() 9 0.0009315 0.0009315 0.0009315 0.03% average_down_faces 48 0.0008862 0.0008862 0.0008862 0.03% Gravity::multilevel_solve_for_new_phi() 10 0.0008512 0.0008512 0.0008512 0.03% TagBoxArray::collate() 6 0.0008053 0.0008053 0.0008053 0.02% FillPatchTwoLevels 31 0.000759 0.000759 0.000759 0.02% MLCGSolver::bicgstab 31 0.0007391 0.0007391 0.0007391 0.02% Castro::FluxRegFineAdd() 9 0.0007204 0.0007204 0.0007204 0.02% main() 1 0.0006683 0.0006683 0.0006683 0.02% AmrLevel::FillPatch() 62 0.0006303 0.0006303 0.0006303 0.02% MultiFab::max() 18 0.0006104 0.0006104 0.0006104 0.02% FillPatchIterator::Initialize 62 0.0005781 0.0005781 0.0005781 0.02% MLCellLinOp::smooth() 688 0.0005273 0.0005273 0.0005273 0.02% Castro::apply_problem_tags() 6 0.0005233 0.0005233 0.0005233 0.02% MultiFab::Multiply() 8 0.0005067 0.0005067 0.0005067 0.02% FabArrayBase::FB::FB() 37 0.0004364 0.0004364 0.0004364 0.01% Castro::Castro() 5 0.0004325 0.0004325 0.0004325 0.01% FabArray::FillBoundary() 1728 0.0004188 0.0004188 0.0004188 0.01% MLMG::interpCorrection_1 17 0.0004155 0.0004155 0.0004155 0.01% Amr::regrid() 5 0.0003878 0.0003878 0.0003878 0.01% Castro::linear_to_hybrid_momentum() 1 0.00037 0.00037 0.00037 0.01% FabArray::ParallelCopy() 954 0.0003386 0.0003386 0.0003386 0.01% Castro::subcycle_advance_ctu() 5 0.0003257 0.0003257 0.0003257 0.01% FabArrayBase::getFB() 1728 0.0002982 0.0002982 0.0002982 0.01% Castro::initialize_advance() 9 0.0002754 0.0002754 0.0002754 0.01% Castro::initialize_do_advance() 9 0.00027 0.00027 0.00027 0.01% Amr::coarseTimeStep() 5 0.000256 0.000256 0.000256 0.01% Gravity::actual_multilevel_solve() 10 0.0002437 0.0002437 0.0002437 0.01% Gravity::update_max_rhs() 10 0.0002417 0.0002417 0.0002417 0.01% AMRErrorTag::operator() 6 0.0002317 0.0002317 0.0002317 0.01% MLCellLinOp::apply() 493 0.0002258 0.0002258 0.0002258 0.01% Castro::construct_new_rotation_source() 9 0.0002188 0.0002188 0.0002188 0.01% Castro::construct_old_rotation_source() 9 0.0002036 0.0002036 0.0002036 0.01% MLLinOp::defineGrids() 10 0.0002032 0.0002032 0.0002032 0.01% Castro::finalize_do_advance() 9 0.0001737 0.0001737 0.0001737 0.01% StateData::define() 25 0.0001567 0.0001567 0.0001567 0.00% Castro::do_advance_ctu() 5 0.0001497 0.0001497 0.0001497 0.00% Castro::construct_new_gravity() 9 0.0001482 0.0001482 0.0001482 0.00% Castro::enforce_consistent_e() 1 0.0001399 0.0001399 0.0001399 0.00% MLCGSolver::ParallelAllReduce 745 0.0001363 0.0001363 0.0001363 0.00% Gravity::average_fine_ec_onto_crse_ec() 8 0.0001324 0.0001324 0.0001324 0.00% Castro::swap_state_time_levels() 9 0.0001267 0.0001267 0.0001267 0.00% Castro::do_new_sources() 9 0.0001227 0.0001227 0.0001227 0.00% Castro::computeNewDt() 8 0.0001171 0.0001171 0.0001171 0.00% Castro::buildMetrics() 5 0.0001112 0.0001112 0.0001112 0.00% MLMG::mgVcycle() 65 0.0001112 0.0001112 0.0001112 0.00% Castro::do_old_sources() 9 0.0001028 0.0001028 0.0001028 0.00% Amr::grid_places() 6 0.0001021 0.0001021 0.0001021 0.00% Amr::timeStep() 9 0.0001008 0.0001008 0.0001008 0.00% MLCellLinOp::correctionResidual() 189 0.0001 0.0001 0.0001 0.00% Castro::construct_old_gravity() 9 9.231e-05 9.231e-05 9.231e-05 0.00% Castro::react_state() 18 7.743e-05 7.743e-05 7.743e-05 0.00% AmrLevel::derive() 6 7.688e-05 7.688e-05 7.688e-05 0.00% Castro::advance() 5 6.791e-05 6.791e-05 6.791e-05 0.00% Castro::post_regrid() 10 6.418e-05 6.418e-05 6.418e-05 0.00% FillPatchSingleLevel 92 5.787e-05 5.787e-05 5.787e-05 0.00% Castro::errorEst() 6 5.587e-05 5.587e-05 5.587e-05 0.00% Amr::writeSmallPlotFile() 1 5.579e-05 5.579e-05 5.579e-05 0.00% MLMG::oneIter() 31 5.504e-05 5.504e-05 5.504e-05 0.00% AmrLevel::AmrLevel(dm) 5 5.383e-05 5.383e-05 5.383e-05 0.00% FabArray::RecvLayoutMask() 8 5.012e-05 5.012e-05 5.012e-05 0.00% MLMG:computeResOfCorrection() 155 4.923e-05 4.923e-05 4.923e-05 0.00% MLMG::solve() 10 4.802e-05 4.802e-05 4.802e-05 0.00% AmrLevel::FillCoarsePatch() 5 4.74e-05 4.74e-05 4.74e-05 0.00% MLCellLinOp::solutionResidual() 71 4.413e-05 4.413e-05 4.413e-05 0.00% MLMG::mgVcycle_down::0 31 4.267e-05 4.267e-05 4.267e-05 0.00% MLMG::mgVcycle_bottom 65 4.165e-05 4.165e-05 4.165e-05 0.00% MLMG::actualBottomSolve() 31 4.135e-05 4.135e-05 4.135e-05 0.00% Castro::create_source_corrector() 9 3.98e-05 3.98e-05 3.98e-05 0.00% FillPatchIterator::FillFromLevel0() 31 3.933e-05 3.933e-05 3.933e-05 0.00% Amr::defBaseLevel() 1 3.669e-05 3.669e-05 3.669e-05 0.00% MLMG::mgVcycle_down::1 31 3.301e-05 3.301e-05 3.301e-05 0.00% FillPatchIterator::FillFromTwoLevels() 31 3.123e-05 3.123e-05 3.123e-05 0.00% MLMG::mgVcycle_down::3 31 3.103e-05 3.103e-05 3.103e-05 0.00% MLMG::ResNormInf() 54 3.011e-05 3.011e-05 3.011e-05 0.00% MLMG::mgVcycle_down::2 31 2.97e-05 2.97e-05 2.97e-05 0.00% makeSFC 50 2.767e-05 2.767e-05 2.767e-05 0.00% MLMG::mgVcycle_down::4 31 2.72e-05 2.72e-05 2.72e-05 0.00% MLPoisson::define() 10 2.641e-05 2.641e-05 2.641e-05 0.00% MLMG::computeResWithCrseSolFineCor() 17 2.593e-05 2.593e-05 2.593e-05 0.00% Gravity::solve_phi_with_mlmg() 10 2.57e-05 2.57e-05 2.57e-05 0.00% MLMG::mgVcycle_up::4 31 2.293e-05 2.293e-05 2.293e-05 0.00% MLMG::mgVcycle_up::0 31 2.12e-05 2.12e-05 2.12e-05 0.00% DistributionMapping::Distribute() 52 2.096e-05 2.096e-05 2.096e-05 0.00% MLMG::mgVcycle_up::3 31 2.029e-05 2.029e-05 2.029e-05 0.00% FabArrayBase::TheFPinfo() 25 1.967e-05 1.967e-05 1.967e-05 0.00% FPinfo::FPinfo() 3 1.931e-05 1.931e-05 1.931e-05 0.00% MLMG::mgVcycle_up::1 31 1.927e-05 1.927e-05 1.927e-05 0.00% MLMG::computeMLResidual() 15 1.824e-05 1.824e-05 1.824e-05 0.00% MLMG::mgVcycle_up::2 31 1.817e-05 1.817e-05 1.817e-05 0.00% Amr::initSubcycle() 1 1.591e-05 1.591e-05 1.591e-05 0.00% MLMG::miniCycle() 34 1.499e-05 1.499e-05 1.499e-05 0.00% MLLinOp::makeAgglomeratedDMap 10 1.482e-05 1.482e-05 1.482e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 1.388e-05 1.388e-05 1.388e-05 0.00% MLMG::computeResWithCrseCorFineCor() 17 1.385e-05 1.385e-05 1.385e-05 0.00% MLLinOp::define() 10 1.379e-05 1.379e-05 1.379e-05 0.00% MLMG::computeResidual() 31 1.362e-05 1.362e-05 1.362e-05 0.00% Amr::FinalizeInit() 1 1.301e-05 1.301e-05 1.301e-05 0.00% ClusterList::intersect() 4 1.215e-05 1.215e-05 1.215e-05 0.00% Castro::check_for_nan() 18 1.122e-05 1.122e-05 1.122e-05 0.00% Castro::post_init() 1 9.825e-06 9.825e-06 9.825e-06 0.00% Castro::apply_source_to_state() 18 8.953e-06 8.953e-06 8.953e-06 0.00% Castro::expand_state() 13 8.658e-06 8.658e-06 8.658e-06 0.00% Castro::construct_new_hybrid_source() 9 8.556e-06 8.556e-06 8.556e-06 0.00% MLMG::getGradSolution() 10 8.308e-06 8.308e-06 8.308e-06 0.00% AmrMesh-cluster 4 7.667e-06 7.667e-06 7.667e-06 0.00% MLPoisson::prepareForSolve() 10 7.565e-06 7.565e-06 7.565e-06 0.00% Gravity::swapTimeLevels() 9 7.558e-06 7.558e-06 7.558e-06 0.00% Amr::InitializeInit() 1 7.512e-06 7.512e-06 7.512e-06 0.00% MLMG::MLResNormInf() 15 7.364e-06 7.364e-06 7.364e-06 0.00% Castro::avgDown(state_indx) 20 7.187e-06 7.187e-06 7.187e-06 0.00% amrex::average_down_w_geom 20 7.065e-06 7.065e-06 7.065e-06 0.00% Castro::construct_old_hybrid_source() 9 6.027e-06 6.027e-06 6.027e-06 0.00% MLMG::MLRhsNormInf() 10 5.802e-06 5.802e-06 5.802e-06 0.00% BoxList::parallelComplementIn() 12 5.223e-06 5.223e-06 5.223e-06 0.00% Castro::init(old) 3 4.716e-06 4.716e-06 4.716e-06 0.00% Gravity::set_mass_offset() 6 4.26e-06 4.26e-06 4.26e-06 0.00% BoxList::complementIn 12 4.146e-06 4.146e-06 4.146e-06 0.00% Castro::derive() 6 4.068e-06 4.068e-06 4.068e-06 0.00% Castro::avgDown() 4 3.712e-06 3.712e-06 3.712e-06 0.00% Castro::computeInitialDt() 2 2.972e-06 2.972e-06 2.972e-06 0.00% Castro::retry_advance_ctu() 5 2.673e-06 2.673e-06 2.673e-06 0.00% Amr::bldFineLevels() 1 2.308e-06 2.308e-06 2.308e-06 0.00% Castro::init() 1 2.254e-06 2.254e-06 2.254e-06 0.00% Amr::init() 1 1.988e-06 1.988e-06 1.988e-06 0.00% ClusterList::chop() 4 1.922e-06 1.922e-06 1.922e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.797e-06 1.797e-06 1.797e-06 0.00% Amr::initialInit() 1 1.195e-06 1.195e-06 1.195e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 3.225 3.225 3.225 100.00% Amr::coarseTimeStep() 5 1.574 1.574 1.574 48.80% Amr::timeStep() 9 1.568 1.568 1.568 48.64% Castro::advance() 5 1.355 1.355 1.355 42.02% Castro::subcycle_advance_ctu() 5 1.33 1.33 1.33 41.24% Castro::do_advance_ctu() 5 1.329 1.329 1.329 41.23% Amr::InitAmr() 1 1.11 1.11 1.11 34.43% Castro::construct_ctu_hydro_source() 9 0.8598 0.8598 0.8598 26.67% Amr::init() 1 0.3844 0.3844 0.3844 11.92% Gravity::multilevel_solve_for_new_phi() 10 0.3315 0.3315 0.3315 10.28% Gravity::actual_multilevel_solve() 10 0.3307 0.3307 0.3307 10.25% Gravity::solve_phi_with_mlmg() 10 0.3275 0.3275 0.3275 10.16% Amr::writePlotFile() 2 0.3022 0.3022 0.3022 9.37% VisMF::Write(FabArray) 3 0.2953 0.2953 0.2953 9.16% Gravity::actual_solve_with_mlmg() 10 0.2528 0.2528 0.2528 7.84% Amr::initialInit() 1 0.2376 0.2376 0.2376 7.37% MLMG::solve() 10 0.1983 0.1983 0.1983 6.15% Castro::construct_new_gravity() 9 0.1904 0.1904 0.1904 5.91% Castro::clean_state() 87 0.169 0.169 0.169 5.24% MLMG::oneIter() 31 0.1676 0.1676 0.1676 5.20% Amr::InitializeInit() 1 0.1648 0.1648 0.1648 5.11% Amr::defBaseLevel() 1 0.1648 0.1648 0.1648 5.11% Castro::Castro() 5 0.1552 0.1552 0.1552 4.81% Castro::initMFs() 5 0.151 0.151 0.151 4.68% Amr::regrid() 5 0.1481 0.1481 0.1481 4.59% MLMG::mgVcycle() 65 0.1438 0.1438 0.1438 4.46% Castro::post_regrid() 10 0.1336 0.1336 0.1336 4.14% MLCellLinOp::applyBC() 1917 0.09267 0.09267 0.09267 2.87% AmrLevel::FillPatch() 62 0.09251 0.09251 0.09251 2.87% Castro::computeTemp() 88 0.08767 0.08767 0.08767 2.72% MLCellLinOp::smooth() 688 0.08213 0.08213 0.08213 2.55% FillPatchIterator::Initialize 62 0.07804 0.07804 0.07804 2.42% Gravity::fill_multipole_BCs() 10 0.07421 0.07421 0.07421 2.30% Amr::FinalizeInit() 1 0.07279 0.07279 0.07279 2.26% Castro::reflux() 4 0.07233 0.07233 0.07233 2.24% FillPatchSingleLevel 92 0.06949 0.06949 0.06949 2.15% FillPatchIterator::FillFromLevel0() 31 0.06687 0.06687 0.06687 2.07% Castro::post_timestep() 9 0.06592 0.06592 0.06592 2.04% Castro::normalize_species() 87 0.06504 0.06504 0.06504 2.02% Castro::post_init() 1 0.0622 0.0622 0.0622 1.93% Castro::problem_post_timestep() 10 0.0535 0.0535 0.0535 1.66% Castro::initialize_do_advance() 9 0.05156 0.05156 0.05156 1.60% StateDataPhysBCFunct::() 95 0.05048 0.05048 0.05048 1.57% Castro::do_new_sources() 9 0.04551 0.04551 0.04551 1.41% FabArray::FillBoundary() 1728 0.04455 0.04455 0.04455 1.38% FillBoundary_nowait() 1728 0.04413 0.04413 0.04413 1.37% MLPoisson::define() 10 0.04374 0.04374 0.04374 1.36% Castro::do_old_sources() 9 0.04327 0.04327 0.04327 1.34% Castro::expand_state() 13 0.04214 0.04214 0.04214 1.31% MLMG::mgVcycle_bottom 65 0.03864 0.03864 0.03864 1.20% FabArray::setVal() 920 0.03837 0.03837 0.03837 1.19% StateData::FillBoundary(geom) 650 0.03485 0.03485 0.03485 1.08% FabArray::ParallelCopy() 954 0.0348 0.0348 0.0348 1.08% FabArray::ParallelCopy_nowait() 954 0.03447 0.03447 0.03447 1.07% Castro::wd_update() 6 0.03443 0.03443 0.03443 1.07% MLMG::actualBottomSolve() 31 0.03148 0.03148 0.03148 0.98% MLCGSolver::bicgstab 31 0.03118 0.03118 0.03118 0.97% FluxRegister::Reflux() 24 0.02973 0.02973 0.02973 0.92% Castro::volInBoundary() 42 0.02963 0.02963 0.02963 0.92% MLMG::mgVcycle_down::0 31 0.02925 0.02925 0.02925 0.91% Castro::construct_new_source() 63 0.02826 0.02826 0.02826 0.88% MLCellLinOp::apply() 493 0.02825 0.02825 0.02825 0.88% Castro::reset_internal_energy(MultiFab) 88 0.02769 0.02769 0.02769 0.86% amrex::Copy() 441 0.02674 0.02674 0.02674 0.83% MLCellLinOp::defineAuxData() 10 0.02507 0.02507 0.02507 0.78% Castro::initialize_advance() 9 0.02415 0.02415 0.02415 0.75% Gravity::get_new_grav_vector() 10 0.02322 0.02322 0.02322 0.72% MLMG::mgVcycle_up::0 31 0.02189 0.02189 0.02189 0.68% Castro::construct_old_gravity() 9 0.0186 0.0186 0.0186 0.58% Gravity::get_old_grav_vector() 9 0.01851 0.01851 0.01851 0.57% MLCellLinOp::correctionResidual() 189 0.01843 0.01843 0.01843 0.57% MLCellLinOp::defineBC() 10 0.01837 0.01837 0.01837 0.57% amrex::Add() 136 0.01727 0.01727 0.01727 0.54% Castro::estTimeStep() 26 0.01653 0.01653 0.01653 0.51% FabArray::Saxpy() 604 0.016 0.016 0.016 0.50% MLCellLinOp::reflux() 30 0.01597 0.01597 0.01597 0.50% BndryData::define() 26 0.01546 0.01546 0.01546 0.48% MLCellLinOp::solutionResidual() 71 0.01487 0.01487 0.01487 0.46% MLMG:computeResOfCorrection() 155 0.01407 0.01407 0.01407 0.44% MLMG::computeResWithCrseSolFineCor() 17 0.01371 0.01371 0.01371 0.43% Amr::grid_places() 6 0.01355 0.01355 0.01355 0.42% AmrMesh::MakeNewGrids() 6 0.01345 0.01345 0.01345 0.42% MLMG::computeMLResidual() 15 0.01302 0.01302 0.01302 0.40% Castro::construct_old_source() 63 0.0128 0.0128 0.0128 0.40% MLPoisson::Fsmooth() 1376 0.0127 0.0127 0.0127 0.39% Castro::enforce_min_density() 87 0.01225 0.01225 0.01225 0.38% MLMG::mgVcycle_down::1 31 0.01165 0.01165 0.01165 0.36% Castro::initData() 1 0.01136 0.01136 0.01136 0.35% amrex::average_down 233 0.009918 0.009918 0.009918 0.31% amrex::Dot() 466 0.009276 0.009276 0.009276 0.29% Castro::FluxRegCrseInit 9 0.009023 0.009023 0.009023 0.28% MLMG::prepareForSolve() 10 0.008923 0.008923 0.008923 0.28% FabArray::norminf() 439 0.00878 0.00878 0.00878 0.27% MLMG::mgVcycle_up::1 31 0.008542 0.008542 0.008542 0.26% Castro::construct_old_gravity_source() 18 0.007934 0.007934 0.007934 0.25% FillPatchIterator::FillFromTwoLevels() 31 0.007427 0.007427 0.007427 0.23% FillPatchTwoLevels 31 0.007396 0.007396 0.007396 0.23% Castro::finalize_do_advance() 9 0.007194 0.007194 0.007194 0.22% MLMG::miniCycle() 34 0.007171 0.007171 0.007171 0.22% Amr::bldFineLevels() 1 0.007088 0.007088 0.007088 0.22% MLMG::addInterpCorrection() 155 0.006948 0.006948 0.006948 0.22% Castro::computeNewDt() 8 0.006934 0.006934 0.006934 0.22% MLPoisson::Fapply() 493 0.0069 0.0069 0.0069 0.21% MLMG::mgVcycle_down::2 31 0.006832 0.006832 0.006832 0.21% FabArray::Xpay() 353 0.006496 0.006496 0.006496 0.20% MLMG::mgVcycle_down::3 31 0.006273 0.006273 0.006273 0.19% MLMG::mgVcycle_down::4 31 0.006122 0.006122 0.006122 0.19% MLMG::computeResidual() 31 0.006026 0.006026 0.006026 0.19% Castro::construct_new_gravity_source() 9 0.005445 0.005445 0.005445 0.17% MLCGSolver::ParallelAllReduce 745 0.005238 0.005238 0.005238 0.16% MLMG::mgVcycle_up::4 31 0.00498 0.00498 0.00498 0.15% FabArrayBase::getCPC() 1522 0.004919 0.004919 0.004919 0.15% Castro::reset_internal_energy(Fab) 1476 0.004851 0.004851 0.004851 0.15% MLMG::mgVcycle_up::2 31 0.00478 0.00478 0.00478 0.15% MLMG::mgVcycle_up::3 31 0.004777 0.004777 0.004777 0.15% MLMG::getGradSolution() 10 0.004439 0.004439 0.004439 0.14% MLCellLinOp::compGrad() 18 0.00443 0.00443 0.00443 0.14% Castro::enforce_speed_limit() 87 0.004385 0.004385 0.004385 0.14% Castro::apply_source_to_state() 18 0.004225 0.004225 0.004225 0.13% FabArrayBase::CPC::define() 534 0.003755 0.003755 0.003755 0.12% Castro::errorEst() 6 0.003751 0.003751 0.003751 0.12% FabArray::setVal(val, thecmd, scomp, ncomp) 580 0.003689 0.003689 0.003689 0.11% FillPatchInterp(MF) 30 0.003679 0.003679 0.003679 0.11% MLMG::computeResWithCrseCorFineCor() 17 0.003623 0.003623 0.003623 0.11% MLCellLinOp::updateSolBC() 25 0.003408 0.003408 0.003408 0.11% FabArray::setDomainBndry() 92 0.003346 0.003346 0.003346 0.10% Castro::check_for_nan() 18 0.003259 0.003259 0.003259 0.10% MultiFab::contains_nan() 18 0.003248 0.003248 0.003248 0.10% check_for_negative_density() 9 0.003245 0.003245 0.003245 0.10% TagBoxArray::mapPRD 6 0.003212 0.003212 0.003212 0.10% Castro::init(old) 3 0.003072 0.003072 0.003072 0.10% Castro::computeInitialDt() 2 0.002695 0.002695 0.002695 0.08% FabArray::mult() 55 0.002595 0.002595 0.002595 0.08% Castro::buildMetrics() 5 0.002561 0.002561 0.002561 0.08% MLCellLinOp::setLevelBC() 18 0.002527 0.002527 0.002527 0.08% MLCellLinOp::updateCorBC() 17 0.002372 0.002372 0.002372 0.07% MLMG::ResNormInf() 54 0.00208 0.00208 0.00208 0.06% Gravity::update_max_rhs() 10 0.002024 0.002024 0.002024 0.06% Castro::apply_tagging_restrictions() 6 0.002016 0.002016 0.002016 0.06% MLPoisson::prepareForSolve() 10 0.002006 0.002006 0.002006 0.06% MLCellLinOp::prepareForSolve() 10 0.001999 0.001999 0.001999 0.06% Gravity::average_fine_ec_onto_crse_ec() 8 0.001988 0.001988 0.001988 0.06% Castro::fill_hybrid_hydro_source() 27 0.001658 0.001658 0.001658 0.05% MLPoisson::FFlux() 150 0.001492 0.001492 0.001492 0.05% average_down_faces 48 0.001316 0.001316 0.001316 0.04% FabArray::norminf(mask) 21 0.001225 0.001225 0.001225 0.04% Castro::avgDown() 4 0.001193 0.001193 0.001193 0.04% Castro::avgDown(state_indx) 20 0.001189 0.001189 0.001189 0.04% amrex::average_down_w_geom 20 0.001182 0.001182 0.001182 0.04% MLMG::MLResNormInf() 15 0.001145 0.001145 0.001145 0.04% Castro::construct_new_hybrid_source() 9 0.001034 0.001034 0.001034 0.03% OwnerMask() 6 0.001012 0.001012 0.001012 0.03% MLMG::interpCorrection_1 17 0.0009877 0.0009877 0.0009877 0.03% Castro::build_fine_mask() 44 0.0009823 0.0009823 0.0009823 0.03% Castro::finalize_advance() 9 0.0009315 0.0009315 0.0009315 0.03% Castro::derive() 6 0.0009244 0.0009244 0.0009244 0.03% AmrLevel::derive() 6 0.0009203 0.0009203 0.0009203 0.03% Castro::init() 1 0.0009113 0.0009113 0.0009113 0.03% AmrLevel::FillCoarsePatch() 5 0.0009091 0.0009091 0.0009091 0.03% TagBoxArray::collate() 6 0.0008053 0.0008053 0.0008053 0.02% FabArrayBase::getFB() 1728 0.0007346 0.0007346 0.0007346 0.02% Castro::FluxRegFineAdd() 9 0.0007204 0.0007204 0.0007204 0.02% Gravity::swapTimeLevels() 9 0.0006937 0.0006937 0.0006937 0.02% Castro::construct_old_hybrid_source() 9 0.0006382 0.0006382 0.0006382 0.02% MultiFab::max() 18 0.0006104 0.0006104 0.0006104 0.02% MLMG::MLRhsNormInf() 10 0.0006031 0.0006031 0.0006031 0.02% Castro::apply_problem_tags() 6 0.0005233 0.0005233 0.0005233 0.02% MultiFab::Multiply() 8 0.0005067 0.0005067 0.0005067 0.02% Castro::react_state() 18 0.0004589 0.0004589 0.0004589 0.01% FabArrayBase::FB::FB() 37 0.0004364 0.0004364 0.0004364 0.01% Castro::linear_to_hybrid_momentum() 1 0.00037 0.00037 0.00037 0.01% MLLinOp::define() 10 0.0002803 0.0002803 0.0002803 0.01% MLLinOp::defineGrids() 10 0.0002665 0.0002665 0.0002665 0.01% AMRErrorTag::operator() 6 0.0002317 0.0002317 0.0002317 0.01% Castro::construct_new_rotation_source() 9 0.0002188 0.0002188 0.0002188 0.01% AmrLevel::AmrLevel(dm) 5 0.0002105 0.0002105 0.0002105 0.01% Castro::construct_old_rotation_source() 9 0.0002036 0.0002036 0.0002036 0.01% StateData::define() 25 0.0001567 0.0001567 0.0001567 0.00% Castro::enforce_consistent_e() 1 0.0001399 0.0001399 0.0001399 0.00% Castro::swap_state_time_levels() 9 0.0001267 0.0001267 0.0001267 0.00% MLLinOp::makeAgglomeratedDMap 10 6.154e-05 6.154e-05 6.154e-05 0.00% Amr::writeSmallPlotFile() 1 5.579e-05 5.579e-05 5.579e-05 0.00% FabArray::RecvLayoutMask() 8 5.012e-05 5.012e-05 5.012e-05 0.00% makeSFC 50 4.672e-05 4.672e-05 4.672e-05 0.00% Castro::create_source_corrector() 9 3.98e-05 3.98e-05 3.98e-05 0.00% FabArrayBase::TheFPinfo() 25 3.898e-05 3.898e-05 3.898e-05 0.00% AmrMesh-cluster 4 2.174e-05 2.174e-05 2.174e-05 0.00% DistributionMapping::Distribute() 52 2.096e-05 2.096e-05 2.096e-05 0.00% FPinfo::FPinfo() 3 1.931e-05 1.931e-05 1.931e-05 0.00% Amr::initSubcycle() 1 1.591e-05 1.591e-05 1.591e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 2 1.578e-05 1.578e-05 1.578e-05 0.00% ClusterList::intersect() 4 1.215e-05 1.215e-05 1.215e-05 0.00% BoxList::parallelComplementIn() 12 9.369e-06 9.369e-06 9.369e-06 0.00% Gravity::set_mass_offset() 6 4.26e-06 4.26e-06 4.26e-06 0.00% BoxList::complementIn 12 4.146e-06 4.146e-06 4.146e-06 0.00% Castro::retry_advance_ctu() 5 2.673e-06 2.673e-06 2.673e-06 0.00% ClusterList::chop() 4 1.922e-06 1.922e-06 1.922e-06 0.00% MLLinOp::makeSubCommunicator() 10 1.797e-06 1.797e-06 1.797e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::mg.v(nvals = 1) :: [2] Device Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Arena::Initialize() 1 1 1699 KiB 9037 MiB Castro::reflux() 436 436 12 MiB 666 MiB Castro::initMFs() 258 258 345 MiB 576 MiB Castro::swap_state_time_levels() 175 175 179 MiB 340 MiB StateData::define() 175 175 218 MiB 340 MiB Castro::initialize_do_advance() 143 143 125 MiB 325 MiB FillPatchIterator::Initialize 899 899 3979 KiB 303 MiB Amr::writePlotFile() 56 56 16 MiB 189 MiB Castro::problem_post_timestep() 162 162 937 KiB 167 MiB Castro::FluxRegCrseInit 660 660 438 KiB 167 MiB FluxRegister::Reflux() 648 648 1497 KiB 160 MiB Castro::initialize_advance() 143 143 50 MiB 121 MiB Castro::construct_new_source() 143 143 249 KiB 67 MiB Castro::construct_old_source() 143 143 134 KiB 67 MiB Castro::buildMetrics() 140 140 37 MiB 61 MiB Gravity::multilevel_solve_for_new_phi() 858 858 28 MiB 60 MiB Castro::Castro() 210 210 3979 KiB 50 MiB MLMG::prepareForSolve() 1902 1902 2648 KiB 42 MiB Gravity::get_old_grav_vector() 155 155 190 KiB 33 MiB Gravity::get_new_grav_vector() 188 188 236 KiB 33 MiB ResizeRandomSeed 1 1 30 MiB 30 MiB MLCellLinOp::defineAuxData() 5608 5608 871 KiB 16 MiB FillPatchTwoLevels 300 300 7463 B 11 MiB FillPatchInterp(MF) 330 330 4575 B 7955 KiB Gravity::fill_multipole_BCs() 346 346 130 KiB 7469 KiB Gravity::update_max_rhs() 286 286 4341 B 7424 KiB Gravity::actual_multilevel_solve() 286 286 745 KiB 7424 KiB Castro::build_fine_mask() 224 224 1183 KiB 6912 KiB AmrLevel::derive() 162 162 2495 B 6912 KiB BndryData::define() 3624 3624 277 KiB 4243 KiB MLCellLinOp::reflux() 450 450 5521 B 2376 KiB Castro::wd_update() 6 6 23 KiB 2240 KiB AmrLevel::FillCoarsePatch() 10 10 398 B 2095 KiB amrex::average_down 1483 1483 2321 B 1474 KiB AmrMesh::MakeNewGrids() 324 324 4381 B 1040 KiB MLMG::addInterpCorrection() 1240 1240 772 B 867 KiB MLCellLinOp::defineBC() 276 276 3758 B 689 KiB Castro::estTimeStep() 26 26 2421 B 480 KiB VisMF::Write(FabArray) 1568 1568 2272 B 320 KiB Castro::normalize_species() 87 87 6553 B 320 KiB Castro::volInBoundary() 70 70 2976 B 320 KiB Gravity::average_fine_ec_onto_crse_ec() 48 48 123 B 204 KiB FabArray::norminf(mask) 29 29 183 B 163 KiB amrex::Dot() 559 559 427 B 160 KiB FabArray::norminf() 470 470 421 B 160 KiB MultiFab::max() 18 18 29 B 160 KiB check_for_negative_density() 9 9 163 B 160 KiB Castro::initData() 1 1 43 B 160 KiB FabArray::setVal() 419 419 54 KiB 120 KiB MLMG::interpCorrection_1 34 34 26 B 91 KiB average_down_faces 99 99 639 B 70 KiB FabArray::ParallelCopy_nowait() 942 942 269 B 57 KiB MLPoisson::Fsmooth() 228 228 3026 B 55 KiB FillBoundary_nowait() 751 751 432 B 53 KiB TagBoxArray::mapPRD 324 324 40 B 45 KiB OwnerMask() 168 168 11 B 38 KiB MLCellLinOp::applyBC() 3834 3834 138 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9557 B 25 KiB MultiFab::contains_nan() 18 18 20 B 20 KiB StateData::FillBoundary(geom) 2538 2538 45 B 13 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B MultiFab::Multiply() 8 8 590 B 3456 B MLCGSolver::bicgstab 155 155 25 B 2688 B TagBoxArray::collate() 14 14 0 B 640 B ------------------------------------------------------------------------------ Managed Memory Usage: ----------------------------------------------------------------- Name Nalloc Nfree AvgMem MaxMem ----------------------------------------------------------------- The_Managed_Arena::Initialize() 1 1 187 B 8192 KiB ----------------------------------------------------------------- Pinned Memory Usage: ------------------------------------------------------------------------------ Name Nalloc Nfree AvgMem MaxMem ------------------------------------------------------------------------------ The_Pinned_Arena::Initialize() 1 1 11 KiB 8192 KiB VisMF::Write(FabArray) 1624 1624 584 KiB 7168 KiB Gravity::fill_multipole_BCs() 30 30 1205 B 278 KiB FabArray::setVal() 419 419 54 KiB 120 KiB FabArray::ParallelCopy_nowait() 942 942 269 B 57 KiB MLPoisson::Fsmooth() 228 228 3026 B 55 KiB FillBoundary_nowait() 751 751 432 B 53 KiB MLCellLinOp::applyBC() 1917 1917 138 B 30 KiB MLCellLinOp::prepareForSolve() 68 68 6 B 25 KiB amrex::Copy() 176 176 9557 B 25 KiB Gravity::get_new_grav_vector() 18 18 3263 B 20 KiB Gravity::get_old_grav_vector() 12 12 6656 B 20 KiB OwnerMask() 6 6 0 B 15 KiB StateData::FillBoundary(geom) 2538 2538 45 B 13 KiB FabArray::setVal(val, thecmd, scomp, ncomp) 244 244 3 B 9808 B amrex::average_down 149 149 1338 B 6304 B MLMG::prepareForSolve() 18 18 226 B 3712 B FabArray::norminf(mask) 29 29 126 B 3472 B MLMG::addInterpCorrection() 62 62 2 B 3456 B MultiFab::Multiply() 8 8 590 B 3456 B Castro::FluxRegCrseInit 12 12 8 B 3456 B FillPatchInterp(MF) 140 140 3 B 3232 B MLCellLinOp::setLevelBC() 68 68 0 B 2592 B average_down_faces 51 51 612 B 2272 B TagBoxArray::collate() 16 16 0 B 640 B amrex::Dot() 559 559 3 B 400 B Castro::build_fine_mask() 8 8 0 B 208 B MLCellLinOp::defineAuxData() 8 8 0 B 208 B Castro::reflux() 4 4 0 B 208 B FabArray::norminf() 470 470 1 B 144 B Castro::wd_update() 6 6 1 B 112 B Castro::problem_post_timestep() 19 19 0 B 48 B Castro::estTimeStep() 26 26 0 B 32 B MultiFab::contains_nan() 18 18 0 B 16 B MultiFab::max() 18 18 0 B 16 B Castro::volInBoundary() 70 70 0 B 16 B Castro::normalize_species() 87 87 0 B 16 B Castro::initData() 1 1 0 B 16 B check_for_negative_density() 9 9 0 B 16 B ------------------------------------------------------------------------------ Total GPU global memory (MB): 12049 Free GPU global memory (MB): 1781 [The Arena] space allocated (MB): 9037 [The Arena] space used (MB): 0 [The Managed Arena] space allocated (MB): 8 [The Managed Arena] space used (MB): 0 [The Pinned Arena] space allocated (MB): 8 [The Pinned Arena] space used (MB): 0 AMReX (24.04-14-g6c6247554f27) finalized