Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.01-46-g89153db246e4) initialized
Starting run at 14:38:24 UTC on 2022-02-01.
Successfully read inputs file ...
Castro git describe: 21.12-28-g756d00e43
AMReX git describe: 22.01-46-g89153db24
Microphysics git describe: 22.02
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.037852098 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.022129203 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.111792097
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.053232476
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.053375357
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.061980259
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.063584318
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.039567653 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.052578189
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.04990207
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.056757271
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.058962792
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.061709454
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.036530103 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.040696439 seconds
Ending run at 14:38:25 UTC on 2022-02-01.
Run time = 0.862110277
Run time without initialization = 0.741239647
Average number of zones advanced per microsecond: 3.537
Average number of zones advanced per microsecond per rank: 3.537
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.8621 ... 0.8621 ... 0.8621
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.2083 0.2083 0.2083 24.16%
VisMF::Write(FabArray) 11 0.1694 0.1694 0.1694 19.65%
MLCellLinOp::applyBC() 4379 0.09543 0.09543 0.09543 11.07%
MLPoisson::Fsmooth() 3240 0.06051 0.06051 0.06051 7.02%
StateData::FillBoundary(geom) 328 0.04276 0.04276 0.04276 4.96%
FabArray::setVal() 1135 0.02603 0.02603 0.02603 3.02%
MLCGSolver::bicgstab 81 0.02222 0.02222 0.02222 2.58%
MultiFab::Dot() 1100 0.02197 0.02197 0.02197 2.55%
FillBoundary_nowait() 3974 0.01759 0.01759 0.01759 2.04%
FabArray::ParallelCopy_nowait() 851 0.01655 0.01655 0.01655 1.92%
MultiFab::LinComb() 1566 0.01273 0.01273 0.01273 1.48%
Castro::computeTemp() 63 0.01241 0.01241 0.01241 1.44%
Gravity::fill_multipole_BCs() 11 0.01141 0.01141 0.01141 1.32%
MLPoisson::Fapply() 1128 0.01108 0.01108 0.01108 1.29%
MLCellLinOp::defineAuxData() 11 0.011 0.011 0.011 1.28%
Castro::enforce_min_density() 62 0.01042 0.01042 0.01042 1.21%
StateDataPhysBCFunct::() 41 0.009377 0.009377 0.009377 1.09%
MLMG::addInterpCorrection() 405 0.007437 0.007437 0.007437 0.86%
amrex::average_down 405 0.00707 0.00707 0.00707 0.82%
MultiFab::Xpay() 578 0.006193 0.006193 0.006193 0.72%
FabArray::setDomainBndry() 41 0.005784 0.005784 0.005784 0.67%
Castro::reset_internal_energy() 63 0.005584 0.005584 0.005584 0.65%
Castro::estTimeStep() 21 0.004705 0.004705 0.004705 0.55%
Amr::checkPoint() 3 0.004694 0.004694 0.004694 0.54%
Castro::expand_state() 10 0.004552 0.004552 0.004552 0.53%
BndryData::define() 11 0.003828 0.003828 0.003828 0.44%
Castro::enforce_speed_limit() 62 0.003511 0.003511 0.003511 0.41%
Castro::do_advance_ctu() 10 0.003155 0.003155 0.003155 0.37%
Amr::writePlotFile() 2 0.002698 0.002698 0.002698 0.31%
Castro::normalize_species() 62 0.002692 0.002692 0.002692 0.31%
Castro::construct_new_gravity_source() 10 0.002543 0.002543 0.002543 0.29%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002407 0.002407 0.002407 0.28%
Gravity::get_new_grav_vector() 11 0.002332 0.002332 0.002332 0.27%
Castro::construct_old_gravity_source() 10 0.001994 0.001994 0.001994 0.23%
MLMG::ResNormInf() 92 0.001913 0.001913 0.001913 0.22%
MultiFab::Saxpy() 20 0.001821 0.001821 0.001821 0.21%
Gravity::get_old_grav_vector() 10 0.001808 0.001808 0.001808 0.21%
MLMG::oneIter() 81 0.001659 0.001659 0.001659 0.19%
MLCellLinOp::setLevelBC() 11 0.001604 0.001604 0.001604 0.19%
MLCellLinOp::smooth() 1620 0.001569 0.001569 0.001569 0.18%
Gravity::actual_solve_with_mlmg() 11 0.001438 0.001438 0.001438 0.17%
MLCellLinOp::prepareForSolve() 11 0.0014 0.0014 0.0014 0.16%
FabArray::mult() 43 0.001351 0.001351 0.001351 0.16%
MLCellLinOp::compGrad() 11 0.001298 0.001298 0.001298 0.15%
Castro::initData() 1 0.001175 0.001175 0.001175 0.14%
FabArrayBase::getCPC() 1313 0.000938 0.000938 0.000938 0.11%
MLMG::prepareForSolve() 11 0.0008999 0.0008999 0.0008999 0.10%
FabArray::FillBoundary() 3974 0.0008544 0.0008544 0.0008544 0.10%
FabArrayBase::getFB() 3974 0.0008271 0.0008271 0.0008271 0.10%
FabArrayBase::CPC::define() 454 0.0007933 0.0007933 0.0007933 0.09%
MLCellLinOp::apply() 1128 0.0006283 0.0006283 0.0006283 0.07%
Gravity::update_max_rhs() 11 0.0006202 0.0006202 0.0006202 0.07%
CGSolver::sxay() 1566 0.0005325 0.0005325 0.0005325 0.06%
Gravity::solve_for_phi() 10 0.0004869 0.0004869 0.0004869 0.06%
MLMG::mgVcycle() 81 0.0004531 0.0004531 0.0004531 0.05%
Amr::InitAmr() 1 0.0004497 0.0004497 0.0004497 0.05%
MultiFab::Copy() 11 0.0004176 0.0004176 0.0004176 0.05%
MLCGSolver::ParallelAllReduce 1495 0.0003949 0.0003949 0.0003949 0.05%
MultiFab::min() 10 0.0003435 0.0003435 0.0003435 0.04%
FabArray::ParallelCopy() 851 0.0003383 0.0003383 0.0003383 0.04%
main() 1 0.0003312 0.0003312 0.0003312 0.04%
FillPatchIterator::Initialize 41 0.000285 0.000285 0.000285 0.03%
MultiFab::max() 11 0.0002653 0.0002653 0.0002653 0.03%
MLCellLinOp::correctionResidual() 486 0.0002596 0.0002596 0.0002596 0.03%
Castro::construct_new_gravity() 10 0.0002436 0.0002436 0.0002436 0.03%
Amr::coarseTimeStep() 10 0.0002386 0.0002386 0.0002386 0.03%
MLCellLinOp::defineBC() 11 0.0002363 0.0002363 0.0002363 0.03%
Gravity::actual_multilevel_solve() 1 0.0002217 0.0002217 0.0002217 0.03%
Amr::timeStep() 10 0.0002147 0.0002147 0.0002147 0.02%
MLMG::MLRhsNormInf() 11 0.0002022 0.0002022 0.0002022 0.02%
MLLinOp::defineGrids() 11 0.0001937 0.0001937 0.0001937 0.02%
Amr::defBaseLevel() 1 0.0001659 0.0001659 0.0001659 0.02%
MLMG:computeResOfCorrection() 405 0.0001536 0.0001536 0.0001536 0.02%
StateData::checkPoint() 12 0.0001434 0.0001434 0.0001434 0.02%
Castro::subcycle_advance_ctu() 10 0.0001356 0.0001356 0.0001356 0.02%
MLMG::actualBottomSolve() 81 0.0001304 0.0001304 0.0001304 0.02%
MLMG::mgVcycle_down::0 81 0.0001113 0.0001113 0.0001113 0.01%
FabArrayBase::FB::FB() 56 0.0001001 0.0001001 0.0001001 0.01%
MLMG::mgVcycle_down::1 81 9.961e-05 9.961e-05 9.961e-05 0.01%
Castro::initialize_advance() 10 9.931e-05 9.931e-05 9.931e-05 0.01%
MLMG::mgVcycle_down::2 81 9.742e-05 9.742e-05 9.742e-05 0.01%
MLMG::mgVcycle_down::4 81 9.483e-05 9.483e-05 9.483e-05 0.01%
MLMG::mgVcycle_down::3 81 9.465e-05 9.465e-05 9.465e-05 0.01%
Castro::post_timestep() 10 9.352e-05 9.352e-05 9.352e-05 0.01%
MLMG::solve() 11 8.712e-05 8.712e-05 8.712e-05 0.01%
Castro::clean_state() 62 8.028e-05 8.028e-05 8.028e-05 0.01%
AmrLevel::checkPoint() 3 7.907e-05 7.907e-05 7.907e-05 0.01%
MLMG::mgVcycle_up::4 81 7.107e-05 7.107e-05 7.107e-05 0.01%
Castro::initialize_do_advance() 10 6.529e-05 6.529e-05 6.529e-05 0.01%
MLMG::mgVcycle_up::3 81 6.024e-05 6.024e-05 6.024e-05 0.01%
MLMG::mgVcycle_up::0 81 6.021e-05 6.021e-05 6.021e-05 0.01%
MLMG::mgVcycle_up::1 81 5.859e-05 5.859e-05 5.859e-05 0.01%
Castro::finalize_advance() 10 5.825e-05 5.825e-05 5.825e-05 0.01%
MLMG::mgVcycle_up::2 81 5.799e-05 5.799e-05 5.799e-05 0.01%
MLCellLinOp::solutionResidual() 92 5.714e-05 5.714e-05 5.714e-05 0.01%
Castro::swap_state_time_levels() 10 4.644e-05 4.644e-05 4.644e-05 0.01%
StateData::define() 4 4.351e-05 4.351e-05 4.351e-05 0.01%
Castro::advance() 10 4.187e-05 4.187e-05 4.187e-05 0.00%
Castro::finalize_do_advance() 10 4.13e-05 4.13e-05 4.13e-05 0.00%
MLMG::computeResidual() 81 3.769e-05 3.769e-05 3.769e-05 0.00%
Castro::enforce_consistent_e() 1 3.491e-05 3.491e-05 3.491e-05 0.00%
MLMG::mgVcycle_bottom 81 3.464e-05 3.464e-05 3.464e-05 0.00%
FillPatchSingleLevel 41 3.382e-05 3.382e-05 3.382e-05 0.00%
Castro::construct_old_source() 50 3.18e-05 3.18e-05 3.18e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 3.117e-05 3.117e-05 3.117e-05 0.00%
MLLinOp::define() 11 2.926e-05 2.926e-05 2.926e-05 0.00%
MLPoisson::define() 11 2.836e-05 2.836e-05 2.836e-05 0.00%
makeSFC 55 2.793e-05 2.793e-05 2.793e-05 0.00%
Amr::writeSmallPlotFile() 1 2.353e-05 2.353e-05 2.353e-05 0.00%
Amr::FinalizeInit() 1 2.187e-05 2.187e-05 2.187e-05 0.00%
Castro::construct_new_source() 50 2.023e-05 2.023e-05 2.023e-05 0.00%
Castro::do_new_sources() 10 1.861e-05 1.861e-05 1.861e-05 0.00%
Castro::do_old_sources() 10 1.859e-05 1.859e-05 1.859e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.801e-05 1.801e-05 1.801e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.766e-05 1.766e-05 1.766e-05 0.00%
DistributionMapping::Distribute() 56 1.523e-05 1.523e-05 1.523e-05 0.00%
Gravity::swapTimeLevels() 10 1.441e-05 1.441e-05 1.441e-05 0.00%
Castro::apply_source_to_state() 20 1.375e-05 1.375e-05 1.375e-05 0.00%
Castro::construct_old_gravity() 10 1.18e-05 1.18e-05 1.18e-05 0.00%
AmrLevel::AmrLevel(dm) 1 1.025e-05 1.025e-05 1.025e-05 0.00%
Amr::initSubcycle() 1 9.961e-06 9.961e-06 9.961e-06 0.00%
MLPoisson::prepareForSolve() 11 8.79e-06 8.79e-06 8.79e-06 0.00%
Castro::computeNewDt() 9 7.995e-06 7.995e-06 7.995e-06 0.00%
MLMG::computeMLResidual() 11 7.404e-06 7.404e-06 7.404e-06 0.00%
AmrLevel::checkPointPost() 3 7.213e-06 7.213e-06 7.213e-06 0.00%
Amr::InitializeInit() 1 6.887e-06 6.887e-06 6.887e-06 0.00%
MLMG::getGradSolution() 11 6.42e-06 6.42e-06 6.42e-06 0.00%
MLMG::buildFineMask() 11 6.318e-06 6.318e-06 6.318e-06 0.00%
Castro::create_source_corrector() 10 5.826e-06 5.826e-06 5.826e-06 0.00%
MLMG::MLResNormInf() 11 5.641e-06 5.641e-06 5.641e-06 0.00%
Castro::post_init() 1 5.107e-06 5.107e-06 5.107e-06 0.00%
Gravity::set_mass_offset() 11 5.01e-06 5.01e-06 5.01e-06 0.00%
Castro::retry_advance_ctu() 10 4.193e-06 4.193e-06 4.193e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.393e-06 3.393e-06 3.393e-06 0.00%
Castro::FluxRegFineAdd() 10 3.237e-06 3.237e-06 3.237e-06 0.00%
Amr::init() 1 3.031e-06 3.031e-06 3.031e-06 0.00%
Castro::computeInitialDt() 2 2.31e-06 2.31e-06 2.31e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.299e-06 2.299e-06 2.299e-06 0.00%
AmrLevel::checkPointPre() 3 1.863e-06 1.863e-06 1.863e-06 0.00%
Castro::post_regrid() 1 1.465e-06 1.465e-06 1.465e-06 0.00%
Amr::initialInit() 1 9.63e-07 9.63e-07 9.63e-07 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.8621 0.8621 0.8621 100.00%
Amr::coarseTimeStep() 10 0.7003 0.7003 0.7003 81.23%
Amr::timeStep() 10 0.6216 0.6216 0.6216 72.10%
Castro::advance() 10 0.6141 0.6141 0.6141 71.22%
Castro::subcycle_advance_ctu() 10 0.6004 0.6004 0.6004 69.64%
Castro::do_advance_ctu() 10 0.6003 0.6003 0.6003 69.63%
Gravity::solve_phi_with_mlmg() 11 0.333 0.333 0.333 38.63%
Gravity::actual_solve_with_mlmg() 11 0.3213 0.3213 0.3213 37.27%
Castro::construct_new_gravity() 10 0.3093 0.3093 0.3093 35.88%
MLMG::solve() 11 0.2979 0.2979 0.2979 34.55%
Gravity::solve_for_phi() 10 0.2869 0.2869 0.2869 33.28%
MLMG::oneIter() 81 0.2821 0.2821 0.2821 32.72%
MLMG::mgVcycle() 81 0.2804 0.2804 0.2804 32.53%
Castro::construct_ctu_hydro_source() 10 0.2083 0.2083 0.2083 24.16%
VisMF::Write(FabArray) 11 0.1694 0.1694 0.1694 19.65%
MLCellLinOp::smooth() 1620 0.1472 0.1472 0.1472 17.08%
Amr::init() 1 0.1203 0.1203 0.1203 13.95%
MLCellLinOp::applyBC() 4379 0.1148 0.1148 0.1148 13.32%
Amr::checkPoint() 3 0.1141 0.1141 0.1141 13.24%
AmrLevel::checkPoint() 3 0.1094 0.1094 0.1094 12.69%
StateData::checkPoint() 12 0.1093 0.1093 0.1093 12.68%
MLMG::mgVcycle_bottom 81 0.08025 0.08025 0.08025 9.31%
MLMG::actualBottomSolve() 81 0.08022 0.08022 0.08022 9.30%
MLCGSolver::bicgstab 81 0.07943 0.07943 0.07943 9.21%
Amr::writePlotFile() 2 0.06294 0.06294 0.06294 7.30%
FillPatchIterator::Initialize 41 0.06253 0.06253 0.06253 7.25%
MLPoisson::Fsmooth() 3240 0.06051 0.06051 0.06051 7.02%
Amr::initialInit() 1 0.06016 0.06016 0.06016 6.98%
FillPatchSingleLevel 41 0.05646 0.05646 0.05646 6.55%
StateDataPhysBCFunct::() 41 0.05214 0.05214 0.05214 6.05%
Amr::FinalizeInit() 1 0.05088 0.05088 0.05088 5.90%
Castro::post_init() 1 0.05016 0.05016 0.05016 5.82%
Gravity::multilevel_solve_for_new_phi() 1 0.04679 0.04679 0.04679 5.43%
Gravity::actual_multilevel_solve() 1 0.04677 0.04677 0.04677 5.43%
StateData::FillBoundary(geom) 328 0.04276 0.04276 0.04276 4.96%
MLCellLinOp::apply() 1128 0.04081 0.04081 0.04081 4.73%
MLMG::mgVcycle_down::0 81 0.03845 0.03845 0.03845 4.46%
Castro::clean_state() 62 0.03404 0.03404 0.03404 3.95%
MLMG::mgVcycle_up::0 81 0.03292 0.03292 0.03292 3.82%
FabArray::setVal() 1135 0.02603 0.02603 0.02603 3.02%
Gravity::get_new_grav_vector() 11 0.02534 0.02534 0.02534 2.94%
Castro::initialize_do_advance() 10 0.02447 0.02447 0.02447 2.84%
MLCellLinOp::correctionResidual() 486 0.02291 0.02291 0.02291 2.66%
MultiFab::Dot() 1100 0.02197 0.02197 0.02197 2.55%
Castro::construct_old_gravity() 10 0.02112 0.02112 0.02112 2.45%
Gravity::get_old_grav_vector() 10 0.02111 0.02111 0.02111 2.45%
MLMG:computeResOfCorrection() 405 0.0198 0.0198 0.0198 2.30%
FabArray::FillBoundary() 3974 0.01938 0.01938 0.01938 2.25%
MLMG::mgVcycle_down::1 81 0.01912 0.01912 0.01912 2.22%
Castro::expand_state() 10 0.01897 0.01897 0.01897 2.20%
MLMG::mgVcycle_down::2 81 0.01856 0.01856 0.01856 2.15%
MLPoisson::define() 11 0.01856 0.01856 0.01856 2.15%
FillBoundary_nowait() 3974 0.01852 0.01852 0.01852 2.15%
Castro::computeTemp() 63 0.01799 0.01799 0.01799 2.09%
FabArray::ParallelCopy() 851 0.01785 0.01785 0.01785 2.07%
MLMG::mgVcycle_down::3 81 0.01783 0.01783 0.01783 2.07%
FabArray::ParallelCopy_nowait() 851 0.01751 0.01751 0.01751 2.03%
MLMG::mgVcycle_down::4 81 0.01697 0.01697 0.01697 1.97%
MLMG::mgVcycle_up::2 81 0.01441 0.01441 0.01441 1.67%
MLMG::addInterpCorrection() 405 0.01421 0.01421 0.01421 1.65%
MLMG::mgVcycle_up::1 81 0.01415 0.01415 0.01415 1.64%
amrex::average_down 405 0.01385 0.01385 0.01385 1.61%
MLMG::mgVcycle_up::3 81 0.01376 0.01376 0.01376 1.60%
MLMG::mgVcycle_up::4 81 0.01356 0.01356 0.01356 1.57%
Castro::initialize_advance() 10 0.01353 0.01353 0.01353 1.57%
MLCGSolver::ParallelAllReduce 1495 0.01328 0.01328 0.01328 1.54%
CGSolver::sxay() 1566 0.01327 0.01327 0.01327 1.54%
MLCellLinOp::defineAuxData() 11 0.01284 0.01284 0.01284 1.49%
MultiFab::LinComb() 1566 0.01273 0.01273 0.01273 1.48%
Gravity::fill_multipole_BCs() 11 0.01141 0.01141 0.01141 1.32%
MLPoisson::Fapply() 1128 0.01108 0.01108 0.01108 1.29%
Castro::enforce_min_density() 62 0.01042 0.01042 0.01042 1.21%
Castro::do_new_sources() 10 0.01018 0.01018 0.01018 1.18%
Amr::InitializeInit() 1 0.009277 0.009277 0.009277 1.08%
Amr::defBaseLevel() 1 0.00927 0.00927 0.00927 1.08%
MLCellLinOp::solutionResidual() 92 0.00786 0.00786 0.00786 0.91%
Castro::do_old_sources() 10 0.007376 0.007376 0.007376 0.86%
Castro::post_timestep() 10 0.007304 0.007304 0.007304 0.85%
MLMG::computeResidual() 81 0.006757 0.006757 0.006757 0.78%
MultiFab::Xpay() 578 0.006193 0.006193 0.006193 0.72%
FabArray::setDomainBndry() 41 0.005784 0.005784 0.005784 0.67%
MLMG::prepareForSolve() 11 0.005688 0.005688 0.005688 0.66%
Castro::reset_internal_energy() 63 0.005584 0.005584 0.005584 0.65%
MLCellLinOp::defineBC() 11 0.005399 0.005399 0.005399 0.63%
BndryData::define() 11 0.005162 0.005162 0.005162 0.60%
Castro::estTimeStep() 21 0.004705 0.004705 0.004705 0.55%
Castro::enforce_speed_limit() 62 0.003511 0.003511 0.003511 0.41%
Castro::initData() 1 0.003322 0.003322 0.003322 0.39%
Castro::normalize_species() 62 0.002692 0.002692 0.002692 0.31%
Castro::construct_new_source() 50 0.002563 0.002563 0.002563 0.30%
Castro::construct_new_gravity_source() 10 0.002543 0.002543 0.002543 0.29%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002407 0.002407 0.002407 0.28%
Castro::construct_old_source() 50 0.002026 0.002026 0.002026 0.24%
Castro::computeNewDt() 9 0.002002 0.002002 0.002002 0.23%
Castro::construct_old_gravity_source() 10 0.001994 0.001994 0.001994 0.23%
MLMG::ResNormInf() 92 0.001913 0.001913 0.001913 0.22%
MLMG::getGradSolution() 11 0.001855 0.001855 0.001855 0.22%
MLCellLinOp::compGrad() 11 0.001849 0.001849 0.001849 0.21%
Castro::apply_source_to_state() 20 0.001835 0.001835 0.001835 0.21%
MultiFab::Saxpy() 20 0.001821 0.001821 0.001821 0.21%
FabArrayBase::getCPC() 1313 0.001731 0.001731 0.001731 0.20%
MLCellLinOp::setLevelBC() 11 0.001604 0.001604 0.001604 0.19%
MLPoisson::prepareForSolve() 11 0.001408 0.001408 0.001408 0.16%
MLCellLinOp::prepareForSolve() 11 0.0014 0.0014 0.0014 0.16%
FabArray::mult() 43 0.001351 0.001351 0.001351 0.16%
MLMG::computeMLResidual() 11 0.001149 0.001149 0.001149 0.13%
Gravity::swapTimeLevels() 10 0.001051 0.001051 0.001051 0.12%
Gravity::update_max_rhs() 11 0.001037 0.001037 0.001037 0.12%
FabArrayBase::getFB() 3974 0.0009272 0.0009272 0.0009272 0.11%
FabArrayBase::CPC::define() 454 0.0007933 0.0007933 0.0007933 0.09%
Castro::post_regrid() 1 0.0004962 0.0004962 0.0004962 0.06%
Castro::computeInitialDt() 2 0.0004874 0.0004874 0.0004874 0.06%
Amr::InitAmr() 1 0.0004596 0.0004596 0.0004596 0.05%
MultiFab::Copy() 11 0.0004176 0.0004176 0.0004176 0.05%
MultiFab::min() 10 0.0003435 0.0003435 0.0003435 0.04%
MLLinOp::define() 11 0.0002853 0.0002853 0.0002853 0.03%
MLMG::MLResNormInf() 11 0.0002668 0.0002668 0.0002668 0.03%
MultiFab::max() 11 0.0002653 0.0002653 0.0002653 0.03%
MLLinOp::defineGrids() 11 0.000256 0.000256 0.000256 0.03%
MLMG::MLRhsNormInf() 11 0.0002022 0.0002022 0.0002022 0.02%
FabArrayBase::FB::FB() 56 0.0001001 0.0001001 0.0001001 0.01%
Castro::finalize_advance() 10 6.149e-05 6.149e-05 6.149e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 6.008e-05 6.008e-05 6.008e-05 0.01%
AmrLevel::AmrLevel(dm) 1 5.376e-05 5.376e-05 5.376e-05 0.01%
Castro::swap_state_time_levels() 10 4.644e-05 4.644e-05 4.644e-05 0.01%
StateData::define() 4 4.351e-05 4.351e-05 4.351e-05 0.01%
makeSFC 55 4.207e-05 4.207e-05 4.207e-05 0.00%
Castro::finalize_do_advance() 10 4.13e-05 4.13e-05 4.13e-05 0.00%
Castro::enforce_consistent_e() 1 3.491e-05 3.491e-05 3.491e-05 0.00%
Amr::writeSmallPlotFile() 1 2.353e-05 2.353e-05 2.353e-05 0.00%
DistributionMapping::Distribute() 56 1.523e-05 1.523e-05 1.523e-05 0.00%
Amr::initSubcycle() 1 9.961e-06 9.961e-06 9.961e-06 0.00%
AmrLevel::checkPointPost() 3 7.213e-06 7.213e-06 7.213e-06 0.00%
MLMG::buildFineMask() 11 6.318e-06 6.318e-06 6.318e-06 0.00%
Castro::create_source_corrector() 10 5.826e-06 5.826e-06 5.826e-06 0.00%
Gravity::set_mass_offset() 11 5.01e-06 5.01e-06 5.01e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.483e-06 4.483e-06 4.483e-06 0.00%
Castro::retry_advance_ctu() 10 4.193e-06 4.193e-06 4.193e-06 0.00%
Castro::FluxRegFineAdd() 10 3.237e-06 3.237e-06 3.237e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.299e-06 2.299e-06 2.299e-06 0.00%
AmrLevel::checkPointPre() 3 1.863e-06 1.863e-06 1.863e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.01-46-g89153db246e4) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.01-46-g89153db246e4) initialized
Starting run at 14:38:26 UTC on 2022-02-01.
Successfully read inputs file ...
Castro git describe: 21.12-28-g756d00e43
AMReX git describe: 22.01-46-g89153db24
Microphysics git describe: 22.02
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.504321142
Restart time = 0.055003329 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.112143832
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.04774553
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.055715398
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.056935977
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.061944833
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.022990536 seconds
Ending run at 14:38:26 UTC on 2022-02-01.
Run time = 0.413401291
Run time without initialization = 0.35783438
Average number of zones advanced per microsecond: 3.663
Average number of zones advanced per microsecond per rank: 3.663
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.4134 ... 0.4134 ... 0.4134
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.1309 0.1309 0.1309 31.65%
VisMF::Read() 3 0.04319 0.04319 0.04319 10.45%
MLCellLinOp::applyBC() 1946 0.03771 0.03771 0.03771 9.12%
MLPoisson::Fsmooth() 1440 0.02554 0.02554 0.02554 6.18%
VisMF::Write(FabArray) 1 0.02171 0.02171 0.02171 5.25%
FabArray::setVal() 537 0.01626 0.01626 0.01626 3.93%
StateData::FillBoundary(geom) 160 0.01137 0.01137 0.01137 2.75%
MLCGSolver::bicgstab 36 0.009224 0.009224 0.009224 2.23%
MultiFab::Dot() 484 0.009157 0.009157 0.009157 2.21%
StateDataPhysBCFunct::() 20 0.007497 0.007497 0.007497 1.81%
FabArray::ParallelCopy_nowait() 380 0.007127 0.007127 0.007127 1.72%
FillBoundary_nowait() 1766 0.007124 0.007124 0.007124 1.72%
Castro::computeTemp() 30 0.006064 0.006064 0.006064 1.47%
Gravity::fill_multipole_BCs() 6 0.006016 0.006016 0.006016 1.46%
MLCellLinOp::defineAuxData() 6 0.00554 0.00554 0.00554 1.34%
MultiFab::LinComb() 690 0.005225 0.005225 0.005225 1.26%
MLPoisson::Fapply() 500 0.00455 0.00455 0.00455 1.10%
Castro::expand_state() 5 0.004011 0.004011 0.004011 0.97%
FabArray::setDomainBndry() 20 0.003924 0.003924 0.003924 0.95%
Castro::estTimeStep() 10 0.003344 0.003344 0.003344 0.81%
Castro::enforce_min_density() 30 0.003198 0.003198 0.003198 0.77%
Amr::restart() 1 0.003052 0.003052 0.003052 0.74%
MLMG::addInterpCorrection() 180 0.002994 0.002994 0.002994 0.72%
amrex::average_down 180 0.002754 0.002754 0.002754 0.67%
MultiFab::Xpay() 258 0.002625 0.002625 0.002625 0.63%
Castro::do_advance_ctu() 5 0.002499 0.002499 0.002499 0.60%
Castro::reset_internal_energy() 30 0.002447 0.002447 0.002447 0.59%
BndryData::define() 6 0.001938 0.001938 0.001938 0.47%
Gravity::get_new_grav_vector() 5 0.00183 0.00183 0.00183 0.44%
Castro::enforce_speed_limit() 30 0.001689 0.001689 0.001689 0.41%
Castro::construct_new_gravity_source() 5 0.001532 0.001532 0.001532 0.37%
MLCellLinOp::compGrad() 6 0.001441 0.001441 0.001441 0.35%
Amr::writePlotFile() 1 0.001369 0.001369 0.001369 0.33%
Castro::construct_old_gravity_source() 5 0.001328 0.001328 0.001328 0.32%
Castro::normalize_species() 30 0.001306 0.001306 0.001306 0.32%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001192 0.001192 0.001192 0.29%
MultiFab::Saxpy() 10 0.0009218 0.0009218 0.0009218 0.22%
Gravity::get_old_grav_vector() 5 0.0009083 0.0009083 0.0009083 0.22%
MLMG::ResNormInf() 42 0.0008414 0.0008414 0.0008414 0.20%
MLCellLinOp::setLevelBC() 6 0.0008223 0.0008223 0.0008223 0.20%
Gravity::actual_solve_with_mlmg() 6 0.0007379 0.0007379 0.0007379 0.18%
MLMG::oneIter() 36 0.000732 0.000732 0.000732 0.18%
MLCellLinOp::prepareForSolve() 6 0.0007071 0.0007071 0.0007071 0.17%
FabArray::mult() 22 0.0006682 0.0006682 0.0006682 0.16%
Gravity::update_max_rhs() 6 0.0005341 0.0005341 0.0005341 0.13%
MLCellLinOp::smooth() 720 0.0004917 0.0004917 0.0004917 0.12%
FabArrayBase::getCPC() 632 0.0004499 0.0004499 0.0004499 0.11%
FabArrayBase::CPC::define() 244 0.0004496 0.0004496 0.0004496 0.11%
MLMG::prepareForSolve() 6 0.0004354 0.0004354 0.0004354 0.11%
FabArray::FillBoundary() 1766 0.0003956 0.0003956 0.0003956 0.10%
MultiFab::Copy() 6 0.0003661 0.0003661 0.0003661 0.09%
Amr::InitAmr() 1 0.0003623 0.0003623 0.0003623 0.09%
FabArrayBase::getFB() 1766 0.0003308 0.0003308 0.0003308 0.08%
main() 1 0.0003034 0.0003034 0.0003034 0.07%
Gravity::solve_for_phi() 5 0.0002361 0.0002361 0.0002361 0.06%
Gravity::actual_multilevel_solve() 1 0.0002298 0.0002298 0.0002298 0.06%
CGSolver::sxay() 690 0.0002096 0.0002096 0.0002096 0.05%
MLCellLinOp::apply() 500 0.0002059 0.0002059 0.0002059 0.05%
MLMG::mgVcycle() 36 0.000188 0.000188 0.000188 0.05%
Castro::construct_new_gravity() 5 0.0001839 0.0001839 0.0001839 0.04%
Castro::construct_new_source() 25 0.0001743 0.0001743 0.0001743 0.04%
MultiFab::min() 5 0.0001619 0.0001619 0.0001619 0.04%
FabArray::ParallelCopy() 380 0.0001403 0.0001403 0.0001403 0.03%
MLCGSolver::ParallelAllReduce 659 0.0001363 0.0001363 0.0001363 0.03%
MultiFab::max() 6 0.0001362 0.0001362 0.0001362 0.03%
FillPatchIterator::Initialize 20 0.0001253 0.0001253 0.0001253 0.03%
Amr::coarseTimeStep() 5 0.0001244 0.0001244 0.0001244 0.03%
MLMG::MLRhsNormInf() 6 0.0001063 0.0001063 0.0001063 0.03%
Amr::timeStep() 5 0.0001028 0.0001028 0.0001028 0.02%
MLCellLinOp::defineBC() 6 0.0001006 0.0001006 0.0001006 0.02%
MLLinOp::defineGrids() 6 9.93e-05 9.93e-05 9.93e-05 0.02%
MLCellLinOp::correctionResidual() 216 8.833e-05 8.833e-05 8.833e-05 0.02%
Castro::create_source_corrector() 5 8.027e-05 8.027e-05 8.027e-05 0.02%
Castro::initialize_do_advance() 5 7.937e-05 7.937e-05 7.937e-05 0.02%
AmrLevel::restart() 1 7.702e-05 7.702e-05 7.702e-05 0.02%
StateData::restartDoit() 4 7.434e-05 7.434e-05 7.434e-05 0.02%
FabArrayBase::FB::FB() 26 7.022e-05 7.022e-05 7.022e-05 0.02%
Castro::post_timestep() 5 6.686e-05 6.686e-05 6.686e-05 0.02%
MLMG:computeResOfCorrection() 180 6.337e-05 6.337e-05 6.337e-05 0.02%
Castro::construct_old_source() 25 6.291e-05 6.291e-05 6.291e-05 0.02%
Castro::subcycle_advance_ctu() 5 6.198e-05 6.198e-05 6.198e-05 0.01%
Castro::initialize_advance() 5 5.561e-05 5.561e-05 5.561e-05 0.01%
MLMG::actualBottomSolve() 36 4.889e-05 4.889e-05 4.889e-05 0.01%
Castro::advance() 5 4.735e-05 4.735e-05 4.735e-05 0.01%
MLMG::mgVcycle_up::4 36 4.241e-05 4.241e-05 4.241e-05 0.01%
MLMG::mgVcycle_down::0 36 3.79e-05 3.79e-05 3.79e-05 0.01%
Castro::clean_state() 30 3.553e-05 3.553e-05 3.553e-05 0.01%
MLLinOp::define() 6 3.467e-05 3.467e-05 3.467e-05 0.01%
MLMG::mgVcycle_down::1 36 3.451e-05 3.451e-05 3.451e-05 0.01%
MLMG::mgVcycle_down::2 36 3.392e-05 3.392e-05 3.392e-05 0.01%
Castro::do_new_sources() 5 3.245e-05 3.245e-05 3.245e-05 0.01%
MLMG::mgVcycle_down::4 36 3.153e-05 3.153e-05 3.153e-05 0.01%
MLMG::mgVcycle_down::3 36 3.111e-05 3.111e-05 3.111e-05 0.01%
MLMG::solve() 6 3.103e-05 3.103e-05 3.103e-05 0.01%
Castro::post_restart() 1 2.81e-05 2.81e-05 2.81e-05 0.01%
Castro::swap_state_time_levels() 5 2.809e-05 2.809e-05 2.809e-05 0.01%
Castro::finalize_advance() 5 2.642e-05 2.642e-05 2.642e-05 0.01%
MLCellLinOp::solutionResidual() 42 2.393e-05 2.393e-05 2.393e-05 0.01%
Amr::writeSmallPlotFile() 1 2.279e-05 2.279e-05 2.279e-05 0.01%
MLMG::mgVcycle_up::0 36 2.185e-05 2.185e-05 2.185e-05 0.01%
Castro::construct_old_gravity() 5 2.181e-05 2.181e-05 2.181e-05 0.01%
MLMG::mgVcycle_up::3 36 2.103e-05 2.103e-05 2.103e-05 0.01%
MLMG::mgVcycle_up::2 36 1.983e-05 1.983e-05 1.983e-05 0.00%
Castro::finalize_do_advance() 5 1.972e-05 1.972e-05 1.972e-05 0.00%
MLMG::mgVcycle_up::1 36 1.916e-05 1.916e-05 1.916e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.842e-05 1.842e-05 1.842e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.579e-05 1.579e-05 1.579e-05 0.00%
makeSFC 30 1.552e-05 1.552e-05 1.552e-05 0.00%
MLMG::computeResidual() 36 1.537e-05 1.537e-05 1.537e-05 0.00%
MLMG::mgVcycle_bottom 36 1.516e-05 1.516e-05 1.516e-05 0.00%
MLPoisson::define() 6 1.493e-05 1.493e-05 1.493e-05 0.00%
FillPatchSingleLevel 20 1.465e-05 1.465e-05 1.465e-05 0.00%
Amr::initSubcycle() 1 8.598e-06 8.598e-06 8.598e-06 0.00%
Castro::do_old_sources() 5 8.458e-06 8.458e-06 8.458e-06 0.00%
DistributionMapping::Distribute() 31 8.324e-06 8.324e-06 8.324e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 7.967e-06 7.967e-06 7.967e-06 0.00%
Castro::apply_source_to_state() 10 6.008e-06 6.008e-06 6.008e-06 0.00%
Gravity::swapTimeLevels() 5 4.702e-06 4.702e-06 4.702e-06 0.00%
MLPoisson::prepareForSolve() 6 4.306e-06 4.306e-06 4.306e-06 0.00%
MLMG::buildFineMask() 6 3.564e-06 3.564e-06 3.564e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.361e-06 3.361e-06 3.361e-06 0.00%
MLMG::computeMLResidual() 6 3.234e-06 3.234e-06 3.234e-06 0.00%
Castro::computeNewDt() 5 3.149e-06 3.149e-06 3.149e-06 0.00%
MLMG::getGradSolution() 6 2.988e-06 2.988e-06 2.988e-06 0.00%
Gravity::set_mass_offset() 6 2.49e-06 2.49e-06 2.49e-06 0.00%
MLMG::MLResNormInf() 6 2.456e-06 2.456e-06 2.456e-06 0.00%
Castro::retry_advance_ctu() 5 1.745e-06 1.745e-06 1.745e-06 0.00%
Castro::FluxRegFineAdd() 5 1.414e-06 1.414e-06 1.414e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.086e-06 1.086e-06 1.086e-06 0.00%
AmrLevel::AmrLevel() 1 1.046e-06 1.046e-06 1.046e-06 0.00%
Amr::init() 1 1.024e-06 1.024e-06 1.024e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.4134 0.4134 0.4134 100.00%
Amr::coarseTimeStep() 5 0.3346 0.3346 0.3346 80.93%
Amr::timeStep() 5 0.3333 0.3333 0.3333 80.62%
Castro::advance() 5 0.3304 0.3304 0.3304 79.92%
Castro::subcycle_advance_ctu() 5 0.3198 0.3198 0.3198 77.36%
Castro::do_advance_ctu() 5 0.3198 0.3198 0.3198 77.34%
Castro::construct_new_gravity() 5 0.1455 0.1455 0.1455 35.20%
Gravity::solve_phi_with_mlmg() 6 0.1415 0.1415 0.1415 34.22%
Gravity::solve_for_phi() 5 0.1356 0.1356 0.1356 32.81%
Gravity::actual_solve_with_mlmg() 6 0.1353 0.1353 0.1353 32.73%
Castro::construct_ctu_hydro_source() 5 0.1309 0.1309 0.1309 31.65%
MLMG::solve() 6 0.1226 0.1226 0.1226 29.66%
MLMG::oneIter() 36 0.1147 0.1147 0.1147 27.75%
MLMG::mgVcycle() 36 0.114 0.114 0.114 27.57%
MLCellLinOp::smooth() 720 0.05971 0.05971 0.05971 14.44%
Amr::init() 1 0.05505 0.05505 0.05505 13.32%
Amr::restart() 1 0.05505 0.05505 0.05505 13.32%
MLCellLinOp::applyBC() 1946 0.04563 0.04563 0.04563 11.04%
AmrLevel::restart() 1 0.04407 0.04407 0.04407 10.66%
StateData::restartDoit() 4 0.04399 0.04399 0.04399 10.64%
VisMF::Read() 3 0.04319 0.04319 0.04319 10.45%
MLMG::mgVcycle_bottom 36 0.03284 0.03284 0.03284 7.94%
MLMG::actualBottomSolve() 36 0.03283 0.03283 0.03283 7.94%
MLCGSolver::bicgstab 36 0.03251 0.03251 0.03251 7.86%
MLPoisson::Fsmooth() 1440 0.02554 0.02554 0.02554 6.18%
FillPatchIterator::Initialize 20 0.02502 0.02502 0.02502 6.05%
Amr::writePlotFile() 1 0.02308 0.02308 0.02308 5.58%
VisMF::Write(FabArray) 1 0.02171 0.02171 0.02171 5.25%
FillPatchSingleLevel 20 0.02097 0.02097 0.02097 5.07%
StateDataPhysBCFunct::() 20 0.01887 0.01887 0.01887 4.56%
MLCellLinOp::apply() 500 0.01642 0.01642 0.01642 3.97%
FabArray::setVal() 537 0.01626 0.01626 0.01626 3.93%
MLMG::mgVcycle_down::0 36 0.01587 0.01587 0.01587 3.84%
Castro::clean_state() 30 0.01474 0.01474 0.01474 3.56%
Castro::initialize_do_advance() 5 0.01412 0.01412 0.01412 3.42%
MLMG::mgVcycle_up::0 36 0.01357 0.01357 0.01357 3.28%
Castro::expand_state() 5 0.01199 0.01199 0.01199 2.90%
StateData::FillBoundary(geom) 160 0.01137 0.01137 0.01137 2.75%
Castro::initialize_advance() 5 0.01052 0.01052 0.01052 2.54%
Gravity::get_new_grav_vector() 5 0.009696 0.009696 0.009696 2.35%
MLPoisson::define() 6 0.009407 0.009407 0.009407 2.28%
MLCellLinOp::correctionResidual() 216 0.009241 0.009241 0.009241 2.24%
MultiFab::Dot() 484 0.009157 0.009157 0.009157 2.21%
Castro::computeTemp() 30 0.008511 0.008511 0.008511 2.06%
Castro::construct_old_gravity() 5 0.008007 0.008007 0.008007 1.94%
MLMG:computeResOfCorrection() 180 0.008003 0.008003 0.008003 1.94%
Gravity::get_old_grav_vector() 5 0.007985 0.007985 0.007985 1.93%
FabArray::FillBoundary() 1766 0.007921 0.007921 0.007921 1.92%
MLMG::mgVcycle_down::1 36 0.007731 0.007731 0.007731 1.87%
FabArray::ParallelCopy() 380 0.00771 0.00771 0.00771 1.86%
FabArray::ParallelCopy_nowait() 380 0.00757 0.00757 0.00757 1.83%
FillBoundary_nowait() 1766 0.007525 0.007525 0.007525 1.82%
MLMG::mgVcycle_down::2 36 0.007448 0.007448 0.007448 1.80%
MLMG::mgVcycle_down::3 36 0.007091 0.007091 0.007091 1.72%
MLMG::mgVcycle_down::4 36 0.006801 0.006801 0.006801 1.65%
Castro::post_restart() 1 0.006723 0.006723 0.006723 1.63%
MLCellLinOp::defineAuxData() 6 0.006481 0.006481 0.006481 1.57%
Castro::do_new_sources() 5 0.006384 0.006384 0.006384 1.54%
Gravity::multilevel_solve_for_new_phi() 1 0.006306 0.006306 0.006306 1.53%
Gravity::actual_multilevel_solve() 1 0.006288 0.006288 0.006288 1.52%
Gravity::fill_multipole_BCs() 6 0.006016 0.006016 0.006016 1.46%
MLMG::mgVcycle_up::2 36 0.005792 0.005792 0.005792 1.40%
MLMG::addInterpCorrection() 180 0.005785 0.005785 0.005785 1.40%
MLMG::mgVcycle_up::1 36 0.005717 0.005717 0.005717 1.38%
amrex::average_down 180 0.005585 0.005585 0.005585 1.35%
MLCGSolver::ParallelAllReduce 659 0.005529 0.005529 0.005529 1.34%
MLMG::mgVcycle_up::3 36 0.005492 0.005492 0.005492 1.33%
MLMG::mgVcycle_up::4 36 0.005458 0.005458 0.005458 1.32%
CGSolver::sxay() 690 0.005434 0.005434 0.005434 1.31%
MultiFab::LinComb() 690 0.005225 0.005225 0.005225 1.26%
MLPoisson::Fapply() 500 0.00455 0.00455 0.00455 1.10%
FabArray::setDomainBndry() 20 0.003924 0.003924 0.003924 0.95%
Castro::do_old_sources() 5 0.003494 0.003494 0.003494 0.85%
MLMG::prepareForSolve() 6 0.003491 0.003491 0.003491 0.84%
MLCellLinOp::solutionResidual() 42 0.003389 0.003389 0.003389 0.82%
Castro::estTimeStep() 10 0.003344 0.003344 0.003344 0.81%
Castro::enforce_min_density() 30 0.003198 0.003198 0.003198 0.77%
MLMG::computeResidual() 36 0.00281 0.00281 0.00281 0.68%
Castro::post_timestep() 5 0.002771 0.002771 0.002771 0.67%
MLCellLinOp::defineBC() 6 0.002745 0.002745 0.002745 0.66%
BndryData::define() 6 0.002644 0.002644 0.002644 0.64%
MultiFab::Xpay() 258 0.002625 0.002625 0.002625 0.63%
Castro::reset_internal_energy() 30 0.002447 0.002447 0.002447 0.59%
MLMG::getGradSolution() 6 0.001734 0.001734 0.001734 0.42%
MLCellLinOp::compGrad() 6 0.001731 0.001731 0.001731 0.42%
Castro::construct_new_source() 25 0.001706 0.001706 0.001706 0.41%
Castro::enforce_speed_limit() 30 0.001689 0.001689 0.001689 0.41%
Castro::construct_new_gravity_source() 5 0.001532 0.001532 0.001532 0.37%
Castro::construct_old_source() 25 0.001391 0.001391 0.001391 0.34%
Castro::construct_old_gravity_source() 5 0.001328 0.001328 0.001328 0.32%
Castro::normalize_species() 30 0.001306 0.001306 0.001306 0.32%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001192 0.001192 0.001192 0.29%
Castro::computeNewDt() 5 0.001172 0.001172 0.001172 0.28%
Castro::apply_source_to_state() 10 0.0009278 0.0009278 0.0009278 0.22%
MultiFab::Saxpy() 10 0.0009218 0.0009218 0.0009218 0.22%
FabArrayBase::getCPC() 632 0.0008995 0.0008995 0.0008995 0.22%
Gravity::swapTimeLevels() 5 0.0008724 0.0008724 0.0008724 0.21%
MLMG::ResNormInf() 42 0.0008414 0.0008414 0.0008414 0.20%
MLCellLinOp::setLevelBC() 6 0.0008223 0.0008223 0.0008223 0.20%
Gravity::update_max_rhs() 6 0.0007632 0.0007632 0.0007632 0.18%
MLPoisson::prepareForSolve() 6 0.0007114 0.0007114 0.0007114 0.17%
MLCellLinOp::prepareForSolve() 6 0.0007071 0.0007071 0.0007071 0.17%
FabArray::mult() 22 0.0006682 0.0006682 0.0006682 0.16%
MLMG::computeMLResidual() 6 0.0005982 0.0005982 0.0005982 0.14%
FabArrayBase::CPC::define() 244 0.0004496 0.0004496 0.0004496 0.11%
FabArrayBase::getFB() 1766 0.000401 0.000401 0.000401 0.10%
Amr::InitAmr() 1 0.0003709 0.0003709 0.0003709 0.09%
MultiFab::Copy() 6 0.0003661 0.0003661 0.0003661 0.09%
MLLinOp::define() 6 0.0001657 0.0001657 0.0001657 0.04%
MultiFab::min() 5 0.0001619 0.0001619 0.0001619 0.04%
MultiFab::max() 6 0.0001362 0.0001362 0.0001362 0.03%
MLMG::MLResNormInf() 6 0.0001345 0.0001345 0.0001345 0.03%
MLLinOp::defineGrids() 6 0.0001311 0.0001311 0.0001311 0.03%
MLMG::MLRhsNormInf() 6 0.0001063 0.0001063 0.0001063 0.03%
Castro::create_source_corrector() 5 8.027e-05 8.027e-05 8.027e-05 0.02%
FabArrayBase::FB::FB() 26 7.022e-05 7.022e-05 7.022e-05 0.02%
MLLinOp::makeAgglomeratedDMap 6 3.068e-05 3.068e-05 3.068e-05 0.01%
Castro::swap_state_time_levels() 5 2.809e-05 2.809e-05 2.809e-05 0.01%
Castro::finalize_advance() 5 2.784e-05 2.784e-05 2.784e-05 0.01%
Amr::writeSmallPlotFile() 1 2.279e-05 2.279e-05 2.279e-05 0.01%
makeSFC 30 2.272e-05 2.272e-05 2.272e-05 0.01%
Castro::finalize_do_advance() 5 1.972e-05 1.972e-05 1.972e-05 0.00%
Amr::initSubcycle() 1 8.598e-06 8.598e-06 8.598e-06 0.00%
DistributionMapping::Distribute() 31 8.324e-06 8.324e-06 8.324e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.494e-06 4.494e-06 4.494e-06 0.00%
MLMG::buildFineMask() 6 3.564e-06 3.564e-06 3.564e-06 0.00%
Gravity::set_mass_offset() 6 2.49e-06 2.49e-06 2.49e-06 0.00%
Castro::retry_advance_ctu() 5 1.745e-06 1.745e-06 1.745e-06 0.00%
Castro::FluxRegFineAdd() 5 1.414e-06 1.414e-06 1.414e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.086e-06 1.086e-06 1.086e-06 0.00%
AmrLevel::AmrLevel() 1 1.046e-06 1.046e-06 1.046e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.01-46-g89153db246e4) finalized