Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.02-5-g3fb8caaf78a0) initialized
Starting run at 08:52:40 UTC on 2022-02-08.
Successfully read inputs file ...
Castro git describe: 22.02-3-g186ee03f5
AMReX git describe: 22.02-5-g3fb8caaf7
Microphysics git describe: 22.02
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.04068734 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.024789269 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.111383481
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.050627553
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.051487456
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.060008559
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.061207888
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.03864963 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.050150399
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.048193161
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.050079638
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.057741727
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.063684577
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.03885579 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.024208289 seconds
Ending run at 08:52:41 UTC on 2022-02-08.
Run time = 0.831332918
Run time without initialization = 0.706781959
Average number of zones advanced per microsecond: 3.709
Average number of zones advanced per microsecond per rank: 3.709
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.8314 ... 0.8314 ... 0.8314
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.2031 0.2031 0.2031 24.43%
VisMF::Write(FabArray) 11 0.1603 0.1603 0.1603 19.29%
MLCellLinOp::applyBC() 4379 0.09421 0.09421 0.09421 11.33%
MLPoisson::Fsmooth() 3240 0.06017 0.06017 0.06017 7.24%
FabArray::setVal() 1135 0.02627 0.02627 0.02627 3.16%
StateData::FillBoundary(geom) 328 0.02482 0.02482 0.02482 2.99%
MLCGSolver::bicgstab 81 0.02203 0.02203 0.02203 2.65%
MultiFab::Dot() 1100 0.02189 0.02189 0.02189 2.63%
FillBoundary_nowait() 3974 0.01732 0.01732 0.01732 2.08%
FabArray::ParallelCopy_nowait() 851 0.01626 0.01626 0.01626 1.96%
StateDataPhysBCFunct::() 41 0.01508 0.01508 0.01508 1.81%
Castro::computeTemp() 63 0.0134 0.0134 0.0134 1.61%
MultiFab::LinComb() 1566 0.01261 0.01261 0.01261 1.52%
Gravity::fill_multipole_BCs() 11 0.01204 0.01204 0.01204 1.45%
MLPoisson::Fapply() 1128 0.01079 0.01079 0.01079 1.30%
MLCellLinOp::defineAuxData() 11 0.01051 0.01051 0.01051 1.26%
Castro::enforce_min_density() 62 0.008461 0.008461 0.008461 1.02%
MLMG::addInterpCorrection() 405 0.007121 0.007121 0.007121 0.86%
amrex::average_down 405 0.006662 0.006662 0.006662 0.80%
MultiFab::Xpay() 578 0.006148 0.006148 0.006148 0.74%
Castro::reset_internal_energy() 63 0.005748 0.005748 0.005748 0.69%
FabArray::setDomainBndry() 41 0.005476 0.005476 0.005476 0.66%
Castro::estTimeStep() 21 0.004916 0.004916 0.004916 0.59%
Castro::expand_state() 10 0.004652 0.004652 0.004652 0.56%
BndryData::define() 11 0.003846 0.003846 0.003846 0.46%
Amr::checkPoint() 3 0.003763 0.003763 0.003763 0.45%
Castro::enforce_speed_limit() 62 0.003647 0.003647 0.003647 0.44%
Castro::do_advance_ctu() 10 0.003184 0.003184 0.003184 0.38%
Amr::writePlotFile() 2 0.003101 0.003101 0.003101 0.37%
Castro::construct_new_gravity_source() 10 0.00299 0.00299 0.00299 0.36%
Gravity::get_new_grav_vector() 11 0.002711 0.002711 0.002711 0.33%
Castro::normalize_species() 62 0.002646 0.002646 0.002646 0.32%
Castro::construct_old_gravity_source() 10 0.00246 0.00246 0.00246 0.30%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002316 0.002316 0.002316 0.28%
MLCellLinOp::compGrad() 11 0.00208 0.00208 0.00208 0.25%
MLMG::ResNormInf() 92 0.001902 0.001902 0.001902 0.23%
MultiFab::Saxpy() 20 0.001817 0.001817 0.001817 0.22%
Gravity::get_old_grav_vector() 10 0.001778 0.001778 0.001778 0.21%
MLMG::oneIter() 81 0.00165 0.00165 0.00165 0.20%
MLCellLinOp::setLevelBC() 11 0.001573 0.001573 0.001573 0.19%
Gravity::actual_solve_with_mlmg() 11 0.001402 0.001402 0.001402 0.17%
MLCellLinOp::prepareForSolve() 11 0.001364 0.001364 0.001364 0.16%
FabArray::mult() 43 0.001341 0.001341 0.001341 0.16%
Castro::initData() 1 0.001249 0.001249 0.001249 0.15%
MLCellLinOp::smooth() 1620 0.001075 0.001075 0.001075 0.13%
FabArrayBase::getCPC() 1313 0.0008897 0.0008897 0.0008897 0.11%
FabArray::FillBoundary() 3974 0.0008013 0.0008013 0.0008013 0.10%
MLMG::prepareForSolve() 11 0.0007877 0.0007877 0.0007877 0.09%
FabArrayBase::getFB() 3974 0.0007555 0.0007555 0.0007555 0.09%
FabArrayBase::CPC::define() 454 0.0007384 0.0007384 0.0007384 0.09%
Gravity::update_max_rhs() 11 0.0005531 0.0005531 0.0005531 0.07%
MultiFab::Copy() 11 0.0005264 0.0005264 0.0005264 0.06%
MLCellLinOp::apply() 1128 0.0005018 0.0005018 0.0005018 0.06%
CGSolver::sxay() 1566 0.0004719 0.0004719 0.0004719 0.06%
Amr::InitAmr() 1 0.0004618 0.0004618 0.0004618 0.06%
Gravity::solve_for_phi() 10 0.0004502 0.0004502 0.0004502 0.05%
MLMG::mgVcycle() 81 0.0003692 0.0003692 0.0003692 0.04%
MultiFab::min() 10 0.0003266 0.0003266 0.0003266 0.04%
MLCGSolver::ParallelAllReduce 1495 0.0003172 0.0003172 0.0003172 0.04%
main() 1 0.0003116 0.0003116 0.0003116 0.04%
FabArray::ParallelCopy() 851 0.0002918 0.0002918 0.0002918 0.04%
Castro::construct_new_gravity() 10 0.0002629 0.0002629 0.0002629 0.03%
FillPatchIterator::Initialize 41 0.0002565 0.0002565 0.0002565 0.03%
MultiFab::max() 11 0.0002553 0.0002553 0.0002553 0.03%
MLCellLinOp::correctionResidual() 486 0.0002336 0.0002336 0.0002336 0.03%
Gravity::actual_multilevel_solve() 1 0.0002207 0.0002207 0.0002207 0.03%
Amr::coarseTimeStep() 10 0.0002132 0.0002132 0.0002132 0.03%
MLMG::MLRhsNormInf() 11 0.0002008 0.0002008 0.0002008 0.02%
MLCellLinOp::defineBC() 11 0.0001913 0.0001913 0.0001913 0.02%
Amr::timeStep() 10 0.0001894 0.0001894 0.0001894 0.02%
MLLinOp::defineGrids() 11 0.0001739 0.0001739 0.0001739 0.02%
Amr::defBaseLevel() 1 0.0001514 0.0001514 0.0001514 0.02%
Castro::subcycle_advance_ctu() 10 0.0001498 0.0001498 0.0001498 0.02%
StateData::checkPoint() 12 0.0001374 0.0001374 0.0001374 0.02%
MLMG:computeResOfCorrection() 405 0.0001336 0.0001336 0.0001336 0.02%
MLMG::actualBottomSolve() 81 0.0001079 0.0001079 0.0001079 0.01%
MLMG::mgVcycle_down::0 81 9.381e-05 9.381e-05 9.381e-05 0.01%
FabArrayBase::FB::FB() 56 9.261e-05 9.261e-05 9.261e-05 0.01%
Castro::initialize_advance() 10 8.011e-05 8.011e-05 8.011e-05 0.01%
Castro::post_timestep() 10 7.991e-05 7.991e-05 7.991e-05 0.01%
MLMG::mgVcycle_down::1 81 7.987e-05 7.987e-05 7.987e-05 0.01%
MLMG::mgVcycle_down::2 81 7.697e-05 7.697e-05 7.697e-05 0.01%
MLMG::solve() 11 7.647e-05 7.647e-05 7.647e-05 0.01%
MLMG::mgVcycle_down::4 81 7.511e-05 7.511e-05 7.511e-05 0.01%
MLMG::mgVcycle_down::3 81 7.353e-05 7.353e-05 7.353e-05 0.01%
AmrLevel::checkPoint() 3 7.186e-05 7.186e-05 7.186e-05 0.01%
Castro::clean_state() 62 6.772e-05 6.772e-05 6.772e-05 0.01%
MLMG::mgVcycle_up::4 81 5.881e-05 5.881e-05 5.881e-05 0.01%
Castro::finalize_advance() 10 5.734e-05 5.734e-05 5.734e-05 0.01%
Castro::initialize_do_advance() 10 5.567e-05 5.567e-05 5.567e-05 0.01%
MLCellLinOp::solutionResidual() 92 5.377e-05 5.377e-05 5.377e-05 0.01%
MLMG::mgVcycle_up::0 81 4.919e-05 4.919e-05 4.919e-05 0.01%
MLMG::mgVcycle_up::1 81 4.545e-05 4.545e-05 4.545e-05 0.01%
MLMG::mgVcycle_up::3 81 4.517e-05 4.517e-05 4.517e-05 0.01%
Castro::swap_state_time_levels() 10 4.374e-05 4.374e-05 4.374e-05 0.01%
MLMG::mgVcycle_up::2 81 4.32e-05 4.32e-05 4.32e-05 0.01%
StateData::define() 4 4.109e-05 4.109e-05 4.109e-05 0.00%
Castro::construct_new_source() 50 4.091e-05 4.091e-05 4.091e-05 0.00%
Castro::advance() 10 4.081e-05 4.081e-05 4.081e-05 0.00%
Castro::finalize_do_advance() 10 3.754e-05 3.754e-05 3.754e-05 0.00%
Castro::enforce_consistent_e() 1 3.588e-05 3.588e-05 3.588e-05 0.00%
MLMG::computeResidual() 81 3.393e-05 3.393e-05 3.393e-05 0.00%
MLMG::mgVcycle_bottom 81 3.191e-05 3.191e-05 3.191e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.883e-05 2.883e-05 2.883e-05 0.00%
MLLinOp::define() 11 2.866e-05 2.866e-05 2.866e-05 0.00%
FillPatchSingleLevel 41 2.758e-05 2.758e-05 2.758e-05 0.00%
makeSFC 55 2.456e-05 2.456e-05 2.456e-05 0.00%
Amr::writeSmallPlotFile() 1 2.19e-05 2.19e-05 2.19e-05 0.00%
MLPoisson::define() 11 2.14e-05 2.14e-05 2.14e-05 0.00%
Castro::construct_old_source() 50 2.049e-05 2.049e-05 2.049e-05 0.00%
Amr::FinalizeInit() 1 2.025e-05 2.025e-05 2.025e-05 0.00%
DistributionMapping::Distribute() 56 1.767e-05 1.767e-05 1.767e-05 0.00%
Castro::do_new_sources() 10 1.688e-05 1.688e-05 1.688e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.634e-05 1.634e-05 1.634e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.549e-05 1.549e-05 1.549e-05 0.00%
Castro::do_old_sources() 10 1.54e-05 1.54e-05 1.54e-05 0.00%
Castro::apply_source_to_state() 20 1.259e-05 1.259e-05 1.259e-05 0.00%
AmrLevel::AmrLevel(dm) 1 9.618e-06 9.618e-06 9.618e-06 0.00%
Amr::initSubcycle() 1 9.584e-06 9.584e-06 9.584e-06 0.00%
Gravity::swapTimeLevels() 10 9.325e-06 9.325e-06 9.325e-06 0.00%
Castro::construct_old_gravity() 10 9.177e-06 9.177e-06 9.177e-06 0.00%
MLPoisson::prepareForSolve() 11 7.58e-06 7.58e-06 7.58e-06 0.00%
Amr::InitializeInit() 1 6.77e-06 6.77e-06 6.77e-06 0.00%
MLMG::computeMLResidual() 11 6.615e-06 6.615e-06 6.615e-06 0.00%
MLMG::getGradSolution() 11 6.269e-06 6.269e-06 6.269e-06 0.00%
Castro::computeNewDt() 9 5.983e-06 5.983e-06 5.983e-06 0.00%
MLMG::buildFineMask() 11 5.619e-06 5.619e-06 5.619e-06 0.00%
Castro::create_source_corrector() 10 5.049e-06 5.049e-06 5.049e-06 0.00%
Gravity::set_mass_offset() 11 4.749e-06 4.749e-06 4.749e-06 0.00%
MLMG::MLResNormInf() 11 4.678e-06 4.678e-06 4.678e-06 0.00%
AmrLevel::checkPointPost() 3 4.3e-06 4.3e-06 4.3e-06 0.00%
Castro::post_init() 1 4.078e-06 4.078e-06 4.078e-06 0.00%
Castro::retry_advance_ctu() 10 3.959e-06 3.959e-06 3.959e-06 0.00%
Castro::FluxRegFineAdd() 10 3.182e-06 3.182e-06 3.182e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.833e-06 2.833e-06 2.833e-06 0.00%
Amr::init() 1 2.471e-06 2.471e-06 2.471e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.359e-06 2.359e-06 2.359e-06 0.00%
Castro::computeInitialDt() 2 2.045e-06 2.045e-06 2.045e-06 0.00%
AmrLevel::checkPointPre() 3 1.651e-06 1.651e-06 1.651e-06 0.00%
Castro::post_regrid() 1 1.21e-06 1.21e-06 1.21e-06 0.00%
Amr::initialInit() 1 1.06e-06 1.06e-06 1.06e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.8313 0.8313 0.8313 100.00%
Amr::coarseTimeStep() 10 0.6823 0.6823 0.6823 82.08%
Amr::timeStep() 10 0.6021 0.6021 0.6021 72.42%
Castro::advance() 10 0.5941 0.5941 0.5941 71.46%
Castro::subcycle_advance_ctu() 10 0.5811 0.5811 0.5811 69.90%
Castro::do_advance_ctu() 10 0.5809 0.5809 0.5809 69.88%
Gravity::solve_phi_with_mlmg() 11 0.3289 0.3289 0.3289 39.56%
Gravity::actual_solve_with_mlmg() 11 0.3166 0.3166 0.3166 38.08%
Castro::construct_new_gravity() 10 0.3017 0.3017 0.3017 36.29%
MLMG::solve() 11 0.2931 0.2931 0.2931 35.26%
Gravity::solve_for_phi() 10 0.2836 0.2836 0.2836 34.11%
MLMG::oneIter() 81 0.2772 0.2772 0.2772 33.35%
MLMG::mgVcycle() 81 0.2756 0.2756 0.2756 33.15%
Castro::construct_ctu_hydro_source() 10 0.2031 0.2031 0.2031 24.43%
VisMF::Write(FabArray) 11 0.1603 0.1603 0.1603 19.29%
MLCellLinOp::smooth() 1620 0.1451 0.1451 0.1451 17.45%
Amr::init() 1 0.1239 0.1239 0.1239 14.91%
Amr::checkPoint() 3 0.1183 0.1183 0.1183 14.23%
AmrLevel::checkPoint() 3 0.1145 0.1145 0.1145 13.78%
StateData::checkPoint() 12 0.1145 0.1145 0.1145 13.77%
MLCellLinOp::applyBC() 4379 0.1132 0.1132 0.1132 13.61%
MLMG::mgVcycle_bottom 81 0.07932 0.07932 0.07932 9.54%
MLMG::actualBottomSolve() 81 0.07929 0.07929 0.07929 9.54%
MLCGSolver::bicgstab 81 0.07854 0.07854 0.07854 9.45%
MLPoisson::Fsmooth() 3240 0.06017 0.06017 0.06017 7.24%
Amr::initialInit() 1 0.05836 0.05836 0.05836 7.02%
FillPatchIterator::Initialize 41 0.04994 0.04994 0.04994 6.01%
Amr::FinalizeInit() 1 0.04924 0.04924 0.04924 5.92%
Amr::writePlotFile() 2 0.0491 0.0491 0.0491 5.91%
Castro::post_init() 1 0.04851 0.04851 0.04851 5.84%
Gravity::multilevel_solve_for_new_phi() 1 0.04596 0.04596 0.04596 5.53%
Gravity::actual_multilevel_solve() 1 0.04595 0.04595 0.04595 5.53%
FillPatchSingleLevel 41 0.0442 0.0442 0.0442 5.32%
MLCellLinOp::apply() 1128 0.04008 0.04008 0.04008 4.82%
StateDataPhysBCFunct::() 41 0.0399 0.0399 0.0399 4.80%
MLMG::mgVcycle_down::0 81 0.03805 0.03805 0.03805 4.58%
Castro::clean_state() 62 0.03323 0.03323 0.03323 4.00%
MLMG::mgVcycle_up::0 81 0.03266 0.03266 0.03266 3.93%
FabArray::setVal() 1135 0.02627 0.02627 0.02627 3.16%
StateData::FillBoundary(geom) 328 0.02482 0.02482 0.02482 2.99%
MLCellLinOp::correctionResidual() 486 0.02247 0.02247 0.02247 2.70%
Castro::initialize_do_advance() 10 0.02234 0.02234 0.02234 2.69%
MultiFab::Dot() 1100 0.02189 0.02189 0.02189 2.63%
Gravity::get_new_grav_vector() 11 0.02025 0.02025 0.02025 2.44%
MLMG:computeResOfCorrection() 405 0.01943 0.01943 0.01943 2.34%
Castro::computeTemp() 63 0.01915 0.01915 0.01915 2.30%
FabArray::FillBoundary() 3974 0.01897 0.01897 0.01897 2.28%
MLMG::mgVcycle_down::1 81 0.01867 0.01867 0.01867 2.25%
FillBoundary_nowait() 3974 0.01817 0.01817 0.01817 2.19%
MLMG::mgVcycle_down::2 81 0.01812 0.01812 0.01812 2.18%
Castro::expand_state() 10 0.01796 0.01796 0.01796 2.16%
MLPoisson::define() 11 0.01785 0.01785 0.01785 2.15%
FabArray::ParallelCopy() 851 0.01748 0.01748 0.01748 2.10%
MLMG::mgVcycle_down::3 81 0.0173 0.0173 0.0173 2.08%
FabArray::ParallelCopy_nowait() 851 0.01719 0.01719 0.01719 2.07%
MLMG::mgVcycle_down::4 81 0.01657 0.01657 0.01657 1.99%
Castro::construct_old_gravity() 10 0.01637 0.01637 0.01637 1.97%
Gravity::get_old_grav_vector() 10 0.01636 0.01636 0.01636 1.97%
MLMG::mgVcycle_up::2 81 0.01407 0.01407 0.01407 1.69%
MLMG::mgVcycle_up::1 81 0.01383 0.01383 0.01383 1.66%
MLMG::addInterpCorrection() 405 0.01372 0.01372 0.01372 1.65%
MLMG::mgVcycle_up::3 81 0.01337 0.01337 0.01337 1.61%
amrex::average_down 405 0.01326 0.01326 0.01326 1.60%
MLMG::mgVcycle_up::4 81 0.01322 0.01322 0.01322 1.59%
MLCGSolver::ParallelAllReduce 1495 0.01317 0.01317 0.01317 1.58%
CGSolver::sxay() 1566 0.01309 0.01309 0.01309 1.57%
Castro::initialize_advance() 10 0.01288 0.01288 0.01288 1.55%
MultiFab::LinComb() 1566 0.01261 0.01261 0.01261 1.52%
MLCellLinOp::defineAuxData() 11 0.01222 0.01222 0.01222 1.47%
Gravity::fill_multipole_BCs() 11 0.01204 0.01204 0.01204 1.45%
Castro::do_new_sources() 10 0.0113 0.0113 0.0113 1.36%
MLPoisson::Fapply() 1128 0.01079 0.01079 0.01079 1.30%
Amr::InitializeInit() 1 0.009123 0.009123 0.009123 1.10%
Amr::defBaseLevel() 1 0.009117 0.009117 0.009117 1.10%
Castro::enforce_min_density() 62 0.008461 0.008461 0.008461 1.02%
Castro::post_timestep() 10 0.00783 0.00783 0.00783 0.94%
MLCellLinOp::solutionResidual() 92 0.007746 0.007746 0.007746 0.93%
Castro::do_old_sources() 10 0.007092 0.007092 0.007092 0.85%
MLMG::computeResidual() 81 0.006688 0.006688 0.006688 0.80%
MultiFab::Xpay() 578 0.006148 0.006148 0.006148 0.74%
MLMG::prepareForSolve() 11 0.00592 0.00592 0.00592 0.71%
Castro::reset_internal_energy() 63 0.005748 0.005748 0.005748 0.69%
FabArray::setDomainBndry() 41 0.005476 0.005476 0.005476 0.66%
MLCellLinOp::defineBC() 11 0.005349 0.005349 0.005349 0.64%
BndryData::define() 11 0.005158 0.005158 0.005158 0.62%
Castro::estTimeStep() 21 0.004916 0.004916 0.004916 0.59%
Castro::enforce_speed_limit() 62 0.003647 0.003647 0.003647 0.44%
Castro::initData() 1 0.003363 0.003363 0.003363 0.40%
Castro::construct_new_source() 50 0.003031 0.003031 0.003031 0.36%
Castro::construct_new_gravity_source() 10 0.00299 0.00299 0.00299 0.36%
Castro::normalize_species() 62 0.002646 0.002646 0.002646 0.32%
MLMG::getGradSolution() 11 0.002631 0.002631 0.002631 0.32%
MLCellLinOp::compGrad() 11 0.002625 0.002625 0.002625 0.32%
Castro::construct_old_source() 50 0.002481 0.002481 0.002481 0.30%
Castro::construct_old_gravity_source() 10 0.00246 0.00246 0.00246 0.30%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002316 0.002316 0.002316 0.28%
Castro::computeNewDt() 9 0.002023 0.002023 0.002023 0.24%
MLMG::ResNormInf() 92 0.001902 0.001902 0.001902 0.23%
Castro::apply_source_to_state() 20 0.001829 0.001829 0.001829 0.22%
MultiFab::Saxpy() 20 0.001817 0.001817 0.001817 0.22%
FabArrayBase::getCPC() 1313 0.001628 0.001628 0.001628 0.20%
MLCellLinOp::setLevelBC() 11 0.001573 0.001573 0.001573 0.19%
MLPoisson::prepareForSolve() 11 0.001372 0.001372 0.001372 0.16%
MLCellLinOp::prepareForSolve() 11 0.001364 0.001364 0.001364 0.16%
FabArray::mult() 43 0.001341 0.001341 0.001341 0.16%
MLMG::computeMLResidual() 11 0.001098 0.001098 0.001098 0.13%
Gravity::swapTimeLevels() 10 0.001078 0.001078 0.001078 0.13%
Gravity::update_max_rhs() 11 0.0009547 0.0009547 0.0009547 0.11%
FabArrayBase::getFB() 3974 0.0008481 0.0008481 0.0008481 0.10%
FabArrayBase::CPC::define() 454 0.0007384 0.0007384 0.0007384 0.09%
Castro::computeInitialDt() 2 0.0006502 0.0006502 0.0006502 0.08%
MultiFab::Copy() 11 0.0005264 0.0005264 0.0005264 0.06%
Castro::post_regrid() 1 0.0004918 0.0004918 0.0004918 0.06%
Amr::InitAmr() 1 0.0004714 0.0004714 0.0004714 0.06%
MultiFab::min() 10 0.0003266 0.0003266 0.0003266 0.04%
MLLinOp::define() 11 0.0002618 0.0002618 0.0002618 0.03%
MLMG::MLResNormInf() 11 0.0002579 0.0002579 0.0002579 0.03%
MultiFab::max() 11 0.0002553 0.0002553 0.0002553 0.03%
MLLinOp::defineGrids() 11 0.0002331 0.0002331 0.0002331 0.03%
MLMG::MLRhsNormInf() 11 0.0002008 0.0002008 0.0002008 0.02%
FabArrayBase::FB::FB() 56 9.261e-05 9.261e-05 9.261e-05 0.01%
Castro::finalize_advance() 10 6.053e-05 6.053e-05 6.053e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.692e-05 5.692e-05 5.692e-05 0.01%
AmrLevel::AmrLevel(dm) 1 5.071e-05 5.071e-05 5.071e-05 0.01%
Castro::swap_state_time_levels() 10 4.374e-05 4.374e-05 4.374e-05 0.01%
makeSFC 55 4.143e-05 4.143e-05 4.143e-05 0.00%
StateData::define() 4 4.109e-05 4.109e-05 4.109e-05 0.00%
Castro::finalize_do_advance() 10 3.754e-05 3.754e-05 3.754e-05 0.00%
Castro::enforce_consistent_e() 1 3.588e-05 3.588e-05 3.588e-05 0.00%
Amr::writeSmallPlotFile() 1 2.19e-05 2.19e-05 2.19e-05 0.00%
DistributionMapping::Distribute() 56 1.767e-05 1.767e-05 1.767e-05 0.00%
Amr::initSubcycle() 1 9.584e-06 9.584e-06 9.584e-06 0.00%
MLMG::buildFineMask() 11 5.619e-06 5.619e-06 5.619e-06 0.00%
Castro::create_source_corrector() 10 5.049e-06 5.049e-06 5.049e-06 0.00%
Gravity::set_mass_offset() 11 4.749e-06 4.749e-06 4.749e-06 0.00%
AmrLevel::checkPointPost() 3 4.3e-06 4.3e-06 4.3e-06 0.00%
Castro::retry_advance_ctu() 10 3.959e-06 3.959e-06 3.959e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.633e-06 3.633e-06 3.633e-06 0.00%
Castro::FluxRegFineAdd() 10 3.182e-06 3.182e-06 3.182e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.359e-06 2.359e-06 2.359e-06 0.00%
AmrLevel::checkPointPre() 3 1.651e-06 1.651e-06 1.651e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.02-5-g3fb8caaf78a0) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.02-5-g3fb8caaf78a0) initialized
Starting run at 08:52:41 UTC on 2022-02-08.
Successfully read inputs file ...
Castro git describe: 22.02-3-g186ee03f5
AMReX git describe: 22.02-5-g3fb8caaf7
Microphysics git describe: 22.02
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.497850345
Restart time = 0.056373003 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.116980344
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.049069471
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.056855137
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.059134133
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.064387316
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.02474054 seconds
Ending run at 08:52:42 UTC on 2022-02-08.
Run time = 0.428490051
Run time without initialization = 0.371507635
Average number of zones advanced per microsecond: 3.528
Average number of zones advanced per microsecond per rank: 3.528
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.4285 ... 0.4285 ... 0.4285
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.1363 0.1363 0.1363 31.80%
VisMF::Read() 3 0.04409 0.04409 0.04409 10.29%
MLCellLinOp::applyBC() 1946 0.03833 0.03833 0.03833 8.94%
MLPoisson::Fsmooth() 1440 0.0261 0.0261 0.0261 6.09%
VisMF::Write(FabArray) 1 0.02345 0.02345 0.02345 5.47%
FabArray::setVal() 537 0.01628 0.01628 0.01628 3.80%
StateData::FillBoundary(geom) 160 0.01166 0.01166 0.01166 2.72%
MLCGSolver::bicgstab 36 0.009572 0.009572 0.009572 2.23%
MultiFab::Dot() 484 0.009548 0.009548 0.009548 2.23%
Castro::computeTemp() 30 0.008202 0.008202 0.008202 1.91%
StateDataPhysBCFunct::() 20 0.007451 0.007451 0.007451 1.74%
FillBoundary_nowait() 1766 0.007292 0.007292 0.007292 1.70%
FabArray::ParallelCopy_nowait() 380 0.007206 0.007206 0.007206 1.68%
Gravity::fill_multipole_BCs() 6 0.006393 0.006393 0.006393 1.49%
MLCellLinOp::defineAuxData() 6 0.005634 0.005634 0.005634 1.31%
MultiFab::LinComb() 690 0.005488 0.005488 0.005488 1.28%
Castro::enforce_min_density() 30 0.005302 0.005302 0.005302 1.24%
MLPoisson::Fapply() 500 0.00475 0.00475 0.00475 1.11%
FabArray::setDomainBndry() 20 0.003861 0.003861 0.003861 0.90%
Castro::expand_state() 5 0.003577 0.003577 0.003577 0.83%
Castro::estTimeStep() 10 0.003336 0.003336 0.003336 0.78%
MLMG::addInterpCorrection() 180 0.003047 0.003047 0.003047 0.71%
Amr::restart() 1 0.003033 0.003033 0.003033 0.71%
amrex::average_down 180 0.002832 0.002832 0.002832 0.66%
MultiFab::Xpay() 258 0.002726 0.002726 0.002726 0.64%
Castro::do_advance_ctu() 5 0.002622 0.002622 0.002622 0.61%
Castro::reset_internal_energy() 30 0.002502 0.002502 0.002502 0.58%
BndryData::define() 6 0.002076 0.002076 0.002076 0.48%
Gravity::get_new_grav_vector() 5 0.001779 0.001779 0.001779 0.42%
MLCellLinOp::compGrad() 6 0.001525 0.001525 0.001525 0.36%
Amr::writePlotFile() 1 0.001372 0.001372 0.001372 0.32%
Castro::normalize_species() 30 0.001328 0.001328 0.001328 0.31%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001233 0.001233 0.001233 0.29%
Castro::construct_new_gravity_source() 5 0.001161 0.001161 0.001161 0.27%
MultiFab::Saxpy() 10 0.0009266 0.0009266 0.0009266 0.22%
Gravity::get_old_grav_vector() 5 0.000923 0.000923 0.000923 0.22%
Castro::construct_new_source() 25 0.000898 0.000898 0.000898 0.21%
MLMG::ResNormInf() 42 0.0008503 0.0008503 0.0008503 0.20%
MLCellLinOp::setLevelBC() 6 0.0008444 0.0008444 0.0008444 0.20%
Castro::construct_old_gravity_source() 5 0.0008203 0.0008203 0.0008203 0.19%
Castro::post_timestep() 5 0.0008129 0.0008129 0.0008129 0.19%
MLMG::oneIter() 36 0.0007455 0.0007455 0.0007455 0.17%
Gravity::actual_solve_with_mlmg() 6 0.0007283 0.0007283 0.0007283 0.17%
MLCellLinOp::prepareForSolve() 6 0.0007059 0.0007059 0.0007059 0.16%
FabArray::mult() 22 0.0006602 0.0006602 0.0006602 0.15%
Gravity::update_max_rhs() 6 0.0005727 0.0005727 0.0005727 0.13%
MLCellLinOp::smooth() 720 0.0004819 0.0004819 0.0004819 0.11%
FabArrayBase::CPC::define() 244 0.0004409 0.0004409 0.0004409 0.10%
MLMG::prepareForSolve() 6 0.0004408 0.0004408 0.0004408 0.10%
Castro::enforce_speed_limit() 30 0.0004355 0.0004355 0.0004355 0.10%
FabArrayBase::getCPC() 632 0.0004316 0.0004316 0.0004316 0.10%
Amr::InitAmr() 1 0.0004077 0.0004077 0.0004077 0.10%
FabArray::FillBoundary() 1766 0.0003963 0.0003963 0.0003963 0.09%
FabArrayBase::getFB() 1766 0.0003058 0.0003058 0.0003058 0.07%
main() 1 0.0002981 0.0002981 0.0002981 0.07%
MultiFab::Copy() 6 0.0002974 0.0002974 0.0002974 0.07%
Gravity::actual_multilevel_solve() 1 0.0002431 0.0002431 0.0002431 0.06%
Gravity::solve_for_phi() 5 0.000238 0.000238 0.000238 0.06%
MLCellLinOp::apply() 500 0.0002192 0.0002192 0.0002192 0.05%
MultiFab::min() 5 0.0001955 0.0001955 0.0001955 0.05%
Castro::construct_new_gravity() 5 0.000188 0.000188 0.000188 0.04%
CGSolver::sxay() 690 0.0001874 0.0001874 0.0001874 0.04%
MLMG::mgVcycle() 36 0.0001616 0.0001616 0.0001616 0.04%
FabArray::ParallelCopy() 380 0.0001437 0.0001437 0.0001437 0.03%
MultiFab::max() 6 0.0001342 0.0001342 0.0001342 0.03%
MLCGSolver::ParallelAllReduce 659 0.0001323 0.0001323 0.0001323 0.03%
FillPatchIterator::Initialize 20 0.0001255 0.0001255 0.0001255 0.03%
Amr::coarseTimeStep() 5 0.0001127 0.0001127 0.0001127 0.03%
MLMG::MLRhsNormInf() 6 0.0001073 0.0001073 0.0001073 0.03%
MLCellLinOp::defineBC() 6 0.0001068 0.0001068 0.0001068 0.02%
MLCellLinOp::correctionResidual() 216 0.0001049 0.0001049 0.0001049 0.02%
MLLinOp::defineGrids() 6 9.677e-05 9.677e-05 9.677e-05 0.02%
Amr::timeStep() 5 9.593e-05 9.593e-05 9.593e-05 0.02%
AmrLevel::restart() 1 7.922e-05 7.922e-05 7.922e-05 0.02%
StateData::restartDoit() 4 7.881e-05 7.881e-05 7.881e-05 0.02%
Castro::construct_old_source() 25 7.621e-05 7.621e-05 7.621e-05 0.02%
Castro::subcycle_advance_ctu() 5 7.411e-05 7.411e-05 7.411e-05 0.02%
Castro::initialize_do_advance() 5 6.452e-05 6.452e-05 6.452e-05 0.02%
MLMG:computeResOfCorrection() 180 6.338e-05 6.338e-05 6.338e-05 0.01%
FabArrayBase::FB::FB() 26 6.338e-05 6.338e-05 6.338e-05 0.01%
Castro::initialize_advance() 5 5.839e-05 5.839e-05 5.839e-05 0.01%
Castro::create_source_corrector() 5 5.272e-05 5.272e-05 5.272e-05 0.01%
MLMG::actualBottomSolve() 36 4.77e-05 4.77e-05 4.77e-05 0.01%
MLMG::mgVcycle_down::0 36 3.877e-05 3.877e-05 3.877e-05 0.01%
Castro::do_new_sources() 5 3.823e-05 3.823e-05 3.823e-05 0.01%
MLMG::solve() 6 3.491e-05 3.491e-05 3.491e-05 0.01%
Castro::advance() 5 3.445e-05 3.445e-05 3.445e-05 0.01%
MLMG::mgVcycle_down::1 36 3.318e-05 3.318e-05 3.318e-05 0.01%
Castro::clean_state() 30 3.26e-05 3.26e-05 3.26e-05 0.01%
MLMG::mgVcycle_down::2 36 3.178e-05 3.178e-05 3.178e-05 0.01%
MLMG::mgVcycle_down::3 36 3.053e-05 3.053e-05 3.053e-05 0.01%
MLMG::mgVcycle_down::4 36 3.04e-05 3.04e-05 3.04e-05 0.01%
MLLinOp::define() 6 2.945e-05 2.945e-05 2.945e-05 0.01%
Castro::finalize_advance() 5 2.721e-05 2.721e-05 2.721e-05 0.01%
Castro::post_restart() 1 2.721e-05 2.721e-05 2.721e-05 0.01%
Castro::swap_state_time_levels() 5 2.665e-05 2.665e-05 2.665e-05 0.01%
MLMG::mgVcycle_up::4 36 2.579e-05 2.579e-05 2.579e-05 0.01%
MLCellLinOp::solutionResidual() 42 2.497e-05 2.497e-05 2.497e-05 0.01%
Castro::construct_old_gravity() 5 2.451e-05 2.451e-05 2.451e-05 0.01%
Amr::writeSmallPlotFile() 1 2.378e-05 2.378e-05 2.378e-05 0.01%
MLMG::mgVcycle_up::0 36 2.22e-05 2.22e-05 2.22e-05 0.01%
MLMG::mgVcycle_up::3 36 2.011e-05 2.011e-05 2.011e-05 0.00%
Castro::finalize_do_advance() 5 1.966e-05 1.966e-05 1.966e-05 0.00%
MLMG::mgVcycle_up::2 36 1.919e-05 1.919e-05 1.919e-05 0.00%
MLMG::mgVcycle_up::1 36 1.915e-05 1.915e-05 1.915e-05 0.00%
MLPoisson::define() 6 1.664e-05 1.664e-05 1.664e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.614e-05 1.614e-05 1.614e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.573e-05 1.573e-05 1.573e-05 0.00%
MLMG::mgVcycle_bottom 36 1.427e-05 1.427e-05 1.427e-05 0.00%
MLMG::computeResidual() 36 1.41e-05 1.41e-05 1.41e-05 0.00%
makeSFC 30 1.387e-05 1.387e-05 1.387e-05 0.00%
FillPatchSingleLevel 20 1.345e-05 1.345e-05 1.345e-05 0.00%
Amr::initSubcycle() 1 1.038e-05 1.038e-05 1.038e-05 0.00%
MLLinOp::makeAgglomeratedDMap 6 8.83e-06 8.83e-06 8.83e-06 0.00%
Castro::do_old_sources() 5 8.246e-06 8.246e-06 8.246e-06 0.00%
DistributionMapping::Distribute() 31 8.021e-06 8.021e-06 8.021e-06 0.00%
Castro::apply_source_to_state() 10 5.655e-06 5.655e-06 5.655e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.811e-06 4.811e-06 4.811e-06 0.00%
MLPoisson::prepareForSolve() 6 4.67e-06 4.67e-06 4.67e-06 0.00%
Gravity::swapTimeLevels() 5 4.418e-06 4.418e-06 4.418e-06 0.00%
Castro::computeNewDt() 5 4.251e-06 4.251e-06 4.251e-06 0.00%
MLMG::buildFineMask() 6 3.556e-06 3.556e-06 3.556e-06 0.00%
MLMG::computeMLResidual() 6 3.46e-06 3.46e-06 3.46e-06 0.00%
MLMG::getGradSolution() 6 2.835e-06 2.835e-06 2.835e-06 0.00%
MLMG::MLResNormInf() 6 2.404e-06 2.404e-06 2.404e-06 0.00%
Gravity::set_mass_offset() 6 2.144e-06 2.144e-06 2.144e-06 0.00%
Castro::retry_advance_ctu() 5 1.573e-06 1.573e-06 1.573e-06 0.00%
Castro::FluxRegFineAdd() 5 1.429e-06 1.429e-06 1.429e-06 0.00%
Amr::init() 1 1.186e-06 1.186e-06 1.186e-06 0.00%
AmrLevel::AmrLevel() 1 1.074e-06 1.074e-06 1.074e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.062e-06 1.062e-06 1.062e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.4285 0.4285 0.4285 100.00%
Amr::coarseTimeStep() 5 0.3465 0.3465 0.3465 80.86%
Amr::timeStep() 5 0.345 0.345 0.345 80.50%
Castro::advance() 5 0.3407 0.3407 0.3407 79.51%
Castro::subcycle_advance_ctu() 5 0.3297 0.3297 0.3297 76.94%
Castro::do_advance_ctu() 5 0.3297 0.3297 0.3297 76.93%
Castro::construct_new_gravity() 5 0.1489 0.1489 0.1489 34.75%
Gravity::solve_phi_with_mlmg() 6 0.1452 0.1452 0.1452 33.88%
Gravity::solve_for_phi() 5 0.1391 0.1391 0.1391 32.46%
Gravity::actual_solve_with_mlmg() 6 0.1386 0.1386 0.1386 32.35%
Castro::construct_ctu_hydro_source() 5 0.1363 0.1363 0.1363 31.80%
MLMG::solve() 6 0.1256 0.1256 0.1256 29.30%
MLMG::oneIter() 36 0.1175 0.1175 0.1175 27.43%
MLMG::mgVcycle() 36 0.1168 0.1168 0.1168 27.25%
MLCellLinOp::smooth() 720 0.06082 0.06082 0.06082 14.19%
Amr::init() 1 0.05642 0.05642 0.05642 13.17%
Amr::restart() 1 0.05642 0.05642 0.05642 13.17%
MLCellLinOp::applyBC() 1946 0.04638 0.04638 0.04638 10.82%
AmrLevel::restart() 1 0.04503 0.04503 0.04503 10.51%
StateData::restartDoit() 4 0.04495 0.04495 0.04495 10.49%
VisMF::Read() 3 0.04409 0.04409 0.04409 10.29%
MLMG::mgVcycle_bottom 36 0.0341 0.0341 0.0341 7.96%
MLMG::actualBottomSolve() 36 0.03408 0.03408 0.03408 7.95%
MLCGSolver::bicgstab 36 0.03376 0.03376 0.03376 7.88%
MLPoisson::Fsmooth() 1440 0.0261 0.0261 0.0261 6.09%
FillPatchIterator::Initialize 20 0.02519 0.02519 0.02519 5.88%
Amr::writePlotFile() 1 0.02483 0.02483 0.02483 5.79%
VisMF::Write(FabArray) 1 0.02345 0.02345 0.02345 5.47%
FillPatchSingleLevel 20 0.02121 0.02121 0.02121 4.95%
StateDataPhysBCFunct::() 20 0.01911 0.01911 0.01911 4.46%
Castro::clean_state() 30 0.0178 0.0178 0.0178 4.15%
MLCellLinOp::apply() 500 0.01682 0.01682 0.01682 3.92%
FabArray::setVal() 537 0.01628 0.01628 0.01628 3.80%
MLMG::mgVcycle_down::0 36 0.01617 0.01617 0.01617 3.77%
Castro::initialize_do_advance() 5 0.01445 0.01445 0.01445 3.37%
MLMG::mgVcycle_up::0 36 0.01379 0.01379 0.01379 3.22%
StateData::FillBoundary(geom) 160 0.01166 0.01166 0.01166 2.72%
Castro::expand_state() 5 0.01159 0.01159 0.01159 2.71%
Castro::initialize_advance() 5 0.01092 0.01092 0.01092 2.55%
Castro::computeTemp() 30 0.0107 0.0107 0.0107 2.50%
MLPoisson::define() 6 0.009666 0.009666 0.009666 2.26%
Gravity::get_new_grav_vector() 5 0.009634 0.009634 0.009634 2.25%
MultiFab::Dot() 484 0.009548 0.009548 0.009548 2.23%
MLCellLinOp::correctionResidual() 216 0.009519 0.009519 0.009519 2.22%
MLMG:computeResOfCorrection() 180 0.008234 0.008234 0.008234 1.92%
FabArray::FillBoundary() 1766 0.008058 0.008058 0.008058 1.88%
Castro::construct_old_gravity() 5 0.008024 0.008024 0.008024 1.87%
Gravity::get_old_grav_vector() 5 0.007999 0.007999 0.007999 1.87%
MLMG::mgVcycle_down::1 36 0.007892 0.007892 0.007892 1.84%
FabArray::ParallelCopy() 380 0.007779 0.007779 0.007779 1.82%
FillBoundary_nowait() 1766 0.007661 0.007661 0.007661 1.79%
FabArray::ParallelCopy_nowait() 380 0.007635 0.007635 0.007635 1.78%
MLMG::mgVcycle_down::2 36 0.007585 0.007585 0.007585 1.77%
MLMG::mgVcycle_down::3 36 0.007266 0.007266 0.007266 1.70%
Castro::post_restart() 1 0.007006 0.007006 0.007006 1.63%
MLMG::mgVcycle_down::4 36 0.006952 0.006952 0.006952 1.62%
MLCellLinOp::defineAuxData() 6 0.006583 0.006583 0.006583 1.54%
Gravity::multilevel_solve_for_new_phi() 1 0.006568 0.006568 0.006568 1.53%
Gravity::actual_multilevel_solve() 1 0.006551 0.006551 0.006551 1.53%
Castro::do_new_sources() 5 0.006397 0.006397 0.006397 1.49%
Gravity::fill_multipole_BCs() 6 0.006393 0.006393 0.006393 1.49%
MLMG::mgVcycle_up::2 36 0.0059 0.0059 0.0059 1.38%
MLMG::addInterpCorrection() 180 0.005884 0.005884 0.005884 1.37%
MLMG::mgVcycle_up::1 36 0.005818 0.005818 0.005818 1.36%
MLCGSolver::ParallelAllReduce 659 0.005754 0.005754 0.005754 1.34%
amrex::average_down 180 0.00569 0.00569 0.00569 1.33%
CGSolver::sxay() 690 0.005675 0.005675 0.005675 1.32%
MLMG::mgVcycle_up::3 36 0.005593 0.005593 0.005593 1.31%
MLMG::mgVcycle_up::4 36 0.005563 0.005563 0.005563 1.30%
MultiFab::LinComb() 690 0.005488 0.005488 0.005488 1.28%
Castro::enforce_min_density() 30 0.005302 0.005302 0.005302 1.24%
MLPoisson::Fapply() 500 0.00475 0.00475 0.00475 1.11%
Castro::post_timestep() 5 0.004173 0.004173 0.004173 0.97%
FabArray::setDomainBndry() 20 0.003861 0.003861 0.003861 0.90%
Castro::do_old_sources() 5 0.003712 0.003712 0.003712 0.87%
MLMG::prepareForSolve() 6 0.003583 0.003583 0.003583 0.84%
MLCellLinOp::solutionResidual() 42 0.003443 0.003443 0.003443 0.80%
Castro::estTimeStep() 10 0.003336 0.003336 0.003336 0.78%
MLCellLinOp::defineBC() 6 0.00291 0.00291 0.00291 0.68%
MLMG::computeResidual() 36 0.002849 0.002849 0.002849 0.66%
BndryData::define() 6 0.002803 0.002803 0.002803 0.65%
MultiFab::Xpay() 258 0.002726 0.002726 0.002726 0.64%
Castro::reset_internal_energy() 30 0.002502 0.002502 0.002502 0.58%
Castro::construct_new_source() 25 0.002059 0.002059 0.002059 0.48%
MLMG::getGradSolution() 6 0.001826 0.001826 0.001826 0.43%
MLCellLinOp::compGrad() 6 0.001823 0.001823 0.001823 0.43%
Castro::computeNewDt() 5 0.001433 0.001433 0.001433 0.33%
Castro::normalize_species() 30 0.001328 0.001328 0.001328 0.31%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001233 0.001233 0.001233 0.29%
Castro::construct_new_gravity_source() 5 0.001161 0.001161 0.001161 0.27%
Gravity::swapTimeLevels() 5 0.0009728 0.0009728 0.0009728 0.23%
Castro::apply_source_to_state() 10 0.0009323 0.0009323 0.0009323 0.22%
MultiFab::Saxpy() 10 0.0009266 0.0009266 0.0009266 0.22%
Castro::construct_old_source() 25 0.0008965 0.0008965 0.0008965 0.21%
FabArrayBase::getCPC() 632 0.0008725 0.0008725 0.0008725 0.20%
MLMG::ResNormInf() 42 0.0008503 0.0008503 0.0008503 0.20%
MLCellLinOp::setLevelBC() 6 0.0008444 0.0008444 0.0008444 0.20%
Castro::construct_old_gravity_source() 5 0.0008203 0.0008203 0.0008203 0.19%
Gravity::update_max_rhs() 6 0.000785 0.000785 0.000785 0.18%
MLPoisson::prepareForSolve() 6 0.0007105 0.0007105 0.0007105 0.17%
MLCellLinOp::prepareForSolve() 6 0.0007059 0.0007059 0.0007059 0.16%
FabArray::mult() 22 0.0006602 0.0006602 0.0006602 0.15%
MLMG::computeMLResidual() 6 0.0006119 0.0006119 0.0006119 0.14%
FabArrayBase::CPC::define() 244 0.0004409 0.0004409 0.0004409 0.10%
Castro::enforce_speed_limit() 30 0.0004355 0.0004355 0.0004355 0.10%
Amr::InitAmr() 1 0.0004181 0.0004181 0.0004181 0.10%
FabArrayBase::getFB() 1766 0.0003691 0.0003691 0.0003691 0.09%
MultiFab::Copy() 6 0.0002974 0.0002974 0.0002974 0.07%
MultiFab::min() 5 0.0001955 0.0001955 0.0001955 0.05%
MLLinOp::define() 6 0.0001567 0.0001567 0.0001567 0.04%
MLMG::MLResNormInf() 6 0.000136 0.000136 0.000136 0.03%
MultiFab::max() 6 0.0001342 0.0001342 0.0001342 0.03%
MLLinOp::defineGrids() 6 0.0001273 0.0001273 0.0001273 0.03%
MLMG::MLRhsNormInf() 6 0.0001073 0.0001073 0.0001073 0.03%
FabArrayBase::FB::FB() 26 6.338e-05 6.338e-05 6.338e-05 0.01%
Castro::create_source_corrector() 5 5.272e-05 5.272e-05 5.272e-05 0.01%
MLLinOp::makeAgglomeratedDMap 6 2.945e-05 2.945e-05 2.945e-05 0.01%
Castro::finalize_advance() 5 2.864e-05 2.864e-05 2.864e-05 0.01%
Castro::swap_state_time_levels() 5 2.665e-05 2.665e-05 2.665e-05 0.01%
Amr::writeSmallPlotFile() 1 2.378e-05 2.378e-05 2.378e-05 0.01%
makeSFC 30 2.062e-05 2.062e-05 2.062e-05 0.00%
Castro::finalize_do_advance() 5 1.966e-05 1.966e-05 1.966e-05 0.00%
Amr::initSubcycle() 1 1.038e-05 1.038e-05 1.038e-05 0.00%
DistributionMapping::Distribute() 31 8.021e-06 8.021e-06 8.021e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 6.086e-06 6.086e-06 6.086e-06 0.00%
MLMG::buildFineMask() 6 3.556e-06 3.556e-06 3.556e-06 0.00%
Gravity::set_mass_offset() 6 2.144e-06 2.144e-06 2.144e-06 0.00%
Castro::retry_advance_ctu() 5 1.573e-06 1.573e-06 1.573e-06 0.00%
Castro::FluxRegFineAdd() 5 1.429e-06 1.429e-06 1.429e-06 0.00%
AmrLevel::AmrLevel() 1 1.074e-06 1.074e-06 1.074e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.062e-06 1.062e-06 1.062e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.02-5-g3fb8caaf78a0) finalized