Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.02-7-g9558c8a1684c) initialized
Starting run at 08:52:06 UTC on 2022-02-09.
Successfully read inputs file ...
Castro git describe: 22.02-4-g0d942c92d
AMReX git describe: 22.02-7-g9558c8a16
Microphysics git describe: 22.02
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.0377873 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.022488298 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.108639637
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.049578707
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.055228373
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.057835203
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.071238933
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.034924845 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.050636741
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.046444601
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.048443634
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.057264615
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.062301391
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.034812297 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.021961265 seconds
Ending run at 08:52:07 UTC on 2022-02-09.
Run time = 0.82154895
Run time without initialization = 0.699820709
Average number of zones advanced per microsecond: 3.746
Average number of zones advanced per microsecond per rank: 3.746
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.8216 ... 0.8216 ... 0.8216
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.1956 0.1956 0.1956 23.81%
VisMF::Write(FabArray) 11 0.1451 0.1451 0.1451 17.66%
MLCellLinOp::applyBC() 4433 0.09278 0.09278 0.09278 11.29%
MLPoisson::Fsmooth() 3280 0.05973 0.05973 0.05973 7.27%
StateDataPhysBCFunct::() 41 0.03468 0.03468 0.03468 4.22%
FabArray::setVal() 1144 0.02615 0.02615 0.02615 3.18%
StateData::FillBoundary(geom) 328 0.0226 0.0226 0.0226 2.75%
MLCGSolver::bicgstab 82 0.02149 0.02149 0.02149 2.62%
MultiFab::Dot() 1114 0.02138 0.02138 0.02138 2.60%
FillBoundary_nowait() 4023 0.01706 0.01706 0.01706 2.08%
FabArray::ParallelCopy_nowait() 861 0.01606 0.01606 0.01606 1.95%
Castro::computeTemp() 63 0.0131 0.0131 0.0131 1.59%
MultiFab::LinComb() 1586 0.0123 0.0123 0.0123 1.50%
Gravity::fill_multipole_BCs() 11 0.01173 0.01173 0.01173 1.43%
MLPoisson::Fapply() 1142 0.01056 0.01056 0.01056 1.29%
Castro::enforce_min_density() 62 0.01014 0.01014 0.01014 1.23%
MLCellLinOp::defineAuxData() 11 0.009967 0.009967 0.009967 1.21%
MLMG::addInterpCorrection() 410 0.007039 0.007039 0.007039 0.86%
amrex::average_down 410 0.00648 0.00648 0.00648 0.79%
MultiFab::Xpay() 585 0.006027 0.006027 0.006027 0.73%
Castro::reset_internal_energy() 63 0.005776 0.005776 0.005776 0.70%
FabArray::setDomainBndry() 41 0.005467 0.005467 0.005467 0.67%
Castro::expand_state() 10 0.004834 0.004834 0.004834 0.59%
Castro::estTimeStep() 21 0.004531 0.004531 0.004531 0.55%
Amr::checkPoint() 3 0.003898 0.003898 0.003898 0.47%
BndryData::define() 11 0.003544 0.003544 0.003544 0.43%
Castro::enforce_speed_limit() 62 0.003463 0.003463 0.003463 0.42%
Castro::do_advance_ctu() 10 0.003216 0.003216 0.003216 0.39%
Castro::construct_new_gravity_source() 10 0.00308 0.00308 0.00308 0.37%
Amr::writePlotFile() 2 0.003049 0.003049 0.003049 0.37%
Gravity::get_new_grav_vector() 11 0.002703 0.002703 0.002703 0.33%
Castro::normalize_species() 62 0.002597 0.002597 0.002597 0.32%
Castro::construct_old_gravity_source() 10 0.002571 0.002571 0.002571 0.31%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002276 0.002276 0.002276 0.28%
MLCellLinOp::compGrad() 11 0.001971 0.001971 0.001971 0.24%
MLMG::ResNormInf() 93 0.001902 0.001902 0.001902 0.23%
MultiFab::Saxpy() 20 0.001808 0.001808 0.001808 0.22%
Gravity::get_old_grav_vector() 10 0.001752 0.001752 0.001752 0.21%
MLMG::oneIter() 82 0.001667 0.001667 0.001667 0.20%
MLCellLinOp::setLevelBC() 11 0.001534 0.001534 0.001534 0.19%
Gravity::actual_solve_with_mlmg() 11 0.001395 0.001395 0.001395 0.17%
MLCellLinOp::prepareForSolve() 11 0.001331 0.001331 0.001331 0.16%
FabArray::mult() 43 0.001321 0.001321 0.001321 0.16%
MLCellLinOp::smooth() 1640 0.001224 0.001224 0.001224 0.15%
Castro::initData() 1 0.001215 0.001215 0.001215 0.15%
FabArrayBase::getCPC() 1323 0.0008955 0.0008955 0.0008955 0.11%
FabArray::FillBoundary() 4023 0.00087 0.00087 0.00087 0.11%
MLMG::prepareForSolve() 11 0.0007759 0.0007759 0.0007759 0.09%
FabArrayBase::getFB() 4023 0.0007405 0.0007405 0.0007405 0.09%
FabArrayBase::CPC::define() 454 0.0007322 0.0007322 0.0007322 0.09%
Gravity::update_max_rhs() 11 0.0005487 0.0005487 0.0005487 0.07%
MLCellLinOp::apply() 1142 0.0005083 0.0005083 0.0005083 0.06%
CGSolver::sxay() 1586 0.0004889 0.0004889 0.0004889 0.06%
MultiFab::Copy() 11 0.0004771 0.0004771 0.0004771 0.06%
Gravity::solve_for_phi() 10 0.0004338 0.0004338 0.0004338 0.05%
Amr::InitAmr() 1 0.0004282 0.0004282 0.0004282 0.05%
MLMG::mgVcycle() 82 0.0003885 0.0003885 0.0003885 0.05%
MLCGSolver::ParallelAllReduce 1514 0.0003332 0.0003332 0.0003332 0.04%
MultiFab::min() 10 0.0003186 0.0003186 0.0003186 0.04%
FabArray::ParallelCopy() 861 0.0003146 0.0003146 0.0003146 0.04%
main() 1 0.0002773 0.0002773 0.0002773 0.03%
MultiFab::max() 11 0.0002587 0.0002587 0.0002587 0.03%
Castro::construct_new_gravity() 10 0.0002502 0.0002502 0.0002502 0.03%
FillPatchIterator::Initialize 41 0.0002448 0.0002448 0.0002448 0.03%
MLCellLinOp::correctionResidual() 492 0.0002325 0.0002325 0.0002325 0.03%
Gravity::actual_multilevel_solve() 1 0.0002235 0.0002235 0.0002235 0.03%
Amr::coarseTimeStep() 10 0.0002059 0.0002059 0.0002059 0.03%
MLCellLinOp::defineBC() 11 0.000201 0.000201 0.000201 0.02%
MLMG::MLRhsNormInf() 11 0.0002002 0.0002002 0.0002002 0.02%
Amr::timeStep() 10 0.0001977 0.0001977 0.0001977 0.02%
MLLinOp::defineGrids() 11 0.0001752 0.0001752 0.0001752 0.02%
Amr::defBaseLevel() 1 0.0001526 0.0001526 0.0001526 0.02%
MLMG:computeResOfCorrection() 410 0.0001505 0.0001505 0.0001505 0.02%
Castro::subcycle_advance_ctu() 10 0.0001431 0.0001431 0.0001431 0.02%
StateData::checkPoint() 12 0.0001366 0.0001366 0.0001366 0.02%
MLMG::actualBottomSolve() 82 0.0001099 0.0001099 0.0001099 0.01%
MLMG::mgVcycle_down::0 82 9.976e-05 9.976e-05 9.976e-05 0.01%
FabArrayBase::FB::FB() 56 9.303e-05 9.303e-05 9.303e-05 0.01%
Castro::post_timestep() 10 9.127e-05 9.127e-05 9.127e-05 0.01%
MLMG::mgVcycle_down::1 82 8.573e-05 8.573e-05 8.573e-05 0.01%
Castro::initialize_advance() 10 8.184e-05 8.184e-05 8.184e-05 0.01%
MLMG::mgVcycle_down::2 82 8.127e-05 8.127e-05 8.127e-05 0.01%
MLMG::mgVcycle_down::4 82 8.051e-05 8.051e-05 8.051e-05 0.01%
MLMG::mgVcycle_down::3 82 7.822e-05 7.822e-05 7.822e-05 0.01%
Castro::advance() 10 7.71e-05 7.71e-05 7.71e-05 0.01%
AmrLevel::checkPoint() 3 7.39e-05 7.39e-05 7.39e-05 0.01%
MLMG::solve() 11 7.339e-05 7.339e-05 7.339e-05 0.01%
Castro::clean_state() 62 7.015e-05 7.015e-05 7.015e-05 0.01%
MLMG::mgVcycle_up::4 82 6.052e-05 6.052e-05 6.052e-05 0.01%
Castro::initialize_do_advance() 10 5.685e-05 5.685e-05 5.685e-05 0.01%
Castro::finalize_advance() 10 5.584e-05 5.584e-05 5.584e-05 0.01%
MLMG::mgVcycle_up::0 82 5.252e-05 5.252e-05 5.252e-05 0.01%
MLCellLinOp::solutionResidual() 93 5.083e-05 5.083e-05 5.083e-05 0.01%
MLMG::mgVcycle_up::3 82 4.993e-05 4.993e-05 4.993e-05 0.01%
MLMG::mgVcycle_up::1 82 4.987e-05 4.987e-05 4.987e-05 0.01%
MLMG::mgVcycle_up::2 82 4.891e-05 4.891e-05 4.891e-05 0.01%
Castro::finalize_do_advance() 10 4.824e-05 4.824e-05 4.824e-05 0.01%
StateData::define() 4 3.998e-05 3.998e-05 3.998e-05 0.00%
MLMG::computeResidual() 82 3.878e-05 3.878e-05 3.878e-05 0.00%
Castro::swap_state_time_levels() 10 3.703e-05 3.703e-05 3.703e-05 0.00%
MLMG::mgVcycle_bottom 82 3.357e-05 3.357e-05 3.357e-05 0.00%
Castro::enforce_consistent_e() 1 3.287e-05 3.287e-05 3.287e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.886e-05 2.886e-05 2.886e-05 0.00%
MLLinOp::define() 11 2.872e-05 2.872e-05 2.872e-05 0.00%
FillPatchSingleLevel 41 2.813e-05 2.813e-05 2.813e-05 0.00%
makeSFC 55 2.806e-05 2.806e-05 2.806e-05 0.00%
Amr::writeSmallPlotFile() 1 2.248e-05 2.248e-05 2.248e-05 0.00%
MLPoisson::define() 11 2.242e-05 2.242e-05 2.242e-05 0.00%
Castro::construct_old_source() 50 2.046e-05 2.046e-05 2.046e-05 0.00%
Amr::FinalizeInit() 1 1.998e-05 1.998e-05 1.998e-05 0.00%
Castro::construct_new_source() 50 1.8e-05 1.8e-05 1.8e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.76e-05 1.76e-05 1.76e-05 0.00%
Castro::do_new_sources() 10 1.753e-05 1.753e-05 1.753e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.649e-05 1.649e-05 1.649e-05 0.00%
Castro::do_old_sources() 10 1.528e-05 1.528e-05 1.528e-05 0.00%
DistributionMapping::Distribute() 56 1.382e-05 1.382e-05 1.382e-05 0.00%
Castro::apply_source_to_state() 20 1.343e-05 1.343e-05 1.343e-05 0.00%
Castro::construct_old_gravity() 10 1.186e-05 1.186e-05 1.186e-05 0.00%
Gravity::swapTimeLevels() 10 1.048e-05 1.048e-05 1.048e-05 0.00%
Amr::initSubcycle() 1 9.108e-06 9.108e-06 9.108e-06 0.00%
AmrLevel::AmrLevel(dm) 1 8.851e-06 8.851e-06 8.851e-06 0.00%
MLPoisson::prepareForSolve() 11 8.213e-06 8.213e-06 8.213e-06 0.00%
MLMG::computeMLResidual() 11 6.829e-06 6.829e-06 6.829e-06 0.00%
MLMG::buildFineMask() 11 6.665e-06 6.665e-06 6.665e-06 0.00%
MLMG::getGradSolution() 11 5.824e-06 5.824e-06 5.824e-06 0.00%
AmrLevel::checkPointPost() 3 5.689e-06 5.689e-06 5.689e-06 0.00%
Amr::InitializeInit() 1 5.506e-06 5.506e-06 5.506e-06 0.00%
Castro::computeNewDt() 9 5.346e-06 5.346e-06 5.346e-06 0.00%
Castro::create_source_corrector() 10 4.882e-06 4.882e-06 4.882e-06 0.00%
Gravity::set_mass_offset() 11 4.798e-06 4.798e-06 4.798e-06 0.00%
MLMG::MLResNormInf() 11 4.734e-06 4.734e-06 4.734e-06 0.00%
Castro::retry_advance_ctu() 10 3.638e-06 3.638e-06 3.638e-06 0.00%
Castro::post_init() 1 3.507e-06 3.507e-06 3.507e-06 0.00%
Castro::FluxRegFineAdd() 10 3.278e-06 3.278e-06 3.278e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.688e-06 2.688e-06 2.688e-06 0.00%
Castro::computeInitialDt() 2 2.332e-06 2.332e-06 2.332e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.286e-06 2.286e-06 2.286e-06 0.00%
Amr::init() 1 2.13e-06 2.13e-06 2.13e-06 0.00%
AmrLevel::checkPointPre() 3 1.958e-06 1.958e-06 1.958e-06 0.00%
Castro::post_regrid() 1 1.226e-06 1.226e-06 1.226e-06 0.00%
Amr::initialInit() 1 1.012e-06 1.012e-06 1.012e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.8216 0.8216 0.8216 100.00%
Amr::coarseTimeStep() 10 0.6777 0.6777 0.6777 82.48%
Amr::timeStep() 10 0.6053 0.6053 0.6053 73.67%
Castro::advance() 10 0.5985 0.5985 0.5985 72.85%
Castro::subcycle_advance_ctu() 10 0.5849 0.5849 0.5849 71.20%
Castro::do_advance_ctu() 10 0.5848 0.5848 0.5848 71.18%
Gravity::solve_phi_with_mlmg() 11 0.3233 0.3233 0.3233 39.35%
Gravity::actual_solve_with_mlmg() 11 0.3113 0.3113 0.3113 37.89%
Castro::construct_new_gravity() 10 0.3049 0.3049 0.3049 37.12%
MLMG::solve() 11 0.2889 0.2889 0.2889 35.16%
Gravity::solve_for_phi() 10 0.2757 0.2757 0.2757 33.55%
MLMG::oneIter() 82 0.2733 0.2733 0.2733 33.26%
MLMG::mgVcycle() 82 0.2716 0.2716 0.2716 33.06%
Castro::construct_ctu_hydro_source() 10 0.1956 0.1956 0.1956 23.81%
VisMF::Write(FabArray) 11 0.1451 0.1451 0.1451 17.66%
MLCellLinOp::smooth() 1640 0.1437 0.1437 0.1437 17.49%
Amr::init() 1 0.1212 0.1212 0.1212 14.75%
MLCellLinOp::applyBC() 4433 0.1115 0.1115 0.1115 13.58%
Amr::checkPoint() 3 0.1077 0.1077 0.1077 13.11%
AmrLevel::checkPoint() 3 0.1038 0.1038 0.1038 12.63%
StateData::checkPoint() 12 0.1037 0.1037 0.1037 12.62%
MLMG::mgVcycle_bottom 82 0.0774 0.0774 0.0774 9.42%
MLMG::actualBottomSolve() 82 0.07736 0.07736 0.07736 9.42%
MLCGSolver::bicgstab 82 0.07663 0.07663 0.07663 9.33%
FillPatchIterator::Initialize 41 0.06725 0.06725 0.06725 8.19%
FillPatchSingleLevel 41 0.06154 0.06154 0.06154 7.49%
Amr::initialInit() 1 0.06078 0.06078 0.06078 7.40%
MLPoisson::Fsmooth() 3280 0.05973 0.05973 0.05973 7.27%
StateDataPhysBCFunct::() 41 0.05728 0.05728 0.05728 6.97%
Amr::FinalizeInit() 1 0.05147 0.05147 0.05147 6.26%
Castro::post_init() 1 0.05076 0.05076 0.05076 6.18%
Gravity::multilevel_solve_for_new_phi() 1 0.04829 0.04829 0.04829 5.88%
Gravity::actual_multilevel_solve() 1 0.04828 0.04828 0.04828 5.88%
Amr::writePlotFile() 2 0.04456 0.04456 0.04456 5.42%
MLCellLinOp::apply() 1142 0.03929 0.03929 0.03929 4.78%
MLMG::mgVcycle_down::0 82 0.03774 0.03774 0.03774 4.59%
Castro::clean_state() 62 0.03438 0.03438 0.03438 4.18%
MLMG::mgVcycle_up::0 82 0.03236 0.03236 0.03236 3.94%
Gravity::get_new_grav_vector() 11 0.03133 0.03133 0.03133 3.81%
FabArray::setVal() 1144 0.02615 0.02615 0.02615 3.18%
Castro::initialize_do_advance() 10 0.0238 0.0238 0.0238 2.90%
StateData::FillBoundary(geom) 328 0.0226 0.0226 0.0226 2.75%
Castro::construct_old_gravity() 10 0.02252 0.02252 0.02252 2.74%
Gravity::get_old_grav_vector() 10 0.02251 0.02251 0.02251 2.74%
MLCellLinOp::correctionResidual() 492 0.02211 0.02211 0.02211 2.69%
MultiFab::Dot() 1114 0.02138 0.02138 0.02138 2.60%
MLMG:computeResOfCorrection() 410 0.01913 0.01913 0.01913 2.33%
Castro::computeTemp() 63 0.01888 0.01888 0.01888 2.30%
FabArray::FillBoundary() 4023 0.01876 0.01876 0.01876 2.28%
Castro::expand_state() 10 0.01858 0.01858 0.01858 2.26%
MLMG::mgVcycle_down::1 82 0.01846 0.01846 0.01846 2.25%
FillBoundary_nowait() 4023 0.01789 0.01789 0.01789 2.18%
MLMG::mgVcycle_down::2 82 0.01789 0.01789 0.01789 2.18%
FabArray::ParallelCopy() 861 0.01729 0.01729 0.01729 2.11%
MLMG::mgVcycle_down::3 82 0.01711 0.01711 0.01711 2.08%
MLPoisson::define() 11 0.01698 0.01698 0.01698 2.07%
FabArray::ParallelCopy_nowait() 861 0.01698 0.01698 0.01698 2.07%
MLMG::mgVcycle_down::4 82 0.0163 0.0163 0.0163 1.98%
MLMG::mgVcycle_up::2 82 0.01392 0.01392 0.01392 1.69%
MLMG::mgVcycle_up::1 82 0.01372 0.01372 0.01372 1.67%
MLMG::addInterpCorrection() 410 0.01353 0.01353 0.01353 1.65%
Castro::initialize_advance() 10 0.01343 0.01343 0.01343 1.63%
MLMG::mgVcycle_up::3 82 0.01324 0.01324 0.01324 1.61%
MLMG::mgVcycle_up::4 82 0.01306 0.01306 0.01306 1.59%
amrex::average_down 410 0.01306 0.01306 0.01306 1.59%
MLCGSolver::ParallelAllReduce 1514 0.01288 0.01288 0.01288 1.57%
CGSolver::sxay() 1586 0.01279 0.01279 0.01279 1.56%
MultiFab::LinComb() 1586 0.0123 0.0123 0.0123 1.50%
Castro::do_new_sources() 10 0.01182 0.01182 0.01182 1.44%
Gravity::fill_multipole_BCs() 11 0.01173 0.01173 0.01173 1.43%
MLCellLinOp::defineAuxData() 11 0.01167 0.01167 0.01167 1.42%
MLPoisson::Fapply() 1142 0.01056 0.01056 0.01056 1.29%
Castro::enforce_min_density() 62 0.01014 0.01014 0.01014 1.23%
Amr::InitializeInit() 1 0.009308 0.009308 0.009308 1.13%
Amr::defBaseLevel() 1 0.009302 0.009302 0.009302 1.13%
Castro::do_old_sources() 10 0.007722 0.007722 0.007722 0.94%
MLCellLinOp::solutionResidual() 93 0.007678 0.007678 0.007678 0.93%
MLMG::computeResidual() 82 0.006644 0.006644 0.006644 0.81%
Castro::post_timestep() 10 0.006604 0.006604 0.006604 0.80%
MultiFab::Xpay() 585 0.006027 0.006027 0.006027 0.73%
Castro::reset_internal_energy() 63 0.005776 0.005776 0.005776 0.70%
MLMG::prepareForSolve() 11 0.005731 0.005731 0.005731 0.70%
FabArray::setDomainBndry() 41 0.005467 0.005467 0.005467 0.67%
MLCellLinOp::defineBC() 11 0.005025 0.005025 0.005025 0.61%
BndryData::define() 11 0.004824 0.004824 0.004824 0.59%
Castro::estTimeStep() 21 0.004531 0.004531 0.004531 0.55%
Castro::initData() 1 0.003477 0.003477 0.003477 0.42%
Castro::enforce_speed_limit() 62 0.003463 0.003463 0.003463 0.42%
Castro::construct_new_source() 50 0.003098 0.003098 0.003098 0.38%
Castro::construct_new_gravity_source() 10 0.00308 0.00308 0.00308 0.37%
Castro::normalize_species() 62 0.002597 0.002597 0.002597 0.32%
Castro::construct_old_source() 50 0.002591 0.002591 0.002591 0.32%
Castro::construct_old_gravity_source() 10 0.002571 0.002571 0.002571 0.31%
MLMG::getGradSolution() 11 0.002514 0.002514 0.002514 0.31%
MLCellLinOp::compGrad() 11 0.002508 0.002508 0.002508 0.31%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002276 0.002276 0.002276 0.28%
MLMG::ResNormInf() 93 0.001902 0.001902 0.001902 0.23%
Castro::computeNewDt() 9 0.001836 0.001836 0.001836 0.22%
Castro::apply_source_to_state() 20 0.001821 0.001821 0.001821 0.22%
MultiFab::Saxpy() 20 0.001808 0.001808 0.001808 0.22%
FabArrayBase::getCPC() 1323 0.001628 0.001628 0.001628 0.20%
MLCellLinOp::setLevelBC() 11 0.001534 0.001534 0.001534 0.19%
MLPoisson::prepareForSolve() 11 0.001339 0.001339 0.001339 0.16%
MLCellLinOp::prepareForSolve() 11 0.001331 0.001331 0.001331 0.16%
FabArray::mult() 43 0.001321 0.001321 0.001321 0.16%
MLMG::computeMLResidual() 11 0.00108 0.00108 0.00108 0.13%
Gravity::swapTimeLevels() 10 0.001033 0.001033 0.001033 0.13%
Gravity::update_max_rhs() 11 0.0009515 0.0009515 0.0009515 0.12%
FabArrayBase::getFB() 4023 0.0008335 0.0008335 0.0008335 0.10%
FabArrayBase::CPC::define() 454 0.0007322 0.0007322 0.0007322 0.09%
Castro::computeInitialDt() 2 0.0006828 0.0006828 0.0006828 0.08%
MultiFab::Copy() 11 0.0004771 0.0004771 0.0004771 0.06%
Castro::post_regrid() 1 0.0004676 0.0004676 0.0004676 0.06%
Amr::InitAmr() 1 0.0004373 0.0004373 0.0004373 0.05%
MultiFab::min() 10 0.0003186 0.0003186 0.0003186 0.04%
MLLinOp::define() 11 0.0002648 0.0002648 0.0002648 0.03%
MultiFab::max() 11 0.0002587 0.0002587 0.0002587 0.03%
MLMG::MLResNormInf() 11 0.0002531 0.0002531 0.0002531 0.03%
MLLinOp::defineGrids() 11 0.0002361 0.0002361 0.0002361 0.03%
MLMG::MLRhsNormInf() 11 0.0002002 0.0002002 0.0002002 0.02%
FabArrayBase::FB::FB() 56 9.303e-05 9.303e-05 9.303e-05 0.01%
Castro::finalize_advance() 10 5.912e-05 5.912e-05 5.912e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.857e-05 5.857e-05 5.857e-05 0.01%
AmrLevel::AmrLevel(dm) 1 4.883e-05 4.883e-05 4.883e-05 0.01%
Castro::finalize_do_advance() 10 4.824e-05 4.824e-05 4.824e-05 0.01%
makeSFC 55 4.096e-05 4.096e-05 4.096e-05 0.00%
StateData::define() 4 3.998e-05 3.998e-05 3.998e-05 0.00%
Castro::swap_state_time_levels() 10 3.703e-05 3.703e-05 3.703e-05 0.00%
Castro::enforce_consistent_e() 1 3.287e-05 3.287e-05 3.287e-05 0.00%
Amr::writeSmallPlotFile() 1 2.248e-05 2.248e-05 2.248e-05 0.00%
DistributionMapping::Distribute() 56 1.382e-05 1.382e-05 1.382e-05 0.00%
Amr::initSubcycle() 1 9.108e-06 9.108e-06 9.108e-06 0.00%
MLMG::buildFineMask() 11 6.665e-06 6.665e-06 6.665e-06 0.00%
AmrLevel::checkPointPost() 3 5.689e-06 5.689e-06 5.689e-06 0.00%
Castro::create_source_corrector() 10 4.882e-06 4.882e-06 4.882e-06 0.00%
Gravity::set_mass_offset() 11 4.798e-06 4.798e-06 4.798e-06 0.00%
Castro::retry_advance_ctu() 10 3.638e-06 3.638e-06 3.638e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.606e-06 3.606e-06 3.606e-06 0.00%
Castro::FluxRegFineAdd() 10 3.278e-06 3.278e-06 3.278e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.286e-06 2.286e-06 2.286e-06 0.00%
AmrLevel::checkPointPre() 3 1.958e-06 1.958e-06 1.958e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.02-7-g9558c8a1684c) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.02-7-g9558c8a1684c) initialized
Starting run at 08:52:07 UTC on 2022-02-09.
Successfully read inputs file ...
Castro git describe: 22.02-4-g0d942c92d
AMReX git describe: 22.02-7-g9558c8a16
Microphysics git describe: 22.02
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.499116084
Restart time = 0.054512448 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.115346865
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.047356222
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.047986628
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.055872268
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.059977321
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.0230935 seconds
Ending run at 08:52:07 UTC on 2022-02-09.
Run time = 0.405061567
Run time without initialization = 0.349989729
Average number of zones advanced per microsecond: 3.745
Average number of zones advanced per microsecond per rank: 3.745
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.4051 ... 0.4051 ... 0.4051
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.1245 0.1245 0.1245 30.75%
VisMF::Read() 3 0.04286 0.04286 0.04286 10.58%
MLCellLinOp::applyBC() 1946 0.03771 0.03771 0.03771 9.31%
MLPoisson::Fsmooth() 1440 0.02555 0.02555 0.02555 6.31%
VisMF::Write(FabArray) 1 0.02183 0.02183 0.02183 5.39%
FabArray::setVal() 537 0.01622 0.01622 0.01622 4.01%
StateData::FillBoundary(geom) 160 0.01095 0.01095 0.01095 2.70%
MLCGSolver::bicgstab 36 0.009317 0.009317 0.009317 2.30%
MultiFab::Dot() 484 0.009178 0.009178 0.009178 2.27%
StateDataPhysBCFunct::() 20 0.00718 0.00718 0.00718 1.77%
FillBoundary_nowait() 1766 0.007136 0.007136 0.007136 1.76%
FabArray::ParallelCopy_nowait() 380 0.007079 0.007079 0.007079 1.75%
Castro::computeTemp() 30 0.006802 0.006802 0.006802 1.68%
Gravity::fill_multipole_BCs() 6 0.006136 0.006136 0.006136 1.51%
MLCellLinOp::defineAuxData() 6 0.005449 0.005449 0.005449 1.35%
MultiFab::LinComb() 690 0.005252 0.005252 0.005252 1.30%
MLPoisson::Fapply() 500 0.004597 0.004597 0.004597 1.13%
FabArray::setDomainBndry() 20 0.004031 0.004031 0.004031 0.99%
Castro::expand_state() 5 0.00402 0.00402 0.00402 0.99%
Castro::enforce_min_density() 30 0.003128 0.003128 0.003128 0.77%
MLMG::addInterpCorrection() 180 0.003025 0.003025 0.003025 0.75%
Amr::restart() 1 0.002938 0.002938 0.002938 0.73%
amrex::average_down 180 0.002806 0.002806 0.002806 0.69%
MultiFab::Xpay() 258 0.00264 0.00264 0.00264 0.65%
Castro::do_advance_ctu() 5 0.002606 0.002606 0.002606 0.64%
Castro::estTimeStep() 10 0.002443 0.002443 0.002443 0.60%
Castro::reset_internal_energy() 30 0.002303 0.002303 0.002303 0.57%
BndryData::define() 6 0.001927 0.001927 0.001927 0.48%
Gravity::get_new_grav_vector() 5 0.00186 0.00186 0.00186 0.46%
Castro::construct_new_gravity_source() 5 0.001606 0.001606 0.001606 0.40%
MLCellLinOp::compGrad() 6 0.001466 0.001466 0.001466 0.36%
Amr::writePlotFile() 1 0.00136 0.00136 0.00136 0.34%
Castro::construct_old_gravity_source() 5 0.001353 0.001353 0.001353 0.33%
Castro::normalize_species() 30 0.001298 0.001298 0.001298 0.32%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001195 0.001195 0.001195 0.29%
Gravity::get_old_grav_vector() 5 0.00103 0.00103 0.00103 0.25%
MultiFab::Saxpy() 10 0.0009146 0.0009146 0.0009146 0.23%
MLMG::ResNormInf() 42 0.0008533 0.0008533 0.0008533 0.21%
MLCellLinOp::setLevelBC() 6 0.0008158 0.0008158 0.0008158 0.20%
Gravity::actual_solve_with_mlmg() 6 0.0007758 0.0007758 0.0007758 0.19%
MLMG::oneIter() 36 0.0007265 0.0007265 0.0007265 0.18%
MLCellLinOp::prepareForSolve() 6 0.0006858 0.0006858 0.0006858 0.17%
FabArray::mult() 22 0.0006527 0.0006527 0.0006527 0.16%
MLCellLinOp::smooth() 720 0.0005517 0.0005517 0.0005517 0.14%
Gravity::update_max_rhs() 6 0.0005195 0.0005195 0.0005195 0.13%
Castro::enforce_speed_limit() 30 0.0005179 0.0005179 0.0005179 0.13%
MLMG::prepareForSolve() 6 0.0004352 0.0004352 0.0004352 0.11%
FabArrayBase::getCPC() 632 0.0004301 0.0004301 0.0004301 0.11%
FabArrayBase::CPC::define() 244 0.0004292 0.0004292 0.0004292 0.11%
FabArray::FillBoundary() 1766 0.0004094 0.0004094 0.0004094 0.10%
Amr::InitAmr() 1 0.0003759 0.0003759 0.0003759 0.09%
MultiFab::Copy() 6 0.0003689 0.0003689 0.0003689 0.09%
FabArrayBase::getFB() 1766 0.0003401 0.0003401 0.0003401 0.08%
main() 1 0.0002691 0.0002691 0.0002691 0.07%
Castro::construct_new_gravity() 5 0.0002459 0.0002459 0.0002459 0.06%
Gravity::solve_for_phi() 5 0.0002344 0.0002344 0.0002344 0.06%
MLCellLinOp::apply() 500 0.0002189 0.0002189 0.0002189 0.05%
Gravity::actual_multilevel_solve() 1 0.000218 0.000218 0.000218 0.05%
CGSolver::sxay() 690 0.0002052 0.0002052 0.0002052 0.05%
MultiFab::min() 5 0.0001592 0.0001592 0.0001592 0.04%
MLMG::mgVcycle() 36 0.0001524 0.0001524 0.0001524 0.04%
MLCGSolver::ParallelAllReduce 659 0.0001468 0.0001468 0.0001468 0.04%
FabArray::ParallelCopy() 380 0.0001457 0.0001457 0.0001457 0.04%
MultiFab::max() 6 0.0001331 0.0001331 0.0001331 0.03%
FillPatchIterator::Initialize 20 0.0001238 0.0001238 0.0001238 0.03%
Castro::advance() 5 0.0001197 0.0001197 0.0001197 0.03%
Amr::coarseTimeStep() 5 0.0001093 0.0001093 0.0001093 0.03%
MLMG::MLRhsNormInf() 6 0.0001055 0.0001055 0.0001055 0.03%
MLCellLinOp::defineBC() 6 0.0001052 0.0001052 0.0001052 0.03%
MLCellLinOp::correctionResidual() 216 0.0001029 0.0001029 0.0001029 0.03%
Amr::timeStep() 5 9.855e-05 9.855e-05 9.855e-05 0.02%
MLLinOp::defineGrids() 6 9.298e-05 9.298e-05 9.298e-05 0.02%
AmrLevel::restart() 1 7.569e-05 7.569e-05 7.569e-05 0.02%
StateData::restartDoit() 4 7.482e-05 7.482e-05 7.482e-05 0.02%
Castro::create_source_corrector() 5 7.391e-05 7.391e-05 7.391e-05 0.02%
Castro::post_timestep() 5 7.346e-05 7.346e-05 7.346e-05 0.02%
MLMG:computeResOfCorrection() 180 6.651e-05 6.651e-05 6.651e-05 0.02%
Castro::subcycle_advance_ctu() 5 6.449e-05 6.449e-05 6.449e-05 0.02%
FabArrayBase::FB::FB() 26 6.195e-05 6.195e-05 6.195e-05 0.02%
MLMG::actualBottomSolve() 36 5.118e-05 5.118e-05 5.118e-05 0.01%
Castro::initialize_do_advance() 5 4.991e-05 4.991e-05 4.991e-05 0.01%
MLMG::mgVcycle_down::0 36 4.077e-05 4.077e-05 4.077e-05 0.01%
MLMG::mgVcycle_down::1 36 3.862e-05 3.862e-05 3.862e-05 0.01%
Castro::initialize_advance() 5 3.827e-05 3.827e-05 3.827e-05 0.01%
Castro::construct_old_gravity() 5 3.81e-05 3.81e-05 3.81e-05 0.01%
MLMG::mgVcycle_down::2 36 3.7e-05 3.7e-05 3.7e-05 0.01%
MLMG::mgVcycle_down::4 36 3.566e-05 3.566e-05 3.566e-05 0.01%
MLMG::mgVcycle_down::3 36 3.511e-05 3.511e-05 3.511e-05 0.01%
MLMG::solve() 6 3.37e-05 3.37e-05 3.37e-05 0.01%
Castro::clean_state() 30 3.249e-05 3.249e-05 3.249e-05 0.01%
Castro::construct_new_source() 25 3.086e-05 3.086e-05 3.086e-05 0.01%
MLLinOp::define() 6 2.767e-05 2.767e-05 2.767e-05 0.01%
Castro::post_restart() 1 2.726e-05 2.726e-05 2.726e-05 0.01%
Castro::finalize_advance() 5 2.699e-05 2.699e-05 2.699e-05 0.01%
Castro::swap_state_time_levels() 5 2.66e-05 2.66e-05 2.66e-05 0.01%
MLMG::mgVcycle_up::4 36 2.66e-05 2.66e-05 2.66e-05 0.01%
Amr::writeSmallPlotFile() 1 2.506e-05 2.506e-05 2.506e-05 0.01%
MLCellLinOp::solutionResidual() 42 2.471e-05 2.471e-05 2.471e-05 0.01%
MLMG::mgVcycle_up::0 36 2.245e-05 2.245e-05 2.245e-05 0.01%
MLMG::mgVcycle_up::3 36 2.167e-05 2.167e-05 2.167e-05 0.01%
Castro::finalize_do_advance() 5 2.103e-05 2.103e-05 2.103e-05 0.01%
MLPoisson::define() 6 2.091e-05 2.091e-05 2.091e-05 0.01%
MLMG::mgVcycle_up::2 36 2.054e-05 2.054e-05 2.054e-05 0.01%
MLMG::mgVcycle_up::1 36 1.964e-05 1.964e-05 1.964e-05 0.00%
MLMG::mgVcycle_bottom 36 1.77e-05 1.77e-05 1.77e-05 0.00%
MLMG::computeResidual() 36 1.648e-05 1.648e-05 1.648e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.569e-05 1.569e-05 1.569e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.496e-05 1.496e-05 1.496e-05 0.00%
makeSFC 30 1.419e-05 1.419e-05 1.419e-05 0.00%
FillPatchSingleLevel 20 1.396e-05 1.396e-05 1.396e-05 0.00%
Castro::construct_old_source() 25 1.144e-05 1.144e-05 1.144e-05 0.00%
Castro::do_new_sources() 5 8.985e-06 8.985e-06 8.985e-06 0.00%
DistributionMapping::Distribute() 31 8.905e-06 8.905e-06 8.905e-06 0.00%
Amr::initSubcycle() 1 8.566e-06 8.566e-06 8.566e-06 0.00%
Castro::do_old_sources() 5 8.116e-06 8.116e-06 8.116e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 7.9e-06 7.9e-06 7.9e-06 0.00%
Castro::apply_source_to_state() 10 5.771e-06 5.771e-06 5.771e-06 0.00%
Gravity::swapTimeLevels() 5 4.408e-06 4.408e-06 4.408e-06 0.00%
MLPoisson::prepareForSolve() 6 4.351e-06 4.351e-06 4.351e-06 0.00%
Castro::computeNewDt() 5 3.876e-06 3.876e-06 3.876e-06 0.00%
MLMG::buildFineMask() 6 3.825e-06 3.825e-06 3.825e-06 0.00%
MLMG::computeMLResidual() 6 3.412e-06 3.412e-06 3.412e-06 0.00%
MLMG::getGradSolution() 6 2.92e-06 2.92e-06 2.92e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.89e-06 2.89e-06 2.89e-06 0.00%
MLMG::MLResNormInf() 6 2.726e-06 2.726e-06 2.726e-06 0.00%
Gravity::set_mass_offset() 6 2.498e-06 2.498e-06 2.498e-06 0.00%
Castro::retry_advance_ctu() 5 1.743e-06 1.743e-06 1.743e-06 0.00%
Castro::FluxRegFineAdd() 5 1.689e-06 1.689e-06 1.689e-06 0.00%
Amr::init() 1 1.197e-06 1.197e-06 1.197e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.177e-06 1.177e-06 1.177e-06 0.00%
AmrLevel::AmrLevel() 1 8.51e-07 8.51e-07 8.51e-07 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.4051 0.4051 0.4051 100.00%
Amr::coarseTimeStep() 5 0.3266 0.3266 0.3266 80.63%
Amr::timeStep() 5 0.3251 0.3251 0.3251 80.24%
Castro::advance() 5 0.3221 0.3221 0.3221 79.51%
Castro::subcycle_advance_ctu() 5 0.3123 0.3123 0.3123 77.09%
Castro::do_advance_ctu() 5 0.3122 0.3122 0.3122 77.07%
Castro::construct_new_gravity() 5 0.1459 0.1459 0.1459 36.02%
Gravity::solve_phi_with_mlmg() 6 0.1419 0.1419 0.1419 35.02%
Gravity::solve_for_phi() 5 0.1361 0.1361 0.1361 33.59%
Gravity::actual_solve_with_mlmg() 6 0.1356 0.1356 0.1356 33.47%
Castro::construct_ctu_hydro_source() 5 0.1245 0.1245 0.1245 30.75%
MLMG::solve() 6 0.123 0.123 0.123 30.35%
MLMG::oneIter() 36 0.1151 0.1151 0.1151 28.41%
MLMG::mgVcycle() 36 0.1144 0.1144 0.1144 28.23%
MLCellLinOp::smooth() 720 0.05982 0.05982 0.05982 14.77%
Amr::init() 1 0.05456 0.05456 0.05456 13.47%
Amr::restart() 1 0.05456 0.05456 0.05456 13.47%
MLCellLinOp::applyBC() 1946 0.04566 0.04566 0.04566 11.27%
AmrLevel::restart() 1 0.04377 0.04377 0.04377 10.81%
StateData::restartDoit() 4 0.04369 0.04369 0.04369 10.79%
VisMF::Read() 3 0.04286 0.04286 0.04286 10.58%
MLMG::mgVcycle_bottom 36 0.03304 0.03304 0.03304 8.16%
MLMG::actualBottomSolve() 36 0.03303 0.03303 0.03303 8.15%
MLCGSolver::bicgstab 36 0.03271 0.03271 0.03271 8.07%
MLPoisson::Fsmooth() 1440 0.02555 0.02555 0.02555 6.31%
FillPatchIterator::Initialize 20 0.02439 0.02439 0.02439 6.02%
Amr::writePlotFile() 1 0.02319 0.02319 0.02319 5.73%
VisMF::Write(FabArray) 1 0.02183 0.02183 0.02183 5.39%
FillPatchSingleLevel 20 0.02024 0.02024 0.02024 5.00%
StateDataPhysBCFunct::() 20 0.01813 0.01813 0.01813 4.47%
MLCellLinOp::apply() 500 0.01647 0.01647 0.01647 4.07%
FabArray::setVal() 537 0.01622 0.01622 0.01622 4.01%
MLMG::mgVcycle_down::0 36 0.01593 0.01593 0.01593 3.93%
Castro::initialize_do_advance() 5 0.01425 0.01425 0.01425 3.52%
Castro::clean_state() 30 0.01408 0.01408 0.01408 3.48%
MLMG::mgVcycle_up::0 36 0.01361 0.01361 0.01361 3.36%
Castro::expand_state() 5 0.01198 0.01198 0.01198 2.96%
StateData::FillBoundary(geom) 160 0.01095 0.01095 0.01095 2.70%
Castro::initialize_advance() 5 0.009679 0.009679 0.009679 2.39%
Gravity::get_new_grav_vector() 5 0.009566 0.009566 0.009566 2.36%
MLCellLinOp::correctionResidual() 216 0.009294 0.009294 0.009294 2.29%
MLPoisson::define() 6 0.009274 0.009274 0.009274 2.29%
MultiFab::Dot() 484 0.009178 0.009178 0.009178 2.27%
Castro::computeTemp() 30 0.009105 0.009105 0.009105 2.25%
MLMG:computeResOfCorrection() 180 0.008047 0.008047 0.008047 1.99%
FabArray::FillBoundary() 1766 0.007948 0.007948 0.007948 1.96%
MLMG::mgVcycle_down::1 36 0.00773 0.00773 0.00773 1.91%
FabArray::ParallelCopy() 380 0.007658 0.007658 0.007658 1.89%
Castro::construct_old_gravity() 5 0.007657 0.007657 0.007657 1.89%
Gravity::get_old_grav_vector() 5 0.007619 0.007619 0.007619 1.88%
FillBoundary_nowait() 1766 0.007538 0.007538 0.007538 1.86%
FabArray::ParallelCopy_nowait() 380 0.007513 0.007513 0.007513 1.85%
MLMG::mgVcycle_down::2 36 0.007456 0.007456 0.007456 1.84%
MLMG::mgVcycle_down::3 36 0.007125 0.007125 0.007125 1.76%
Castro::do_new_sources() 5 0.007043 0.007043 0.007043 1.74%
MLMG::mgVcycle_down::4 36 0.006795 0.006795 0.006795 1.68%
Castro::post_restart() 1 0.006625 0.006625 0.006625 1.64%
MLCellLinOp::defineAuxData() 6 0.006357 0.006357 0.006357 1.57%
Gravity::multilevel_solve_for_new_phi() 1 0.00624 0.00624 0.00624 1.54%
Gravity::actual_multilevel_solve() 1 0.006224 0.006224 0.006224 1.54%
Gravity::fill_multipole_BCs() 6 0.006136 0.006136 0.006136 1.51%
MLMG::mgVcycle_up::2 36 0.005807 0.005807 0.005807 1.43%
MLMG::addInterpCorrection() 180 0.005801 0.005801 0.005801 1.43%
MLMG::mgVcycle_up::1 36 0.005733 0.005733 0.005733 1.42%
amrex::average_down 180 0.005593 0.005593 0.005593 1.38%
MLCGSolver::ParallelAllReduce 659 0.005559 0.005559 0.005559 1.37%
MLMG::mgVcycle_up::3 36 0.005508 0.005508 0.005508 1.36%
MLMG::mgVcycle_up::4 36 0.005468 0.005468 0.005468 1.35%
CGSolver::sxay() 690 0.005458 0.005458 0.005458 1.35%
MultiFab::LinComb() 690 0.005252 0.005252 0.005252 1.30%
MLPoisson::Fapply() 500 0.004597 0.004597 0.004597 1.13%
FabArray::setDomainBndry() 20 0.004031 0.004031 0.004031 0.99%
Castro::do_old_sources() 5 0.0036 0.0036 0.0036 0.89%
MLMG::prepareForSolve() 6 0.003482 0.003482 0.003482 0.86%
MLCellLinOp::solutionResidual() 42 0.003385 0.003385 0.003385 0.84%
Castro::enforce_min_density() 30 0.003128 0.003128 0.003128 0.77%
Castro::post_timestep() 5 0.002861 0.002861 0.002861 0.71%
MLMG::computeResidual() 36 0.002806 0.002806 0.002806 0.69%
MLCellLinOp::defineBC() 6 0.002745 0.002745 0.002745 0.68%
BndryData::define() 6 0.00264 0.00264 0.00264 0.65%
MultiFab::Xpay() 258 0.00264 0.00264 0.00264 0.65%
Castro::estTimeStep() 10 0.002443 0.002443 0.002443 0.60%
Castro::reset_internal_energy() 30 0.002303 0.002303 0.002303 0.57%
MLMG::getGradSolution() 6 0.001757 0.001757 0.001757 0.43%
MLCellLinOp::compGrad() 6 0.001754 0.001754 0.001754 0.43%
Castro::construct_new_source() 25 0.001637 0.001637 0.001637 0.40%
Castro::construct_new_gravity_source() 5 0.001606 0.001606 0.001606 0.40%
Castro::computeNewDt() 5 0.001468 0.001468 0.001468 0.36%
Castro::construct_old_source() 25 0.001364 0.001364 0.001364 0.34%
Castro::construct_old_gravity_source() 5 0.001353 0.001353 0.001353 0.33%
Castro::normalize_species() 30 0.001298 0.001298 0.001298 0.32%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001195 0.001195 0.001195 0.29%
Castro::apply_source_to_state() 10 0.0009204 0.0009204 0.0009204 0.23%
MultiFab::Saxpy() 10 0.0009146 0.0009146 0.0009146 0.23%
Gravity::swapTimeLevels() 5 0.000877 0.000877 0.000877 0.22%
FabArrayBase::getCPC() 632 0.0008593 0.0008593 0.0008593 0.21%
MLMG::ResNormInf() 42 0.0008533 0.0008533 0.0008533 0.21%
MLCellLinOp::setLevelBC() 6 0.0008158 0.0008158 0.0008158 0.20%
Gravity::update_max_rhs() 6 0.0007292 0.0007292 0.0007292 0.18%
MLPoisson::prepareForSolve() 6 0.0006902 0.0006902 0.0006902 0.17%
MLCellLinOp::prepareForSolve() 6 0.0006858 0.0006858 0.0006858 0.17%
FabArray::mult() 22 0.0006527 0.0006527 0.0006527 0.16%
MLMG::computeMLResidual() 6 0.0005988 0.0005988 0.0005988 0.15%
Castro::enforce_speed_limit() 30 0.0005179 0.0005179 0.0005179 0.13%
FabArrayBase::CPC::define() 244 0.0004292 0.0004292 0.0004292 0.11%
FabArrayBase::getFB() 1766 0.000402 0.000402 0.000402 0.10%
Amr::InitAmr() 1 0.0003845 0.0003845 0.0003845 0.09%
MultiFab::Copy() 6 0.0003689 0.0003689 0.0003689 0.09%
MultiFab::min() 5 0.0001592 0.0001592 0.0001592 0.04%
MLLinOp::define() 6 0.0001515 0.0001515 0.0001515 0.04%
MLMG::MLResNormInf() 6 0.0001357 0.0001357 0.0001357 0.03%
MultiFab::max() 6 0.0001331 0.0001331 0.0001331 0.03%
MLLinOp::defineGrids() 6 0.0001238 0.0001238 0.0001238 0.03%
MLMG::MLRhsNormInf() 6 0.0001055 0.0001055 0.0001055 0.03%
Castro::create_source_corrector() 5 7.391e-05 7.391e-05 7.391e-05 0.02%
FabArrayBase::FB::FB() 26 6.195e-05 6.195e-05 6.195e-05 0.02%
MLLinOp::makeAgglomeratedDMap 6 2.966e-05 2.966e-05 2.966e-05 0.01%
Castro::finalize_advance() 5 2.868e-05 2.868e-05 2.868e-05 0.01%
Castro::swap_state_time_levels() 5 2.66e-05 2.66e-05 2.66e-05 0.01%
Amr::writeSmallPlotFile() 1 2.506e-05 2.506e-05 2.506e-05 0.01%
makeSFC 30 2.176e-05 2.176e-05 2.176e-05 0.01%
Castro::finalize_do_advance() 5 2.103e-05 2.103e-05 2.103e-05 0.01%
DistributionMapping::Distribute() 31 8.905e-06 8.905e-06 8.905e-06 0.00%
Amr::initSubcycle() 1 8.566e-06 8.566e-06 8.566e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.23e-06 4.23e-06 4.23e-06 0.00%
MLMG::buildFineMask() 6 3.825e-06 3.825e-06 3.825e-06 0.00%
Gravity::set_mass_offset() 6 2.498e-06 2.498e-06 2.498e-06 0.00%
Castro::retry_advance_ctu() 5 1.743e-06 1.743e-06 1.743e-06 0.00%
Castro::FluxRegFineAdd() 5 1.689e-06 1.689e-06 1.689e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.177e-06 1.177e-06 1.177e-06 0.00%
AmrLevel::AmrLevel() 1 8.51e-07 8.51e-07 8.51e-07 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.02-7-g9558c8a1684c) finalized