Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.02-14-gaa0dc39f50f7) initialized
Starting run at 08:53:57 UTC on 2022-02-22.
Successfully read inputs file ...
Castro git describe: 22.02-4-g0d942c92d
AMReX git describe: 22.02-14-gaa0dc39f5
Microphysics git describe: 22.02-4-ga894d649
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.038097979 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.022580371 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.110055024
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.049727132
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.049147507
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.060818427
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.060979661
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.034960001 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.061617462
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.064459566
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.063556242
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.061286856
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.064044627
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.035156502 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.021914796 seconds
Ending run at 08:53:58 UTC on 2022-02-22.
Run time = 0.859898387
Run time without initialization = 0.738208236
Average number of zones advanced per microsecond: 3.551
Average number of zones advanced per microsecond per rank: 3.551
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.8599 ... 0.8599 ... 0.8599
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.2471 0.2471 0.2471 28.74%
VisMF::Write(FabArray) 11 0.1459 0.1459 0.1459 16.96%
MLCellLinOp::applyBC() 4433 0.0927 0.0927 0.0927 10.78%
MLPoisson::Fsmooth() 3280 0.05872 0.05872 0.05872 6.83%
FabArray::setVal() 1144 0.02546 0.02546 0.02546 2.96%
StateData::FillBoundary(geom) 328 0.02317 0.02317 0.02317 2.69%
MLCGSolver::bicgstab 82 0.02128 0.02128 0.02128 2.47%
MultiFab::Dot() 1114 0.02116 0.02116 0.02116 2.46%
FillBoundary_nowait() 4023 0.01704 0.01704 0.01704 1.98%
FabArray::ParallelCopy_nowait() 861 0.01582 0.01582 0.01582 1.84%
StateDataPhysBCFunct::() 41 0.01543 0.01543 0.01543 1.79%
Castro::computeTemp() 63 0.01376 0.01376 0.01376 1.60%
Castro::enforce_min_density() 62 0.01262 0.01262 0.01262 1.47%
Gravity::fill_multipole_BCs() 11 0.01219 0.01219 0.01219 1.42%
MultiFab::LinComb() 1586 0.01211 0.01211 0.01211 1.41%
MLPoisson::Fapply() 1142 0.0104 0.0104 0.0104 1.21%
MLCellLinOp::defineAuxData() 11 0.01015 0.01015 0.01015 1.18%
MLMG::addInterpCorrection() 410 0.007007 0.007007 0.007007 0.81%
amrex::average_down 410 0.006489 0.006489 0.006489 0.75%
Castro::estTimeStep() 21 0.00596 0.00596 0.00596 0.69%
MultiFab::Xpay() 585 0.005955 0.005955 0.005955 0.69%
Castro::reset_internal_energy() 63 0.005925 0.005925 0.005925 0.69%
FabArray::setDomainBndry() 41 0.005794 0.005794 0.005794 0.67%
Castro::enforce_speed_limit() 62 0.005672 0.005672 0.005672 0.66%
Castro::expand_state() 10 0.004713 0.004713 0.004713 0.55%
Amr::checkPoint() 3 0.003862 0.003862 0.003862 0.45%
BndryData::define() 11 0.003542 0.003542 0.003542 0.41%
Castro::do_advance_ctu() 10 0.003245 0.003245 0.003245 0.38%
Castro::construct_new_gravity_source() 10 0.003032 0.003032 0.003032 0.35%
Amr::writePlotFile() 2 0.002984 0.002984 0.002984 0.35%
Gravity::get_new_grav_vector() 11 0.002794 0.002794 0.002794 0.32%
Castro::construct_old_gravity_source() 10 0.002658 0.002658 0.002658 0.31%
Castro::normalize_species() 62 0.002603 0.002603 0.002603 0.30%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002225 0.002225 0.002225 0.26%
MLCellLinOp::compGrad() 11 0.001901 0.001901 0.001901 0.22%
MLMG::ResNormInf() 93 0.001879 0.001879 0.001879 0.22%
Gravity::get_old_grav_vector() 10 0.001859 0.001859 0.001859 0.22%
MultiFab::Saxpy() 20 0.001815 0.001815 0.001815 0.21%
MLMG::oneIter() 82 0.001648 0.001648 0.001648 0.19%
MLCellLinOp::setLevelBC() 11 0.001515 0.001515 0.001515 0.18%
Gravity::actual_solve_with_mlmg() 11 0.001443 0.001443 0.001443 0.17%
MLCellLinOp::prepareForSolve() 11 0.001336 0.001336 0.001336 0.16%
FabArray::mult() 43 0.001317 0.001317 0.001317 0.15%
Castro::initData() 1 0.001163 0.001163 0.001163 0.14%
MLCellLinOp::smooth() 1640 0.001162 0.001162 0.001162 0.14%
FabArrayBase::getCPC() 1323 0.0008738 0.0008738 0.0008738 0.10%
FabArray::FillBoundary() 4023 0.0008185 0.0008185 0.0008185 0.10%
MLMG::prepareForSolve() 11 0.0008038 0.0008038 0.0008038 0.09%
FabArrayBase::CPC::define() 454 0.0007747 0.0007747 0.0007747 0.09%
FabArrayBase::getFB() 4023 0.0007156 0.0007156 0.0007156 0.08%
MLCellLinOp::apply() 1142 0.0005363 0.0005363 0.0005363 0.06%
Gravity::update_max_rhs() 11 0.0005362 0.0005362 0.0005362 0.06%
MultiFab::Copy() 11 0.000507 0.000507 0.000507 0.06%
CGSolver::sxay() 1586 0.000476 0.000476 0.000476 0.06%
Amr::InitAmr() 1 0.0004629 0.0004629 0.0004629 0.05%
Gravity::solve_for_phi() 10 0.0004339 0.0004339 0.0004339 0.05%
MLMG::mgVcycle() 82 0.0003797 0.0003797 0.0003797 0.04%
MultiFab::min() 10 0.0003216 0.0003216 0.0003216 0.04%
MLCGSolver::ParallelAllReduce 1514 0.0003164 0.0003164 0.0003164 0.04%
main() 1 0.000297 0.000297 0.000297 0.03%
FabArray::ParallelCopy() 861 0.0002826 0.0002826 0.0002826 0.03%
MultiFab::max() 11 0.0002537 0.0002537 0.0002537 0.03%
FillPatchIterator::Initialize 41 0.000252 0.000252 0.000252 0.03%
Castro::construct_new_gravity() 10 0.0002347 0.0002347 0.0002347 0.03%
Gravity::actual_multilevel_solve() 1 0.0002255 0.0002255 0.0002255 0.03%
MLCellLinOp::correctionResidual() 492 0.0002188 0.0002188 0.0002188 0.03%
Amr::coarseTimeStep() 10 0.0002123 0.0002123 0.0002123 0.02%
Amr::timeStep() 10 0.0001962 0.0001962 0.0001962 0.02%
MLCellLinOp::defineBC() 11 0.0001957 0.0001957 0.0001957 0.02%
MLMG::MLRhsNormInf() 11 0.0001944 0.0001944 0.0001944 0.02%
MLLinOp::defineGrids() 11 0.0001782 0.0001782 0.0001782 0.02%
MLMG:computeResOfCorrection() 410 0.0001698 0.0001698 0.0001698 0.02%
Amr::defBaseLevel() 1 0.0001538 0.0001538 0.0001538 0.02%
StateData::checkPoint() 12 0.0001398 0.0001398 0.0001398 0.02%
Castro::subcycle_advance_ctu() 10 0.0001265 0.0001265 0.0001265 0.01%
MLMG::actualBottomSolve() 82 0.0001124 0.0001124 0.0001124 0.01%
FabArrayBase::FB::FB() 56 9.774e-05 9.774e-05 9.774e-05 0.01%
MLMG::mgVcycle_down::0 82 9.425e-05 9.425e-05 9.425e-05 0.01%
MLMG::mgVcycle_down::1 82 8.086e-05 8.086e-05 8.086e-05 0.01%
Castro::clean_state() 62 8.018e-05 8.018e-05 8.018e-05 0.01%
Castro::initialize_advance() 10 8.017e-05 8.017e-05 8.017e-05 0.01%
Castro::advance() 10 7.985e-05 7.985e-05 7.985e-05 0.01%
MLMG::mgVcycle_down::2 82 7.744e-05 7.744e-05 7.744e-05 0.01%
MLMG::mgVcycle_down::4 82 7.594e-05 7.594e-05 7.594e-05 0.01%
AmrLevel::checkPoint() 3 7.579e-05 7.579e-05 7.579e-05 0.01%
MLMG::mgVcycle_down::3 82 7.449e-05 7.449e-05 7.449e-05 0.01%
MLMG::solve() 11 6.882e-05 6.882e-05 6.882e-05 0.01%
MLMG::mgVcycle_up::4 82 6.088e-05 6.088e-05 6.088e-05 0.01%
Castro::finalize_advance() 10 5.733e-05 5.733e-05 5.733e-05 0.01%
Castro::initialize_do_advance() 10 5.727e-05 5.727e-05 5.727e-05 0.01%
Castro::create_source_corrector() 10 5.067e-05 5.067e-05 5.067e-05 0.01%
MLCellLinOp::solutionResidual() 93 5.04e-05 5.04e-05 5.04e-05 0.01%
MLMG::mgVcycle_up::0 82 4.897e-05 4.897e-05 4.897e-05 0.01%
MLMG::mgVcycle_up::1 82 4.73e-05 4.73e-05 4.73e-05 0.01%
MLMG::mgVcycle_up::3 82 4.714e-05 4.714e-05 4.714e-05 0.01%
MLMG::mgVcycle_up::2 82 4.516e-05 4.516e-05 4.516e-05 0.01%
StateData::define() 4 3.943e-05 3.943e-05 3.943e-05 0.00%
Castro::finalize_do_advance() 10 3.863e-05 3.863e-05 3.863e-05 0.00%
Castro::swap_state_time_levels() 10 3.793e-05 3.793e-05 3.793e-05 0.00%
MLMG::mgVcycle_bottom 82 3.624e-05 3.624e-05 3.624e-05 0.00%
Castro::enforce_consistent_e() 1 3.482e-05 3.482e-05 3.482e-05 0.00%
MLMG::computeResidual() 82 3.448e-05 3.448e-05 3.448e-05 0.00%
Castro::construct_new_source() 50 3.337e-05 3.337e-05 3.337e-05 0.00%
MLLinOp::define() 11 2.906e-05 2.906e-05 2.906e-05 0.00%
FillPatchSingleLevel 41 2.744e-05 2.744e-05 2.744e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.705e-05 2.705e-05 2.705e-05 0.00%
makeSFC 55 2.577e-05 2.577e-05 2.577e-05 0.00%
Amr::writeSmallPlotFile() 1 2.365e-05 2.365e-05 2.365e-05 0.00%
MLPoisson::define() 11 2.291e-05 2.291e-05 2.291e-05 0.00%
Amr::FinalizeInit() 1 2.071e-05 2.071e-05 2.071e-05 0.00%
Amr::initSubcycle() 1 1.972e-05 1.972e-05 1.972e-05 0.00%
Castro::construct_old_source() 50 1.755e-05 1.755e-05 1.755e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.723e-05 1.723e-05 1.723e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.717e-05 1.717e-05 1.717e-05 0.00%
Castro::do_new_sources() 10 1.688e-05 1.688e-05 1.688e-05 0.00%
Castro::apply_source_to_state() 20 1.512e-05 1.512e-05 1.512e-05 0.00%
Castro::do_old_sources() 10 1.496e-05 1.496e-05 1.496e-05 0.00%
DistributionMapping::Distribute() 56 1.421e-05 1.421e-05 1.421e-05 0.00%
Castro::construct_old_gravity() 10 1.023e-05 1.023e-05 1.023e-05 0.00%
Gravity::swapTimeLevels() 10 9.629e-06 9.629e-06 9.629e-06 0.00%
AmrLevel::AmrLevel(dm) 1 8.534e-06 8.534e-06 8.534e-06 0.00%
MLPoisson::prepareForSolve() 11 7.93e-06 7.93e-06 7.93e-06 0.00%
Castro::post_timestep() 10 7.258e-06 7.258e-06 7.258e-06 0.00%
MLMG::computeMLResidual() 11 6.56e-06 6.56e-06 6.56e-06 0.00%
Amr::InitializeInit() 1 6.363e-06 6.363e-06 6.363e-06 0.00%
AmrLevel::checkPointPost() 3 6.071e-06 6.071e-06 6.071e-06 0.00%
Castro::computeNewDt() 9 6.052e-06 6.052e-06 6.052e-06 0.00%
MLMG::getGradSolution() 11 5.705e-06 5.705e-06 5.705e-06 0.00%
MLMG::buildFineMask() 11 5.361e-06 5.361e-06 5.361e-06 0.00%
MLMG::MLResNormInf() 11 4.394e-06 4.394e-06 4.394e-06 0.00%
Gravity::set_mass_offset() 11 4.083e-06 4.083e-06 4.083e-06 0.00%
Castro::post_init() 1 3.908e-06 3.908e-06 3.908e-06 0.00%
Castro::retry_advance_ctu() 10 3.79e-06 3.79e-06 3.79e-06 0.00%
Castro::FluxRegFineAdd() 10 3.223e-06 3.223e-06 3.223e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.81e-06 2.81e-06 2.81e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.163e-06 2.163e-06 2.163e-06 0.00%
Amr::init() 1 2.155e-06 2.155e-06 2.155e-06 0.00%
AmrLevel::checkPointPre() 3 2.063e-06 2.063e-06 2.063e-06 0.00%
Castro::computeInitialDt() 2 1.755e-06 1.755e-06 1.755e-06 0.00%
Amr::initialInit() 1 1.177e-06 1.177e-06 1.177e-06 0.00%
Castro::post_regrid() 1 1.139e-06 1.139e-06 1.139e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.8599 0.8599 0.8599 100.00%
Amr::coarseTimeStep() 10 0.7161 0.7161 0.7161 83.27%
Amr::timeStep() 10 0.6424 0.6424 0.6424 74.70%
Castro::advance() 10 0.6335 0.6335 0.6335 73.67%
Castro::subcycle_advance_ctu() 10 0.6193 0.6193 0.6193 72.02%
Castro::do_advance_ctu() 10 0.6192 0.6192 0.6192 72.01%
Gravity::solve_phi_with_mlmg() 11 0.3214 0.3214 0.3214 37.37%
Gravity::actual_solve_with_mlmg() 11 0.309 0.309 0.309 35.93%
Castro::construct_new_gravity() 10 0.2917 0.2917 0.2917 33.92%
MLMG::solve() 11 0.2864 0.2864 0.2864 33.31%
Gravity::solve_for_phi() 10 0.2739 0.2739 0.2739 31.85%
MLMG::oneIter() 82 0.2709 0.2709 0.2709 31.50%
MLMG::mgVcycle() 82 0.2692 0.2692 0.2692 31.31%
Castro::construct_ctu_hydro_source() 10 0.2471 0.2471 0.2471 28.74%
VisMF::Write(FabArray) 11 0.1459 0.1459 0.1459 16.96%
MLCellLinOp::smooth() 1640 0.1425 0.1425 0.1425 16.57%
Amr::init() 1 0.1211 0.1211 0.1211 14.08%
MLCellLinOp::applyBC() 4433 0.1114 0.1114 0.1114 12.95%
Amr::checkPoint() 3 0.1083 0.1083 0.1083 12.60%
AmrLevel::checkPoint() 3 0.1045 0.1045 0.1045 12.15%
StateData::checkPoint() 12 0.1044 0.1044 0.1044 12.14%
MLMG::mgVcycle_bottom 82 0.07669 0.07669 0.07669 8.92%
MLMG::actualBottomSolve() 82 0.07665 0.07665 0.07665 8.91%
MLCGSolver::bicgstab 82 0.07593 0.07593 0.07593 8.83%
Amr::initialInit() 1 0.06027 0.06027 0.06027 7.01%
MLPoisson::Fsmooth() 3280 0.05872 0.05872 0.05872 6.83%
Amr::FinalizeInit() 1 0.0514 0.0514 0.0514 5.98%
Castro::post_init() 1 0.05072 0.05072 0.05072 5.90%
FillPatchIterator::Initialize 41 0.04882 0.04882 0.04882 5.68%
Gravity::multilevel_solve_for_new_phi() 1 0.04819 0.04819 0.04819 5.60%
Gravity::actual_multilevel_solve() 1 0.04817 0.04817 0.04817 5.60%
Amr::writePlotFile() 2 0.04461 0.04461 0.04461 5.19%
FillPatchSingleLevel 41 0.04277 0.04277 0.04277 4.97%
Castro::clean_state() 62 0.04 0.04 0.04 4.65%
MLCellLinOp::apply() 1142 0.03914 0.03914 0.03914 4.55%
StateDataPhysBCFunct::() 41 0.0386 0.0386 0.0386 4.49%
MLMG::mgVcycle_down::0 82 0.03744 0.03744 0.03744 4.35%
MLMG::mgVcycle_up::0 82 0.03207 0.03207 0.03207 3.73%
FabArray::setVal() 1144 0.02546 0.02546 0.02546 2.96%
Castro::initialize_do_advance() 10 0.02405 0.02405 0.02405 2.80%
StateData::FillBoundary(geom) 328 0.02317 0.02317 0.02317 2.69%
MLCellLinOp::correctionResidual() 492 0.02192 0.02192 0.02192 2.55%
MultiFab::Dot() 1114 0.02116 0.02116 0.02116 2.46%
Gravity::get_new_grav_vector() 11 0.01994 0.01994 0.01994 2.32%
Castro::computeTemp() 63 0.01968 0.01968 0.01968 2.29%
MLMG:computeResOfCorrection() 410 0.01899 0.01899 0.01899 2.21%
FabArray::FillBoundary() 4023 0.01867 0.01867 0.01867 2.17%
MLMG::mgVcycle_down::1 82 0.01831 0.01831 0.01831 2.13%
Castro::expand_state() 10 0.01826 0.01826 0.01826 2.12%
FillBoundary_nowait() 4023 0.01785 0.01785 0.01785 2.08%
MLMG::mgVcycle_down::2 82 0.01771 0.01771 0.01771 2.06%
MLPoisson::define() 11 0.01712 0.01712 0.01712 1.99%
FabArray::ParallelCopy() 861 0.01704 0.01704 0.01704 1.98%
MLMG::mgVcycle_down::3 82 0.01693 0.01693 0.01693 1.97%
FabArray::ParallelCopy_nowait() 861 0.01675 0.01675 0.01675 1.95%
MLMG::mgVcycle_down::4 82 0.01618 0.01618 0.01618 1.88%
Castro::construct_old_gravity() 10 0.01574 0.01574 0.01574 1.83%
Gravity::get_old_grav_vector() 10 0.01573 0.01573 0.01573 1.83%
Castro::initialize_advance() 10 0.01402 0.01402 0.01402 1.63%
MLMG::mgVcycle_up::2 82 0.01382 0.01382 0.01382 1.61%
MLMG::mgVcycle_up::1 82 0.01359 0.01359 0.01359 1.58%
MLMG::addInterpCorrection() 410 0.01345 0.01345 0.01345 1.56%
MLMG::mgVcycle_up::3 82 0.01313 0.01313 0.01313 1.53%
MLMG::mgVcycle_up::4 82 0.01296 0.01296 0.01296 1.51%
amrex::average_down 410 0.01293 0.01293 0.01293 1.50%
MLCGSolver::ParallelAllReduce 1514 0.01277 0.01277 0.01277 1.48%
Castro::enforce_min_density() 62 0.01262 0.01262 0.01262 1.47%
CGSolver::sxay() 1586 0.01259 0.01259 0.01259 1.46%
Gravity::fill_multipole_BCs() 11 0.01219 0.01219 0.01219 1.42%
MultiFab::LinComb() 1586 0.01211 0.01211 0.01211 1.41%
MLCellLinOp::defineAuxData() 11 0.01183 0.01183 0.01183 1.38%
Castro::do_new_sources() 10 0.01152 0.01152 0.01152 1.34%
MLPoisson::Fapply() 1142 0.0104 0.0104 0.0104 1.21%
Castro::do_old_sources() 10 0.00932 0.00932 0.00932 1.08%
Amr::InitializeInit() 1 0.008866 0.008866 0.008866 1.03%
Amr::defBaseLevel() 1 0.00886 0.00886 0.00886 1.03%
Castro::post_timestep() 10 0.008713 0.008713 0.008713 1.01%
MLCellLinOp::solutionResidual() 93 0.007622 0.007622 0.007622 0.89%
MLMG::computeResidual() 82 0.006585 0.006585 0.006585 0.77%
Castro::estTimeStep() 21 0.00596 0.00596 0.00596 0.69%
MultiFab::Xpay() 585 0.005955 0.005955 0.005955 0.69%
Castro::reset_internal_energy() 63 0.005925 0.005925 0.005925 0.69%
FabArray::setDomainBndry() 41 0.005794 0.005794 0.005794 0.67%
MLMG::prepareForSolve() 11 0.005785 0.005785 0.005785 0.67%
Castro::enforce_speed_limit() 62 0.005672 0.005672 0.005672 0.66%
MLCellLinOp::defineBC() 11 0.004995 0.004995 0.004995 0.58%
BndryData::define() 11 0.004799 0.004799 0.004799 0.56%
Castro::initData() 1 0.003256 0.003256 0.003256 0.38%
Castro::construct_new_source() 50 0.003066 0.003066 0.003066 0.36%
Castro::construct_new_gravity_source() 10 0.003032 0.003032 0.003032 0.35%
Castro::computeNewDt() 9 0.002804 0.002804 0.002804 0.33%
Castro::construct_old_source() 50 0.002676 0.002676 0.002676 0.31%
Castro::construct_old_gravity_source() 10 0.002658 0.002658 0.002658 0.31%
Castro::normalize_species() 62 0.002603 0.002603 0.002603 0.30%
MLMG::getGradSolution() 11 0.002438 0.002438 0.002438 0.28%
MLCellLinOp::compGrad() 11 0.002432 0.002432 0.002432 0.28%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.002225 0.002225 0.002225 0.26%
MLMG::ResNormInf() 93 0.001879 0.001879 0.001879 0.22%
Castro::apply_source_to_state() 20 0.00183 0.00183 0.00183 0.21%
MultiFab::Saxpy() 20 0.001815 0.001815 0.001815 0.21%
FabArrayBase::getCPC() 1323 0.001648 0.001648 0.001648 0.19%
MLCellLinOp::setLevelBC() 11 0.001515 0.001515 0.001515 0.18%
MLPoisson::prepareForSolve() 11 0.001344 0.001344 0.001344 0.16%
MLCellLinOp::prepareForSolve() 11 0.001336 0.001336 0.001336 0.16%
FabArray::mult() 43 0.001317 0.001317 0.001317 0.15%
MLMG::computeMLResidual() 11 0.001078 0.001078 0.001078 0.13%
Gravity::swapTimeLevels() 10 0.001035 0.001035 0.001035 0.12%
Gravity::update_max_rhs() 11 0.0009315 0.0009315 0.0009315 0.11%
FabArrayBase::getFB() 4023 0.0008133 0.0008133 0.0008133 0.09%
FabArrayBase::CPC::define() 454 0.0007747 0.0007747 0.0007747 0.09%
Castro::computeInitialDt() 2 0.0006842 0.0006842 0.0006842 0.08%
MultiFab::Copy() 11 0.000507 0.000507 0.000507 0.06%
Amr::InitAmr() 1 0.0004826 0.0004826 0.0004826 0.06%
Castro::post_regrid() 1 0.0004513 0.0004513 0.0004513 0.05%
MultiFab::min() 10 0.0003216 0.0003216 0.0003216 0.04%
MLLinOp::define() 11 0.0002658 0.0002658 0.0002658 0.03%
MultiFab::max() 11 0.0002537 0.0002537 0.0002537 0.03%
MLMG::MLResNormInf() 11 0.0002501 0.0002501 0.0002501 0.03%
MLLinOp::defineGrids() 11 0.0002368 0.0002368 0.0002368 0.03%
MLMG::MLRhsNormInf() 11 0.0001944 0.0001944 0.0001944 0.02%
FabArrayBase::FB::FB() 56 9.774e-05 9.774e-05 9.774e-05 0.01%
Castro::finalize_advance() 10 6.055e-05 6.055e-05 6.055e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.636e-05 5.636e-05 5.636e-05 0.01%
Castro::create_source_corrector() 10 5.067e-05 5.067e-05 5.067e-05 0.01%
AmrLevel::AmrLevel(dm) 1 4.796e-05 4.796e-05 4.796e-05 0.01%
StateData::define() 4 3.943e-05 3.943e-05 3.943e-05 0.00%
makeSFC 55 3.913e-05 3.913e-05 3.913e-05 0.00%
Castro::finalize_do_advance() 10 3.863e-05 3.863e-05 3.863e-05 0.00%
Castro::swap_state_time_levels() 10 3.793e-05 3.793e-05 3.793e-05 0.00%
Castro::enforce_consistent_e() 1 3.482e-05 3.482e-05 3.482e-05 0.00%
Amr::writeSmallPlotFile() 1 2.365e-05 2.365e-05 2.365e-05 0.00%
Amr::initSubcycle() 1 1.972e-05 1.972e-05 1.972e-05 0.00%
DistributionMapping::Distribute() 56 1.421e-05 1.421e-05 1.421e-05 0.00%
AmrLevel::checkPointPost() 3 6.071e-06 6.071e-06 6.071e-06 0.00%
MLMG::buildFineMask() 11 5.361e-06 5.361e-06 5.361e-06 0.00%
Gravity::set_mass_offset() 11 4.083e-06 4.083e-06 4.083e-06 0.00%
Castro::retry_advance_ctu() 10 3.79e-06 3.79e-06 3.79e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.653e-06 3.653e-06 3.653e-06 0.00%
Castro::FluxRegFineAdd() 10 3.223e-06 3.223e-06 3.223e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.163e-06 2.163e-06 2.163e-06 0.00%
AmrLevel::checkPointPre() 3 2.063e-06 2.063e-06 2.063e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.02-14-gaa0dc39f50f7) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.02-14-gaa0dc39f50f7) initialized
Starting run at 08:53:59 UTC on 2022-02-22.
Successfully read inputs file ...
Castro git describe: 22.02-4-g0d942c92d
AMReX git describe: 22.02-14-gaa0dc39f5
Microphysics git describe: 22.02-4-ga894d649
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.487296739
Restart time = 0.054344509 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.112007802
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.05195183
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.06395246
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.066930062
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.066743119
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.039844719 seconds
Ending run at 08:53:59 UTC on 2022-02-22.
Run time = 0.456690033
Run time without initialization = 0.401769733
Average number of zones advanced per microsecond: 3.262
Average number of zones advanced per microsecond per rank: 3.262
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.4567 ... 0.4567 ... 0.4567
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.1560 0.1560 0.1560 34.15%
VisMF::Read() 3 0.0429 0.0429 0.0429 9.39%
MLCellLinOp::applyBC() 1946 0.03747 0.03747 0.03747 8.20%
MLPoisson::Fsmooth() 1440 0.02543 0.02543 0.02543 5.57%
VisMF::Write(FabArray) 1 0.02165 0.02165 0.02165 4.74%
Amr::writePlotFile() 1 0.01827 0.01827 0.01827 4.00%
FabArray::setVal() 537 0.01617 0.01617 0.01617 3.54%
StateData::FillBoundary(geom) 160 0.01129 0.01129 0.01129 2.47%
MLCGSolver::bicgstab 36 0.009161 0.009161 0.009161 2.01%
MultiFab::Dot() 484 0.009092 0.009092 0.009092 1.99%
FillBoundary_nowait() 1766 0.007184 0.007184 0.007184 1.57%
FabArray::ParallelCopy_nowait() 380 0.007027 0.007027 0.007027 1.54%
Castro::enforce_min_density() 30 0.006933 0.006933 0.006933 1.52%
Castro::computeTemp() 30 0.006809 0.006809 0.006809 1.49%
StateDataPhysBCFunct::() 20 0.0067 0.0067 0.0067 1.47%
Gravity::fill_multipole_BCs() 6 0.006297 0.006297 0.006297 1.38%
MLCellLinOp::defineAuxData() 6 0.00542 0.00542 0.00542 1.19%
MultiFab::LinComb() 690 0.005249 0.005249 0.005249 1.15%
MLPoisson::Fapply() 500 0.004566 0.004566 0.004566 1.00%
Castro::expand_state() 5 0.004102 0.004102 0.004102 0.90%
FabArray::setDomainBndry() 20 0.003905 0.003905 0.003905 0.86%
MLMG::addInterpCorrection() 180 0.002987 0.002987 0.002987 0.65%
Amr::restart() 1 0.002941 0.002941 0.002941 0.64%
amrex::average_down 180 0.002742 0.002742 0.002742 0.60%
MultiFab::Xpay() 258 0.002618 0.002618 0.002618 0.57%
Castro::estTimeStep() 10 0.002387 0.002387 0.002387 0.52%
Castro::do_advance_ctu() 5 0.002376 0.002376 0.002376 0.52%
Castro::reset_internal_energy() 30 0.002322 0.002322 0.002322 0.51%
BndryData::define() 6 0.001913 0.001913 0.001913 0.42%
Gravity::get_new_grav_vector() 5 0.001777 0.001777 0.001777 0.39%
Castro::construct_new_gravity_source() 5 0.001579 0.001579 0.001579 0.35%
Castro::subcycle_advance_ctu() 5 0.001427 0.001427 0.001427 0.31%
MLCellLinOp::compGrad() 6 0.001393 0.001393 0.001393 0.31%
Castro::construct_old_gravity_source() 5 0.001311 0.001311 0.001311 0.29%
Castro::normalize_species() 30 0.001293 0.001293 0.001293 0.28%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001199 0.001199 0.001199 0.26%
Gravity::get_old_grav_vector() 5 0.0009574 0.0009574 0.0009574 0.21%
MultiFab::Saxpy() 10 0.0009177 0.0009177 0.0009177 0.20%
MLMG::ResNormInf() 42 0.0008344 0.0008344 0.0008344 0.18%
MLCellLinOp::setLevelBC() 6 0.0008062 0.0008062 0.0008062 0.18%
Gravity::actual_solve_with_mlmg() 6 0.0007281 0.0007281 0.0007281 0.16%
MLMG::oneIter() 36 0.0007152 0.0007152 0.0007152 0.16%
MLCellLinOp::prepareForSolve() 6 0.0006819 0.0006819 0.0006819 0.15%
FabArray::mult() 22 0.0006482 0.0006482 0.0006482 0.14%
Gravity::update_max_rhs() 6 0.0005443 0.0005443 0.0005443 0.12%
MLCellLinOp::smooth() 720 0.0004954 0.0004954 0.0004954 0.11%
Castro::enforce_speed_limit() 30 0.0004765 0.0004765 0.0004765 0.10%
MLMG::prepareForSolve() 6 0.000428 0.000428 0.000428 0.09%
FabArrayBase::getCPC() 632 0.0004271 0.0004271 0.0004271 0.09%
FabArrayBase::CPC::define() 244 0.0004233 0.0004233 0.0004233 0.09%
FabArray::FillBoundary() 1766 0.0004093 0.0004093 0.0004093 0.09%
Amr::InitAmr() 1 0.0003891 0.0003891 0.0003891 0.09%
MultiFab::Copy() 6 0.0003591 0.0003591 0.0003591 0.08%
FabArrayBase::getFB() 1766 0.0003344 0.0003344 0.0003344 0.07%
main() 1 0.0002857 0.0002857 0.0002857 0.06%
Gravity::actual_multilevel_solve() 1 0.0002334 0.0002334 0.0002334 0.05%
Gravity::solve_for_phi() 5 0.0002184 0.0002184 0.0002184 0.05%
CGSolver::sxay() 690 0.0002122 0.0002122 0.0002122 0.05%
MLCellLinOp::apply() 500 0.0002045 0.0002045 0.0002045 0.04%
Castro::construct_new_gravity() 5 0.0001789 0.0001789 0.0001789 0.04%
MLMG::mgVcycle() 36 0.0001603 0.0001603 0.0001603 0.04%
MultiFab::min() 5 0.0001572 0.0001572 0.0001572 0.03%
FabArray::ParallelCopy() 380 0.0001436 0.0001436 0.0001436 0.03%
MLCGSolver::ParallelAllReduce 659 0.0001402 0.0001402 0.0001402 0.03%
MultiFab::max() 6 0.0001336 0.0001336 0.0001336 0.03%
Castro::construct_new_source() 25 0.0001242 0.0001242 0.0001242 0.03%
FillPatchIterator::Initialize 20 0.0001219 0.0001219 0.0001219 0.03%
Amr::coarseTimeStep() 5 0.0001114 0.0001114 0.0001114 0.02%
MLCellLinOp::defineBC() 6 0.0001095 0.0001095 0.0001095 0.02%
MLMG::MLRhsNormInf() 6 0.0001055 0.0001055 0.0001055 0.02%
Amr::timeStep() 5 9.92e-05 9.92e-05 9.92e-05 0.02%
MLLinOp::defineGrids() 6 9.609e-05 9.609e-05 9.609e-05 0.02%
MLCellLinOp::correctionResidual() 216 9.382e-05 9.382e-05 9.382e-05 0.02%
Castro::advance() 5 7.954e-05 7.954e-05 7.954e-05 0.02%
AmrLevel::restart() 1 7.596e-05 7.596e-05 7.596e-05 0.02%
StateData::restartDoit() 4 7.573e-05 7.573e-05 7.573e-05 0.02%
Castro::construct_old_source() 25 7.079e-05 7.079e-05 7.079e-05 0.02%
FabArrayBase::FB::FB() 26 6.643e-05 6.643e-05 6.643e-05 0.01%
MLMG:computeResOfCorrection() 180 6.594e-05 6.594e-05 6.594e-05 0.01%
MLMG::actualBottomSolve() 36 4.759e-05 4.759e-05 4.759e-05 0.01%
Castro::initialize_do_advance() 5 4.049e-05 4.049e-05 4.049e-05 0.01%
MLMG::mgVcycle_down::0 36 3.912e-05 3.912e-05 3.912e-05 0.01%
Castro::initialize_advance() 5 3.758e-05 3.758e-05 3.758e-05 0.01%
MLMG::mgVcycle_down::1 36 3.678e-05 3.678e-05 3.678e-05 0.01%
MLMG::mgVcycle_down::2 36 3.371e-05 3.371e-05 3.371e-05 0.01%
Castro::do_new_sources() 5 3.285e-05 3.285e-05 3.285e-05 0.01%
MLMG::mgVcycle_down::3 36 3.271e-05 3.271e-05 3.271e-05 0.01%
MLMG::solve() 6 3.263e-05 3.263e-05 3.263e-05 0.01%
MLMG::mgVcycle_down::4 36 3.236e-05 3.236e-05 3.236e-05 0.01%
Castro::clean_state() 30 3.167e-05 3.167e-05 3.167e-05 0.01%
Castro::post_restart() 1 2.931e-05 2.931e-05 2.931e-05 0.01%
MLMG::mgVcycle_up::4 36 2.808e-05 2.808e-05 2.808e-05 0.01%
Castro::finalize_advance() 5 2.7e-05 2.7e-05 2.7e-05 0.01%
MLLinOp::define() 6 2.629e-05 2.629e-05 2.629e-05 0.01%
Castro::swap_state_time_levels() 5 2.55e-05 2.55e-05 2.55e-05 0.01%
MLMG::mgVcycle_up::0 36 2.528e-05 2.528e-05 2.528e-05 0.01%
MLMG::mgVcycle_up::3 36 2.348e-05 2.348e-05 2.348e-05 0.01%
Amr::writeSmallPlotFile() 1 2.313e-05 2.313e-05 2.313e-05 0.01%
MLMG::mgVcycle_up::2 36 2.268e-05 2.268e-05 2.268e-05 0.00%
MLCellLinOp::solutionResidual() 42 2.264e-05 2.264e-05 2.264e-05 0.00%
MLMG::mgVcycle_up::1 36 2.167e-05 2.167e-05 2.167e-05 0.00%
Castro::finalize_do_advance() 5 2.125e-05 2.125e-05 2.125e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.814e-05 1.814e-05 1.814e-05 0.00%
Castro::create_source_corrector() 5 1.764e-05 1.764e-05 1.764e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.731e-05 1.731e-05 1.731e-05 0.00%
MLMG::computeResidual() 36 1.558e-05 1.558e-05 1.558e-05 0.00%
MLMG::mgVcycle_bottom 36 1.502e-05 1.502e-05 1.502e-05 0.00%
MLPoisson::define() 6 1.477e-05 1.477e-05 1.477e-05 0.00%
makeSFC 30 1.435e-05 1.435e-05 1.435e-05 0.00%
FillPatchSingleLevel 20 1.428e-05 1.428e-05 1.428e-05 0.00%
Amr::initSubcycle() 1 9.637e-06 9.637e-06 9.637e-06 0.00%
DistributionMapping::Distribute() 31 8.923e-06 8.923e-06 8.923e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 8.525e-06 8.525e-06 8.525e-06 0.00%
Castro::do_old_sources() 5 8.513e-06 8.513e-06 8.513e-06 0.00%
Castro::construct_old_gravity() 5 6.32e-06 6.32e-06 6.32e-06 0.00%
Castro::apply_source_to_state() 10 6.23e-06 6.23e-06 6.23e-06 0.00%
Castro::post_timestep() 5 4.756e-06 4.756e-06 4.756e-06 0.00%
Gravity::swapTimeLevels() 5 4.475e-06 4.475e-06 4.475e-06 0.00%
MLPoisson::prepareForSolve() 6 4.425e-06 4.425e-06 4.425e-06 0.00%
MLMG::buildFineMask() 6 3.462e-06 3.462e-06 3.462e-06 0.00%
Castro::computeNewDt() 5 3.333e-06 3.333e-06 3.333e-06 0.00%
MLMG::computeMLResidual() 6 3.137e-06 3.137e-06 3.137e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.093e-06 3.093e-06 3.093e-06 0.00%
MLMG::getGradSolution() 6 3.022e-06 3.022e-06 3.022e-06 0.00%
MLMG::MLResNormInf() 6 2.807e-06 2.807e-06 2.807e-06 0.00%
Gravity::set_mass_offset() 6 2.283e-06 2.283e-06 2.283e-06 0.00%
Castro::retry_advance_ctu() 5 1.566e-06 1.566e-06 1.566e-06 0.00%
Castro::FluxRegFineAdd() 5 1.533e-06 1.533e-06 1.533e-06 0.00%
AmrLevel::AmrLevel() 1 1.128e-06 1.128e-06 1.128e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.126e-06 1.126e-06 1.126e-06 0.00%
Amr::init() 1 9.76e-07 9.76e-07 9.76e-07 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.4567 0.4567 0.4567 100.00%
Amr::coarseTimeStep() 5 0.3617 0.3617 0.3617 79.19%
Amr::timeStep() 5 0.3603 0.3603 0.3603 78.88%
Castro::advance() 5 0.3572 0.3572 0.3572 78.21%
Castro::subcycle_advance_ctu() 5 0.3462 0.3462 0.3462 75.81%
Castro::do_advance_ctu() 5 0.3448 0.3448 0.3448 75.50%
Castro::construct_ctu_hydro_source() 5 0.156 0.156 0.156 34.15%
Castro::construct_new_gravity() 5 0.1447 0.1447 0.1447 31.69%
Gravity::solve_phi_with_mlmg() 6 0.1409 0.1409 0.1409 30.84%
Gravity::solve_for_phi() 5 0.1353 0.1353 0.1353 29.62%
Gravity::actual_solve_with_mlmg() 6 0.1344 0.1344 0.1344 29.43%
MLMG::solve() 6 0.122 0.122 0.122 26.71%
MLMG::oneIter() 36 0.1142 0.1142 0.1142 25.00%
MLMG::mgVcycle() 36 0.1135 0.1135 0.1135 24.84%
MLCellLinOp::smooth() 720 0.05945 0.05945 0.05945 13.02%
Amr::init() 1 0.05438 0.05438 0.05438 11.91%
Amr::restart() 1 0.05438 0.05438 0.05438 11.91%
MLCellLinOp::applyBC() 1946 0.04546 0.04546 0.04546 9.95%
AmrLevel::restart() 1 0.04376 0.04376 0.04376 9.58%
StateData::restartDoit() 4 0.04368 0.04368 0.04368 9.56%
VisMF::Read() 3 0.0429 0.0429 0.0429 9.39%
Amr::writePlotFile() 1 0.03993 0.03993 0.03993 8.74%
MLMG::mgVcycle_bottom 36 0.03275 0.03275 0.03275 7.17%
MLMG::actualBottomSolve() 36 0.03273 0.03273 0.03273 7.17%
MLCGSolver::bicgstab 36 0.03242 0.03242 0.03242 7.10%
MLPoisson::Fsmooth() 1440 0.02543 0.02543 0.02543 5.57%
FillPatchIterator::Initialize 20 0.02411 0.02411 0.02411 5.28%
VisMF::Write(FabArray) 1 0.02165 0.02165 0.02165 4.74%
FillPatchSingleLevel 20 0.02008 0.02008 0.02008 4.40%
StateDataPhysBCFunct::() 20 0.01799 0.01799 0.01799 3.94%
Castro::clean_state() 30 0.01787 0.01787 0.01787 3.91%
MLCellLinOp::apply() 500 0.01642 0.01642 0.01642 3.59%
FabArray::setVal() 537 0.01617 0.01617 0.01617 3.54%
MLMG::mgVcycle_down::0 36 0.01584 0.01584 0.01584 3.47%
Castro::initialize_do_advance() 5 0.01477 0.01477 0.01477 3.23%
MLMG::mgVcycle_up::0 36 0.01354 0.01354 0.01354 2.97%
Castro::expand_state() 5 0.01162 0.01162 0.01162 2.54%
StateData::FillBoundary(geom) 160 0.01129 0.01129 0.01129 2.47%
Castro::initialize_advance() 5 0.01087 0.01087 0.01087 2.38%
Gravity::get_new_grav_vector() 5 0.009295 0.009295 0.009295 2.04%
MLCellLinOp::correctionResidual() 216 0.00924 0.00924 0.00924 2.02%
MLPoisson::define() 6 0.009238 0.009238 0.009238 2.02%
Castro::computeTemp() 30 0.009132 0.009132 0.009132 2.00%
MultiFab::Dot() 484 0.009092 0.009092 0.009092 1.99%
MLMG:computeResOfCorrection() 180 0.008 0.008 0.008 1.75%
FabArray::FillBoundary() 1766 0.007994 0.007994 0.007994 1.75%
Castro::construct_old_gravity() 5 0.007892 0.007892 0.007892 1.73%
Gravity::get_old_grav_vector() 5 0.007885 0.007885 0.007885 1.73%
MLMG::mgVcycle_down::1 36 0.007669 0.007669 0.007669 1.68%
FabArray::ParallelCopy() 380 0.007593 0.007593 0.007593 1.66%
FillBoundary_nowait() 1766 0.007585 0.007585 0.007585 1.66%
FabArray::ParallelCopy_nowait() 380 0.00745 0.00745 0.00745 1.63%
MLMG::mgVcycle_down::2 36 0.007394 0.007394 0.007394 1.62%
MLMG::mgVcycle_down::3 36 0.007046 0.007046 0.007046 1.54%
Castro::do_new_sources() 5 0.007024 0.007024 0.007024 1.54%
Castro::enforce_min_density() 30 0.006933 0.006933 0.006933 1.52%
MLMG::mgVcycle_down::4 36 0.006751 0.006751 0.006751 1.48%
Castro::post_restart() 1 0.006471 0.006471 0.006471 1.42%
MLCellLinOp::defineAuxData() 6 0.00634 0.00634 0.00634 1.39%
Gravity::fill_multipole_BCs() 6 0.006297 0.006297 0.006297 1.38%
Gravity::multilevel_solve_for_new_phi() 1 0.006071 0.006071 0.006071 1.33%
Gravity::actual_multilevel_solve() 1 0.006053 0.006053 0.006053 1.33%
MLMG::mgVcycle_up::2 36 0.00576 0.00576 0.00576 1.26%
MLMG::addInterpCorrection() 180 0.005736 0.005736 0.005736 1.26%
MLMG::mgVcycle_up::1 36 0.005676 0.005676 0.005676 1.24%
amrex::average_down 180 0.005508 0.005508 0.005508 1.21%
MLCGSolver::ParallelAllReduce 659 0.005505 0.005505 0.005505 1.21%
MLMG::mgVcycle_up::3 36 0.005469 0.005469 0.005469 1.20%
CGSolver::sxay() 690 0.005461 0.005461 0.005461 1.20%
MLMG::mgVcycle_up::4 36 0.005414 0.005414 0.005414 1.19%
MultiFab::LinComb() 690 0.005249 0.005249 0.005249 1.15%
Castro::do_old_sources() 5 0.004963 0.004963 0.004963 1.09%
MLPoisson::Fapply() 500 0.004566 0.004566 0.004566 1.00%
FabArray::setDomainBndry() 20 0.003905 0.003905 0.003905 0.86%
MLMG::prepareForSolve() 6 0.003417 0.003417 0.003417 0.75%
MLCellLinOp::solutionResidual() 42 0.003379 0.003379 0.003379 0.74%
Castro::post_timestep() 5 0.002956 0.002956 0.002956 0.65%
MLMG::computeResidual() 36 0.002798 0.002798 0.002798 0.61%
MLCellLinOp::defineBC() 6 0.002728 0.002728 0.002728 0.60%
BndryData::define() 6 0.002619 0.002619 0.002619 0.57%
MultiFab::Xpay() 258 0.002618 0.002618 0.002618 0.57%
Castro::estTimeStep() 10 0.002387 0.002387 0.002387 0.52%
Castro::reset_internal_energy() 30 0.002322 0.002322 0.002322 0.51%
Castro::construct_new_source() 25 0.001703 0.001703 0.001703 0.37%
MLMG::getGradSolution() 6 0.001687 0.001687 0.001687 0.37%
MLCellLinOp::compGrad() 6 0.001684 0.001684 0.001684 0.37%
Castro::construct_new_gravity_source() 5 0.001579 0.001579 0.001579 0.35%
Castro::construct_old_source() 25 0.001382 0.001382 0.001382 0.30%
Castro::construct_old_gravity_source() 5 0.001311 0.001311 0.001311 0.29%
Castro::computeNewDt() 5 0.001311 0.001311 0.001311 0.29%
Castro::normalize_species() 30 0.001293 0.001293 0.001293 0.28%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001199 0.001199 0.001199 0.26%
Castro::apply_source_to_state() 10 0.0009239 0.0009239 0.0009239 0.20%
MultiFab::Saxpy() 10 0.0009177 0.0009177 0.0009177 0.20%
Gravity::swapTimeLevels() 5 0.0008771 0.0008771 0.0008771 0.19%
FabArrayBase::getCPC() 632 0.0008504 0.0008504 0.0008504 0.19%
MLMG::ResNormInf() 42 0.0008344 0.0008344 0.0008344 0.18%
MLCellLinOp::setLevelBC() 6 0.0008062 0.0008062 0.0008062 0.18%
Gravity::update_max_rhs() 6 0.0007539 0.0007539 0.0007539 0.17%
MLPoisson::prepareForSolve() 6 0.0006863 0.0006863 0.0006863 0.15%
MLCellLinOp::prepareForSolve() 6 0.0006819 0.0006819 0.0006819 0.15%
FabArray::mult() 22 0.0006482 0.0006482 0.0006482 0.14%
MLMG::computeMLResidual() 6 0.0005996 0.0005996 0.0005996 0.13%
Castro::enforce_speed_limit() 30 0.0004765 0.0004765 0.0004765 0.10%
FabArrayBase::CPC::define() 244 0.0004233 0.0004233 0.0004233 0.09%
FabArrayBase::getFB() 1766 0.0004009 0.0004009 0.0004009 0.09%
Amr::InitAmr() 1 0.0003988 0.0003988 0.0003988 0.09%
MultiFab::Copy() 6 0.0003591 0.0003591 0.0003591 0.08%
MultiFab::min() 5 0.0001572 0.0001572 0.0001572 0.03%
MLLinOp::define() 6 0.0001541 0.0001541 0.0001541 0.03%
MLMG::MLResNormInf() 6 0.0001343 0.0001343 0.0001343 0.03%
MultiFab::max() 6 0.0001336 0.0001336 0.0001336 0.03%
MLLinOp::defineGrids() 6 0.0001278 0.0001278 0.0001278 0.03%
MLMG::MLRhsNormInf() 6 0.0001055 0.0001055 0.0001055 0.02%
FabArrayBase::FB::FB() 26 6.643e-05 6.643e-05 6.643e-05 0.01%
MLLinOp::makeAgglomeratedDMap 6 3.056e-05 3.056e-05 3.056e-05 0.01%
Castro::finalize_advance() 5 2.853e-05 2.853e-05 2.853e-05 0.01%
Castro::swap_state_time_levels() 5 2.55e-05 2.55e-05 2.55e-05 0.01%
Amr::writeSmallPlotFile() 1 2.313e-05 2.313e-05 2.313e-05 0.01%
makeSFC 30 2.204e-05 2.204e-05 2.204e-05 0.00%
Castro::finalize_do_advance() 5 2.125e-05 2.125e-05 2.125e-05 0.00%
Castro::create_source_corrector() 5 1.764e-05 1.764e-05 1.764e-05 0.00%
Amr::initSubcycle() 1 9.637e-06 9.637e-06 9.637e-06 0.00%
DistributionMapping::Distribute() 31 8.923e-06 8.923e-06 8.923e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.332e-06 4.332e-06 4.332e-06 0.00%
MLMG::buildFineMask() 6 3.462e-06 3.462e-06 3.462e-06 0.00%
Gravity::set_mass_offset() 6 2.283e-06 2.283e-06 2.283e-06 0.00%
Castro::retry_advance_ctu() 5 1.566e-06 1.566e-06 1.566e-06 0.00%
Castro::FluxRegFineAdd() 5 1.533e-06 1.533e-06 1.533e-06 0.00%
AmrLevel::AmrLevel() 1 1.128e-06 1.128e-06 1.128e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.126e-06 1.126e-06 1.126e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 10619
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.02-14-gaa0dc39f50f7) finalized