Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.05-5-gf29ddb5a82b2) initialized
Starting run at 16:21:18 UTC on 2022-05-08.
Successfully read inputs file ...
Castro git describe: 22.05-11-gb985d1be3
AMReX git describe: 22.05-5-gf29ddb5a8
Microphysics git describe: 22.05
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.03960931 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.022851135 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.049232477
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.050786772
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.061164319
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.060185712
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.060340114
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.036194511 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.051419621
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.050105317
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.060521425
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.061468043
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.062449123
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.036368286 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.022713866 seconds
Ending run at 16:21:19 UTC on 2022-05-08.
Run time = 0.778256232
Run time without initialization = 0.663465405
Average number of zones advanced per microsecond: 3.951
Average number of zones advanced per microsecond per rank: 3.951
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.7783 ... 0.7783 ... 0.7783
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.1685 0.1685 0.1685 21.64%
VisMF::Write(FabArray) 11 0.1512 0.1512 0.1512 19.42%
MLCellLinOp::applyBC() 4433 0.08096 0.08096 0.08096 10.40%
MLPoisson::Fsmooth() 3280 0.06463 0.06463 0.06463 8.30%
MLCGSolver::bicgstab 82 0.02448 0.02448 0.02448 3.14%
StateData::FillBoundary(geom) 328 0.02286 0.02286 0.02286 2.94%
MultiFab::Dot() 1114 0.02251 0.02251 0.02251 2.89%
Castro::normalize_species() 62 0.01937 0.01937 0.01937 2.49%
Castro::computeTemp() 63 0.01536 0.01536 0.01536 1.97%
MultiFab::LinComb() 1586 0.01472 0.01472 0.01472 1.89%
FabArray::setVal() 1144 0.01442 0.01442 0.01442 1.85%
FillBoundary_nowait() 4023 0.01427 0.01427 0.01427 1.83%
FabArray::ParallelCopy_nowait() 861 0.01325 0.01325 0.01325 1.70%
Castro::enforce_min_density() 62 0.01278 0.01278 0.01278 1.64%
MLPoisson::Fapply() 1142 0.01196 0.01196 0.01196 1.54%
StateDataPhysBCFunct::() 41 0.01178 0.01178 0.01178 1.51%
MLCellLinOp::defineAuxData() 11 0.01169 0.01169 0.01169 1.50%
Gravity::fill_multipole_BCs() 11 0.008612 0.008612 0.008612 1.11%
MLMG::addInterpCorrection() 410 0.007443 0.007443 0.007443 0.96%
Castro::estTimeStep() 21 0.006991 0.006991 0.006991 0.90%
amrex::average_down 410 0.006966 0.006966 0.006966 0.90%
MultiFab::Xpay() 585 0.006722 0.006722 0.006722 0.86%
Castro::do_advance_ctu() 10 0.00502 0.00502 0.00502 0.65%
Castro::reset_internal_energy(MultiFab) 63 0.004609 0.004609 0.004609 0.59%
Amr::checkPoint() 3 0.004177 0.004177 0.004177 0.54%
BndryData::define() 11 0.003972 0.003972 0.003972 0.51%
Castro::construct_new_gravity_source() 10 0.003183 0.003183 0.003183 0.41%
Castro::construct_old_gravity_source() 10 0.00262 0.00262 0.00262 0.34%
Amr::writePlotFile() 2 0.002449 0.002449 0.002449 0.31%
Castro::enforce_speed_limit() 62 0.002003 0.002003 0.002003 0.26%
MLMG::ResNormInf() 93 0.001977 0.001977 0.001977 0.25%
Gravity::get_new_grav_vector() 11 0.00193 0.00193 0.00193 0.25%
MultiFab::Saxpy() 20 0.001805 0.001805 0.001805 0.23%
Castro::expand_state() 10 0.001725 0.001725 0.001725 0.22%
Gravity::get_old_grav_vector() 10 0.001725 0.001725 0.001725 0.22%
Castro::reset_internal_energy(Fab) 504 0.001702 0.001702 0.001702 0.22%
MLMG::oneIter() 82 0.001698 0.001698 0.001698 0.22%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001669 0.001669 0.001669 0.21%
MLCellLinOp::setLevelBC() 11 0.001548 0.001548 0.001548 0.20%
Gravity::actual_solve_with_mlmg() 11 0.001347 0.001347 0.001347 0.17%
FabArray::mult() 43 0.001331 0.001331 0.001331 0.17%
FabArray::setDomainBndry() 41 0.001304 0.001304 0.001304 0.17%
MLCellLinOp::prepareForSolve() 11 0.001196 0.001196 0.001196 0.15%
Castro::initData() 1 0.00119 0.00119 0.00119 0.15%
MultiFab::contains_nan() 20 0.001168 0.001168 0.001168 0.15%
MLCellLinOp::smooth() 1640 0.001165 0.001165 0.001165 0.15%
MLMG::prepareForSolve() 11 0.001067 0.001067 0.001067 0.14%
MLCellLinOp::compGrad() 11 0.0009099 0.0009099 0.0009099 0.12%
FabArray::FillBoundary() 4023 0.0008606 0.0008606 0.0008606 0.11%
FabArrayBase::getCPC() 1323 0.0008255 0.0008255 0.0008255 0.11%
Castro::subcycle_advance_ctu() 10 0.0007351 0.0007351 0.0007351 0.09%
FabArrayBase::getFB() 4023 0.0007039 0.0007039 0.0007039 0.09%
FabArrayBase::CPC::define() 454 0.000698 0.000698 0.000698 0.09%
MLCellLinOp::apply() 1142 0.0005079 0.0005079 0.0005079 0.07%
CGSolver::sxay() 1586 0.0004263 0.0004263 0.0004263 0.05%
Amr::InitAmr() 1 0.0004234 0.0004234 0.0004234 0.05%
Gravity::update_max_rhs() 11 0.0004101 0.0004101 0.0004101 0.05%
MLLinOp::defineGrids() 11 0.0003929 0.0003929 0.0003929 0.05%
Gravity::solve_for_phi() 10 0.0003693 0.0003693 0.0003693 0.05%
MLMG::mgVcycle() 82 0.0003602 0.0003602 0.0003602 0.05%
MLCGSolver::ParallelAllReduce 1514 0.0003256 0.0003256 0.0003256 0.04%
FillPatchIterator::Initialize 41 0.0002915 0.0002915 0.0002915 0.04%
main() 1 0.0002864 0.0002864 0.0002864 0.04%
MLCellLinOp::defineBC() 11 0.0002859 0.0002859 0.0002859 0.04%
FabArray::ParallelCopy() 861 0.0002811 0.0002811 0.0002811 0.04%
MultiFab::Copy() 11 0.0002661 0.0002661 0.0002661 0.03%
MultiFab::max() 11 0.0002535 0.0002535 0.0002535 0.03%
MLCellLinOp::correctionResidual() 492 0.0002454 0.0002454 0.0002454 0.03%
Amr::coarseTimeStep() 10 0.0002084 0.0002084 0.0002084 0.03%
Castro::construct_new_gravity() 10 0.0002056 0.0002056 0.0002056 0.03%
MLMG::MLRhsNormInf() 11 0.0002044 0.0002044 0.0002044 0.03%
Amr::timeStep() 10 0.0001976 0.0001976 0.0001976 0.03%
MLMG:computeResOfCorrection() 410 0.00013 0.00013 0.00013 0.02%
StateData::checkPoint() 12 0.0001295 0.0001295 0.0001295 0.02%
MLMG::actualBottomSolve() 82 0.0001121 0.0001121 0.0001121 0.01%
MLMG::mgVcycle_down::0 82 9.285e-05 9.285e-05 9.285e-05 0.01%
FabArrayBase::FB::FB() 56 8.711e-05 8.711e-05 8.711e-05 0.01%
Castro::initialize_advance() 10 8.504e-05 8.504e-05 8.504e-05 0.01%
Castro::Castro() 1 8.327e-05 8.327e-05 8.327e-05 0.01%
MLMG::mgVcycle_down::1 82 8.045e-05 8.045e-05 8.045e-05 0.01%
MLMG::mgVcycle_down::2 82 7.892e-05 7.892e-05 7.892e-05 0.01%
Castro::advance() 10 7.76e-05 7.76e-05 7.76e-05 0.01%
MLMG::solve() 11 7.538e-05 7.538e-05 7.538e-05 0.01%
MLMG::mgVcycle_down::3 82 7.519e-05 7.519e-05 7.519e-05 0.01%
MLMG::mgVcycle_down::4 82 7.498e-05 7.498e-05 7.498e-05 0.01%
AmrLevel::checkPoint() 3 7.357e-05 7.357e-05 7.357e-05 0.01%
Castro::clean_state() 62 7.283e-05 7.283e-05 7.283e-05 0.01%
Castro::finalize_advance() 10 7.049e-05 7.049e-05 7.049e-05 0.01%
Castro::initialize_do_advance() 10 6.365e-05 6.365e-05 6.365e-05 0.01%
Castro::construct_new_source() 50 6.138e-05 6.138e-05 6.138e-05 0.01%
MLMG::mgVcycle_up::4 82 5.985e-05 5.985e-05 5.985e-05 0.01%
Castro::post_timestep() 10 5.585e-05 5.585e-05 5.585e-05 0.01%
MLCellLinOp::solutionResidual() 93 5.089e-05 5.089e-05 5.089e-05 0.01%
MLMG::mgVcycle_up::0 82 5.051e-05 5.051e-05 5.051e-05 0.01%
MLMG::mgVcycle_up::1 82 4.754e-05 4.754e-05 4.754e-05 0.01%
MLMG::mgVcycle_up::3 82 4.742e-05 4.742e-05 4.742e-05 0.01%
StateData::define() 4 4.667e-05 4.667e-05 4.667e-05 0.01%
MLMG::mgVcycle_up::2 82 4.565e-05 4.565e-05 4.565e-05 0.01%
Castro::swap_state_time_levels() 10 4.3e-05 4.3e-05 4.3e-05 0.01%
Castro::finalize_do_advance() 10 3.732e-05 3.732e-05 3.732e-05 0.00%
Castro::enforce_consistent_e() 1 3.493e-05 3.493e-05 3.493e-05 0.00%
MLMG::mgVcycle_bottom 82 3.339e-05 3.339e-05 3.339e-05 0.00%
Gravity::actual_multilevel_solve() 1 3.189e-05 3.189e-05 3.189e-05 0.00%
MLMG::computeResidual() 82 3.047e-05 3.047e-05 3.047e-05 0.00%
FillPatchSingleLevel 41 2.877e-05 2.877e-05 2.877e-05 0.00%
Castro::initMFs() 1 2.848e-05 2.848e-05 2.848e-05 0.00%
Amr::writeSmallPlotFile() 1 2.661e-05 2.661e-05 2.661e-05 0.00%
makeSFC 55 2.619e-05 2.619e-05 2.619e-05 0.00%
MLLinOp::define() 11 2.477e-05 2.477e-05 2.477e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.431e-05 2.431e-05 2.431e-05 0.00%
Castro::buildMetrics() 1 2.32e-05 2.32e-05 2.32e-05 0.00%
Amr::FinalizeInit() 1 2.202e-05 2.202e-05 2.202e-05 0.00%
MLPoisson::define() 11 2.111e-05 2.111e-05 2.111e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.847e-05 1.847e-05 1.847e-05 0.00%
Castro::construct_old_source() 50 1.784e-05 1.784e-05 1.784e-05 0.00%
Castro::do_new_sources() 10 1.744e-05 1.744e-05 1.744e-05 0.00%
DistributionMapping::Distribute() 56 1.562e-05 1.562e-05 1.562e-05 0.00%
Amr::defBaseLevel() 1 1.533e-05 1.533e-05 1.533e-05 0.00%
Castro::do_old_sources() 10 1.525e-05 1.525e-05 1.525e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.429e-05 1.429e-05 1.429e-05 0.00%
Castro::check_for_nan() 20 1.262e-05 1.262e-05 1.262e-05 0.00%
Castro::apply_source_to_state() 20 1.229e-05 1.229e-05 1.229e-05 0.00%
Castro::construct_old_gravity() 10 1.143e-05 1.143e-05 1.143e-05 0.00%
Amr::initSubcycle() 1 9.321e-06 9.321e-06 9.321e-06 0.00%
Gravity::swapTimeLevels() 10 9.23e-06 9.23e-06 9.23e-06 0.00%
AmrLevel::AmrLevel(dm) 1 9.2e-06 9.2e-06 9.2e-06 0.00%
MLPoisson::prepareForSolve() 11 8.686e-06 8.686e-06 8.686e-06 0.00%
MLMG::computeMLResidual() 11 6.681e-06 6.681e-06 6.681e-06 0.00%
AmrLevel::checkPointPost() 3 6.561e-06 6.561e-06 6.561e-06 0.00%
Amr::InitializeInit() 1 6.191e-06 6.191e-06 6.191e-06 0.00%
Castro::computeNewDt() 9 6.13e-06 6.13e-06 6.13e-06 0.00%
MLMG::getGradSolution() 11 5.606e-06 5.606e-06 5.606e-06 0.00%
MLMG::buildFineMask() 11 5.588e-06 5.588e-06 5.588e-06 0.00%
MLMG::MLResNormInf() 11 4.588e-06 4.588e-06 4.588e-06 0.00%
Castro::retry_advance_ctu() 10 4.587e-06 4.587e-06 4.587e-06 0.00%
Castro::post_init() 1 4.348e-06 4.348e-06 4.348e-06 0.00%
Gravity::set_mass_offset() 11 4.309e-06 4.309e-06 4.309e-06 0.00%
Castro::create_source_corrector() 10 4.271e-06 4.271e-06 4.271e-06 0.00%
Castro::FluxRegCrseInit 10 3.564e-06 3.564e-06 3.564e-06 0.00%
Castro::computeInitialDt() 2 2.62e-06 2.62e-06 2.62e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.61e-06 2.61e-06 2.61e-06 0.00%
Amr::init() 1 2.461e-06 2.461e-06 2.461e-06 0.00%
Castro::FluxRegFineAdd() 10 2.351e-06 2.351e-06 2.351e-06 0.00%
AmrLevel::checkPointPre() 3 2.258e-06 2.258e-06 2.258e-06 0.00%
MLLinOp::makeSubCommunicator() 11 1.751e-06 1.751e-06 1.751e-06 0.00%
Castro::post_regrid() 1 1.271e-06 1.271e-06 1.271e-06 0.00%
Amr::initialInit() 1 1.077e-06 1.077e-06 1.077e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.7783 0.7783 0.7783 100.00%
Amr::coarseTimeStep() 10 0.6405 0.6405 0.6405 82.30%
Amr::timeStep() 10 0.5637 0.5637 0.5637 72.43%
Castro::advance() 10 0.5547 0.5547 0.5547 71.27%
Castro::subcycle_advance_ctu() 10 0.5415 0.5415 0.5415 69.57%
Castro::do_advance_ctu() 10 0.5407 0.5407 0.5407 69.48%
Gravity::solve_phi_with_mlmg() 11 0.3184 0.3184 0.3184 40.91%
Gravity::actual_solve_with_mlmg() 11 0.3096 0.3096 0.3096 39.77%
Castro::construct_new_gravity() 10 0.2888 0.2888 0.2888 37.11%
MLMG::solve() 11 0.2865 0.2865 0.2865 36.81%
Gravity::solve_for_phi() 10 0.2741 0.2741 0.2741 35.22%
MLMG::oneIter() 82 0.2717 0.2717 0.2717 34.92%
MLMG::mgVcycle() 82 0.2701 0.2701 0.2701 34.70%
Castro::construct_ctu_hydro_source() 10 0.1685 0.1685 0.1685 21.64%
VisMF::Write(FabArray) 11 0.1512 0.1512 0.1512 19.42%
MLCellLinOp::smooth() 1640 0.1375 0.1375 0.1375 17.67%
Amr::init() 1 0.1142 0.1142 0.1142 14.68%
Amr::checkPoint() 3 0.1123 0.1123 0.1123 14.43%
AmrLevel::checkPoint() 3 0.1081 0.1081 0.1081 13.89%
StateData::checkPoint() 12 0.108 0.108 0.108 13.88%
MLCellLinOp::applyBC() 4433 0.09688 0.09688 0.09688 12.45%
MLMG::mgVcycle_bottom 82 0.08377 0.08377 0.08377 10.76%
MLMG::actualBottomSolve() 82 0.08373 0.08373 0.08373 10.76%
MLCGSolver::bicgstab 82 0.08288 0.08288 0.08288 10.65%
MLPoisson::Fsmooth() 3280 0.06463 0.06463 0.06463 8.30%
Castro::clean_state() 62 0.05515 0.05515 0.05515 7.09%
Amr::initialInit() 1 0.05166 0.05166 0.05166 6.64%
Amr::FinalizeInit() 1 0.04775 0.04775 0.04775 6.14%
Castro::post_init() 1 0.04643 0.04643 0.04643 5.97%
Amr::writePlotFile() 2 0.0457 0.0457 0.0457 5.87%
Gravity::multilevel_solve_for_new_phi() 1 0.04467 0.04467 0.04467 5.74%
Gravity::actual_multilevel_solve() 1 0.04465 0.04465 0.04465 5.74%
FillPatchIterator::Initialize 41 0.04024 0.04024 0.04024 5.17%
FillPatchSingleLevel 41 0.03865 0.03865 0.03865 4.97%
MLCellLinOp::apply() 1142 0.03714 0.03714 0.03714 4.77%
MLMG::mgVcycle_down::0 82 0.03553 0.03553 0.03553 4.57%
StateDataPhysBCFunct::() 41 0.03464 0.03464 0.03464 4.45%
MLMG::mgVcycle_up::0 82 0.03046 0.03046 0.03046 3.91%
StateData::FillBoundary(geom) 328 0.02286 0.02286 0.02286 2.94%
MultiFab::Dot() 1114 0.02251 0.02251 0.02251 2.89%
MLCellLinOp::correctionResidual() 492 0.02171 0.02171 0.02171 2.79%
Castro::computeTemp() 63 0.02167 0.02167 0.02167 2.78%
Castro::initialize_do_advance() 10 0.02129 0.02129 0.02129 2.74%
Castro::normalize_species() 62 0.01937 0.01937 0.01937 2.49%
MLPoisson::define() 11 0.01875 0.01875 0.01875 2.41%
MLMG:computeResOfCorrection() 410 0.01871 0.01871 0.01871 2.40%
MLMG::mgVcycle_down::1 82 0.01799 0.01799 0.01799 2.31%
MLMG::mgVcycle_down::2 82 0.01756 0.01756 0.01756 2.26%
MLMG::mgVcycle_down::3 82 0.01664 0.01664 0.01664 2.14%
Gravity::get_new_grav_vector() 11 0.01611 0.01611 0.01611 2.07%
FabArray::FillBoundary() 4023 0.01592 0.01592 0.01592 2.05%
MLMG::mgVcycle_down::4 82 0.01585 0.01585 0.01585 2.04%
CGSolver::sxay() 1586 0.01515 0.01515 0.01515 1.95%
FillBoundary_nowait() 4023 0.01506 0.01506 0.01506 1.94%
MultiFab::LinComb() 1586 0.01472 0.01472 0.01472 1.89%
FabArray::setVal() 1144 0.01442 0.01442 0.01442 1.85%
FabArray::ParallelCopy() 861 0.01442 0.01442 0.01442 1.85%
Castro::construct_old_gravity() 10 0.01419 0.01419 0.01419 1.82%
Gravity::get_old_grav_vector() 10 0.01418 0.01418 0.01418 1.82%
FabArray::ParallelCopy_nowait() 861 0.01413 0.01413 0.01413 1.82%
MLCGSolver::ParallelAllReduce 1514 0.01345 0.01345 0.01345 1.73%
MLMG::mgVcycle_up::2 82 0.01342 0.01342 0.01342 1.72%
MLMG::mgVcycle_up::1 82 0.01321 0.01321 0.01321 1.70%
Castro::initialize_advance() 10 0.0131 0.0131 0.0131 1.68%
MLCellLinOp::defineAuxData() 11 0.01305 0.01305 0.01305 1.68%
Castro::do_new_sources() 10 0.01285 0.01285 0.01285 1.65%
Castro::enforce_min_density() 62 0.01278 0.01278 0.01278 1.64%
MLMG::mgVcycle_up::3 82 0.01274 0.01274 0.01274 1.64%
MLMG::addInterpCorrection() 410 0.01266 0.01266 0.01266 1.63%
MLMG::mgVcycle_up::4 82 0.01252 0.01252 0.01252 1.61%
amrex::average_down 410 0.01218 0.01218 0.01218 1.57%
MLPoisson::Fapply() 1142 0.01196 0.01196 0.01196 1.54%
Castro::do_old_sources() 10 0.01178 0.01178 0.01178 1.51%
Castro::expand_state() 10 0.01159 0.01159 0.01159 1.49%
Castro::post_timestep() 10 0.008793 0.008793 0.008793 1.13%
Gravity::fill_multipole_BCs() 11 0.008612 0.008612 0.008612 1.11%
MLCellLinOp::solutionResidual() 93 0.007216 0.007216 0.007216 0.93%
Castro::estTimeStep() 21 0.006991 0.006991 0.006991 0.90%
MultiFab::Xpay() 585 0.006722 0.006722 0.006722 0.86%
Castro::reset_internal_energy(MultiFab) 63 0.006311 0.006311 0.006311 0.81%
MLMG::computeResidual() 82 0.006214 0.006214 0.006214 0.80%
MLMG::prepareForSolve() 11 0.005257 0.005257 0.005257 0.68%
MLCellLinOp::defineBC() 11 0.005208 0.005208 0.005208 0.67%
BndryData::define() 11 0.004922 0.004922 0.004922 0.63%
Amr::InitializeInit() 1 0.003907 0.003907 0.003907 0.50%
Amr::defBaseLevel() 1 0.003901 0.003901 0.003901 0.50%
Castro::computeNewDt() 9 0.003475 0.003475 0.003475 0.45%
Castro::initData() 1 0.003394 0.003394 0.003394 0.44%
Castro::construct_new_source() 50 0.003245 0.003245 0.003245 0.42%
Castro::construct_new_gravity_source() 10 0.003183 0.003183 0.003183 0.41%
Castro::construct_old_source() 50 0.002638 0.002638 0.002638 0.34%
Castro::construct_old_gravity_source() 10 0.00262 0.00262 0.00262 0.34%
Castro::enforce_speed_limit() 62 0.002003 0.002003 0.002003 0.26%
MLMG::ResNormInf() 93 0.001977 0.001977 0.001977 0.25%
Castro::apply_source_to_state() 20 0.001817 0.001817 0.001817 0.23%
MultiFab::Saxpy() 20 0.001805 0.001805 0.001805 0.23%
Castro::reset_internal_energy(Fab) 504 0.001702 0.001702 0.001702 0.22%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001669 0.001669 0.001669 0.21%
MLCellLinOp::setLevelBC() 11 0.001548 0.001548 0.001548 0.20%
FabArrayBase::getCPC() 1323 0.001523 0.001523 0.001523 0.20%
MLMG::getGradSolution() 11 0.00139 0.00139 0.00139 0.18%
MLCellLinOp::compGrad() 11 0.001384 0.001384 0.001384 0.18%
FabArray::mult() 43 0.001331 0.001331 0.001331 0.17%
FabArray::setDomainBndry() 41 0.001304 0.001304 0.001304 0.17%
MLPoisson::prepareForSolve() 11 0.001205 0.001205 0.001205 0.15%
MLCellLinOp::prepareForSolve() 11 0.001196 0.001196 0.001196 0.15%
Castro::check_for_nan() 20 0.001181 0.001181 0.001181 0.15%
MultiFab::contains_nan() 20 0.001168 0.001168 0.001168 0.15%
Castro::post_regrid() 1 0.001088 0.001088 0.001088 0.14%
MLMG::computeMLResidual() 11 0.00104 0.00104 0.00104 0.13%
Gravity::update_max_rhs() 11 0.0008128 0.0008128 0.0008128 0.10%
FabArrayBase::getFB() 4023 0.000791 0.000791 0.000791 0.10%
FabArrayBase::CPC::define() 454 0.000698 0.000698 0.000698 0.09%
Castro::computeInitialDt() 2 0.0006832 0.0006832 0.0006832 0.09%
MLLinOp::define() 11 0.0004738 0.0004738 0.0004738 0.06%
MLLinOp::defineGrids() 11 0.0004491 0.0004491 0.0004491 0.06%
Gravity::swapTimeLevels() 10 0.0004417 0.0004417 0.0004417 0.06%
Amr::InitAmr() 1 0.0004327 0.0004327 0.0004327 0.06%
Castro::Castro() 1 0.000431 0.000431 0.000431 0.06%
MultiFab::Copy() 11 0.0002661 0.0002661 0.0002661 0.03%
MLMG::MLResNormInf() 11 0.0002602 0.0002602 0.0002602 0.03%
MultiFab::max() 11 0.0002535 0.0002535 0.0002535 0.03%
MLMG::MLRhsNormInf() 11 0.0002044 0.0002044 0.0002044 0.03%
Castro::buildMetrics() 1 0.0001585 0.0001585 0.0001585 0.02%
FabArrayBase::FB::FB() 56 8.711e-05 8.711e-05 8.711e-05 0.01%
Castro::finalize_advance() 10 7.641e-05 7.641e-05 7.641e-05 0.01%
AmrLevel::AmrLevel(dm) 1 5.587e-05 5.587e-05 5.587e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.443e-05 5.443e-05 5.443e-05 0.01%
StateData::define() 4 4.667e-05 4.667e-05 4.667e-05 0.01%
Castro::swap_state_time_levels() 10 4.3e-05 4.3e-05 4.3e-05 0.01%
makeSFC 55 4.014e-05 4.014e-05 4.014e-05 0.01%
Castro::finalize_do_advance() 10 3.732e-05 3.732e-05 3.732e-05 0.00%
Castro::enforce_consistent_e() 1 3.493e-05 3.493e-05 3.493e-05 0.00%
Castro::initMFs() 1 2.848e-05 2.848e-05 2.848e-05 0.00%
Amr::writeSmallPlotFile() 1 2.661e-05 2.661e-05 2.661e-05 0.00%
DistributionMapping::Distribute() 56 1.562e-05 1.562e-05 1.562e-05 0.00%
Amr::initSubcycle() 1 9.321e-06 9.321e-06 9.321e-06 0.00%
AmrLevel::checkPointPost() 3 6.561e-06 6.561e-06 6.561e-06 0.00%
MLMG::buildFineMask() 11 5.588e-06 5.588e-06 5.588e-06 0.00%
Castro::retry_advance_ctu() 10 4.587e-06 4.587e-06 4.587e-06 0.00%
Gravity::set_mass_offset() 11 4.309e-06 4.309e-06 4.309e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.276e-06 4.276e-06 4.276e-06 0.00%
Castro::create_source_corrector() 10 4.271e-06 4.271e-06 4.271e-06 0.00%
Castro::FluxRegCrseInit 10 3.564e-06 3.564e-06 3.564e-06 0.00%
Castro::FluxRegFineAdd() 10 2.351e-06 2.351e-06 2.351e-06 0.00%
AmrLevel::checkPointPre() 3 2.258e-06 2.258e-06 2.258e-06 0.00%
MLLinOp::makeSubCommunicator() 11 1.751e-06 1.751e-06 1.751e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 2465
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.05-5-gf29ddb5a82b2) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.05-5-gf29ddb5a82b2) initialized
Starting run at 16:21:19 UTC on 2022-05-08.
Successfully read inputs file ...
Castro git describe: 22.05-11-gb985d1be3
AMReX git describe: 22.05-5-gf29ddb5a8
Microphysics git describe: 22.05
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.432634618
Restart time = 0.047247674 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.050838138
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.048139911
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.05636857
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.06074774
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.063389025
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.050736578 seconds
Ending run at 16:21:20 UTC on 2022-05-08.
Run time = 0.378368319
Run time without initialization = 0.330575725
Average number of zones advanced per microsecond: 3.965
Average number of zones advanced per microsecond per rank: 3.965
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.3784 ... 0.3784 ... 0.3784
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.0834 0.0834 0.0834 22.04%
VisMF::Read() 3 0.03998 0.03998 0.03998 10.56%
MLCellLinOp::applyBC() 1946 0.03392 0.03392 0.03392 8.96%
Amr::writePlotFile() 1 0.02793 0.02793 0.02793 7.38%
MLPoisson::Fsmooth() 1440 0.0269 0.0269 0.0269 7.11%
VisMF::Write(FabArray) 1 0.02289 0.02289 0.02289 6.05%
StateData::FillBoundary(geom) 160 0.0111 0.0111 0.0111 2.93%
MLCGSolver::bicgstab 36 0.01017 0.01017 0.01017 2.69%
MultiFab::Dot() 484 0.009326 0.009326 0.009326 2.46%
Castro::normalize_species() 30 0.009285 0.009285 0.009285 2.45%
Castro::computeTemp() 30 0.006891 0.006891 0.006891 1.82%
FabArray::setVal() 537 0.006673 0.006673 0.006673 1.76%
Castro::enforce_min_density() 30 0.00628 0.00628 0.00628 1.66%
MLCellLinOp::defineAuxData() 6 0.00618 0.00618 0.00618 1.63%
FillBoundary_nowait() 1766 0.006132 0.006132 0.006132 1.62%
MultiFab::LinComb() 690 0.00606 0.00606 0.00606 1.60%
FabArray::ParallelCopy_nowait() 380 0.005807 0.005807 0.005807 1.53%
StateDataPhysBCFunct::() 20 0.005553 0.005553 0.005553 1.47%
MLPoisson::Fapply() 500 0.005035 0.005035 0.005035 1.33%
Gravity::fill_multipole_BCs() 6 0.00487 0.00487 0.00487 1.29%
Castro::estTimeStep() 10 0.00339 0.00339 0.00339 0.90%
MLMG::addInterpCorrection() 180 0.003151 0.003151 0.003151 0.83%
Amr::restart() 1 0.003 0.003 0.003 0.79%
amrex::average_down 180 0.002965 0.002965 0.002965 0.78%
MultiFab::Xpay() 258 0.002877 0.002877 0.002877 0.76%
Castro::do_advance_ctu() 5 0.002193 0.002193 0.002193 0.58%
Castro::reset_internal_energy(MultiFab) 30 0.002164 0.002164 0.002164 0.57%
BndryData::define() 6 0.002096 0.002096 0.002096 0.55%
Castro::construct_new_gravity_source() 5 0.001593 0.001593 0.001593 0.42%
Castro::construct_old_gravity_source() 5 0.001352 0.001352 0.001352 0.36%
MultiFab::Saxpy() 10 0.0009171 0.0009171 0.0009171 0.24%
Gravity::get_old_grav_vector() 5 0.0008743 0.0008743 0.0008743 0.23%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0008723 0.0008723 0.0008723 0.23%
Castro::expand_state() 5 0.0008658 0.0008658 0.0008658 0.23%
MLMG::ResNormInf() 42 0.0008634 0.0008634 0.0008634 0.23%
Gravity::get_new_grav_vector() 5 0.0008608 0.0008608 0.0008608 0.23%
MLCellLinOp::setLevelBC() 6 0.0008195 0.0008195 0.0008195 0.22%
Castro::reset_internal_energy(Fab) 240 0.0007873 0.0007873 0.0007873 0.21%
MLMG::oneIter() 36 0.0007315 0.0007315 0.0007315 0.19%
Gravity::actual_solve_with_mlmg() 6 0.0007208 0.0007208 0.0007208 0.19%
FabArray::mult() 22 0.000647 0.000647 0.000647 0.17%
MLCellLinOp::prepareForSolve() 6 0.0006411 0.0006411 0.0006411 0.17%
FabArray::setDomainBndry() 20 0.0006341 0.0006341 0.0006341 0.17%
Castro::enforce_speed_limit() 30 0.0006333 0.0006333 0.0006333 0.17%
MultiFab::contains_nan() 10 0.0005793 0.0005793 0.0005793 0.15%
MLMG::prepareForSolve() 6 0.0005697 0.0005697 0.0005697 0.15%
MLCellLinOp::compGrad() 6 0.0004912 0.0004912 0.0004912 0.13%
MLCellLinOp::smooth() 720 0.0004784 0.0004784 0.0004784 0.13%
FabArrayBase::CPC::define() 244 0.0004083 0.0004083 0.0004083 0.11%
FabArrayBase::getCPC() 632 0.0003834 0.0003834 0.0003834 0.10%
Amr::InitAmr() 1 0.0003722 0.0003722 0.0003722 0.10%
FabArray::FillBoundary() 1766 0.0003651 0.0003651 0.0003651 0.10%
FabArrayBase::getFB() 1766 0.0002855 0.0002855 0.0002855 0.08%
main() 1 0.0002734 0.0002734 0.0002734 0.07%
MLCellLinOp::apply() 500 0.0002264 0.0002264 0.0002264 0.06%
Gravity::update_max_rhs() 6 0.000219 0.000219 0.000219 0.06%
CGSolver::sxay() 690 0.0002042 0.0002042 0.0002042 0.05%
Gravity::solve_for_phi() 5 0.0001811 0.0001811 0.0001811 0.05%
MLLinOp::defineGrids() 6 0.0001791 0.0001791 0.0001791 0.05%
MLMG::mgVcycle() 36 0.0001623 0.0001623 0.0001623 0.04%
MLCellLinOp::defineBC() 6 0.0001508 0.0001508 0.0001508 0.04%
Castro::subcycle_advance_ctu() 5 0.0001471 0.0001471 0.0001471 0.04%
MLCGSolver::ParallelAllReduce 659 0.0001456 0.0001456 0.0001456 0.04%
MultiFab::Copy() 6 0.0001379 0.0001379 0.0001379 0.04%
FillPatchIterator::Initialize 20 0.000137 0.000137 0.000137 0.04%
FabArray::ParallelCopy() 380 0.0001351 0.0001351 0.0001351 0.04%
MultiFab::max() 6 0.0001346 0.0001346 0.0001346 0.04%
Amr::coarseTimeStep() 5 0.0001149 0.0001149 0.0001149 0.03%
Castro::construct_new_gravity() 5 0.000106 0.000106 0.000106 0.03%
Amr::timeStep() 5 0.0001057 0.0001057 0.0001057 0.03%
MLMG::MLRhsNormInf() 6 0.0001048 0.0001048 0.0001048 0.03%
MLCellLinOp::correctionResidual() 216 0.0001002 0.0001002 0.0001002 0.03%
StateData::restartDoit() 4 7.487e-05 7.487e-05 7.487e-05 0.02%
AmrLevel::restart() 1 7.12e-05 7.12e-05 7.12e-05 0.02%
Castro::advance() 5 6.999e-05 6.999e-05 6.999e-05 0.02%
Castro::create_source_corrector() 5 6.229e-05 6.229e-05 6.229e-05 0.02%
FabArrayBase::FB::FB() 26 6.01e-05 6.01e-05 6.01e-05 0.02%
MLMG:computeResOfCorrection() 180 5.277e-05 5.277e-05 5.277e-05 0.01%
MLMG::actualBottomSolve() 36 4.777e-05 4.777e-05 4.777e-05 0.01%
Castro::initialize_do_advance() 5 4.533e-05 4.533e-05 4.533e-05 0.01%
Castro::initialize_advance() 5 4.133e-05 4.133e-05 4.133e-05 0.01%
Castro::post_restart() 1 3.783e-05 3.783e-05 3.783e-05 0.01%
Castro::clean_state() 30 3.734e-05 3.734e-05 3.734e-05 0.01%
MLMG::mgVcycle_down::0 36 3.645e-05 3.645e-05 3.645e-05 0.01%
MLMG::mgVcycle_down::1 36 3.627e-05 3.627e-05 3.627e-05 0.01%
Castro::construct_new_source() 25 3.595e-05 3.595e-05 3.595e-05 0.01%
Castro::construct_old_source() 25 3.543e-05 3.543e-05 3.543e-05 0.01%
MLMG::mgVcycle_down::2 36 3.523e-05 3.523e-05 3.523e-05 0.01%
MLMG::mgVcycle_down::4 36 3.415e-05 3.415e-05 3.415e-05 0.01%
MLMG::solve() 6 3.4e-05 3.4e-05 3.4e-05 0.01%
MLMG::mgVcycle_down::3 36 3.291e-05 3.291e-05 3.291e-05 0.01%
Castro::buildMetrics() 1 3.271e-05 3.271e-05 3.271e-05 0.01%
Gravity::actual_multilevel_solve() 1 3.096e-05 3.096e-05 3.096e-05 0.01%
Castro::initMFs() 1 3.09e-05 3.09e-05 3.09e-05 0.01%
Castro::swap_state_time_levels() 5 2.976e-05 2.976e-05 2.976e-05 0.01%
MLMG::mgVcycle_up::4 36 2.749e-05 2.749e-05 2.749e-05 0.01%
Amr::writeSmallPlotFile() 1 2.673e-05 2.673e-05 2.673e-05 0.01%
MLMG::mgVcycle_up::3 36 2.57e-05 2.57e-05 2.57e-05 0.01%
Castro::finalize_advance() 5 2.533e-05 2.533e-05 2.533e-05 0.01%
MLMG::mgVcycle_up::0 36 2.416e-05 2.416e-05 2.416e-05 0.01%
MLMG::mgVcycle_up::2 36 2.253e-05 2.253e-05 2.253e-05 0.01%
MLCellLinOp::solutionResidual() 42 2.245e-05 2.245e-05 2.245e-05 0.01%
MLMG::mgVcycle_up::1 36 2.143e-05 2.143e-05 2.143e-05 0.01%
MLLinOp::define() 6 2.035e-05 2.035e-05 2.035e-05 0.01%
Castro::finalize_do_advance() 5 1.885e-05 1.885e-05 1.885e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.757e-05 1.757e-05 1.757e-05 0.00%
MLMG::mgVcycle_bottom 36 1.66e-05 1.66e-05 1.66e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.491e-05 1.491e-05 1.491e-05 0.00%
MLMG::computeResidual() 36 1.452e-05 1.452e-05 1.452e-05 0.00%
FillPatchSingleLevel 20 1.368e-05 1.368e-05 1.368e-05 0.00%
makeSFC 30 1.309e-05 1.309e-05 1.309e-05 0.00%
MLPoisson::define() 6 1.295e-05 1.295e-05 1.295e-05 0.00%
DistributionMapping::Distribute() 31 9.634e-06 9.634e-06 9.634e-06 0.00%
Castro::do_new_sources() 5 8.83e-06 8.83e-06 8.83e-06 0.00%
Castro::do_old_sources() 5 8.42e-06 8.42e-06 8.42e-06 0.00%
Amr::initSubcycle() 1 8.381e-06 8.381e-06 8.381e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 7.63e-06 7.63e-06 7.63e-06 0.00%
Castro::check_for_nan() 10 6.935e-06 6.935e-06 6.935e-06 0.00%
Castro::apply_source_to_state() 10 5.461e-06 5.461e-06 5.461e-06 0.00%
Castro::construct_old_gravity() 5 5.094e-06 5.094e-06 5.094e-06 0.00%
Castro::post_timestep() 5 4.999e-06 4.999e-06 4.999e-06 0.00%
MLPoisson::prepareForSolve() 6 4.447e-06 4.447e-06 4.447e-06 0.00%
Gravity::swapTimeLevels() 5 4.11e-06 4.11e-06 4.11e-06 0.00%
Castro::computeNewDt() 5 3.676e-06 3.676e-06 3.676e-06 0.00%
MLMG::buildFineMask() 6 3.21e-06 3.21e-06 3.21e-06 0.00%
MLMG::computeMLResidual() 6 3.101e-06 3.101e-06 3.101e-06 0.00%
MLMG::getGradSolution() 6 2.962e-06 2.962e-06 2.962e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.957e-06 2.957e-06 2.957e-06 0.00%
MLMG::MLResNormInf() 6 2.404e-06 2.404e-06 2.404e-06 0.00%
Gravity::set_mass_offset() 6 2.105e-06 2.105e-06 2.105e-06 0.00%
Castro::retry_advance_ctu() 5 2.091e-06 2.091e-06 2.091e-06 0.00%
Castro::FluxRegCrseInit 5 1.831e-06 1.831e-06 1.831e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.278e-06 1.278e-06 1.278e-06 0.00%
Castro::FluxRegFineAdd() 5 1.215e-06 1.215e-06 1.215e-06 0.00%
Amr::init() 1 1.036e-06 1.036e-06 1.036e-06 0.00%
AmrLevel::AmrLevel() 1 1.017e-06 1.017e-06 1.017e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.3784 0.3784 0.3784 100.00%
Amr::coarseTimeStep() 5 0.2796 0.2796 0.2796 73.89%
Amr::timeStep() 5 0.2775 0.2775 0.2775 73.35%
Castro::advance() 5 0.2728 0.2728 0.2728 72.10%
Castro::subcycle_advance_ctu() 5 0.2658 0.2658 0.2658 70.25%
Castro::do_advance_ctu() 5 0.2657 0.2657 0.2657 70.21%
Castro::construct_new_gravity() 5 0.1414 0.1414 0.1414 37.36%
Gravity::solve_phi_with_mlmg() 6 0.1378 0.1378 0.1378 36.41%
Gravity::solve_for_phi() 5 0.1342 0.1342 0.1342 35.47%
Gravity::actual_solve_with_mlmg() 6 0.1328 0.1328 0.1328 35.09%
MLMG::solve() 6 0.1205 0.1205 0.1205 31.86%
MLMG::oneIter() 36 0.1136 0.1136 0.1136 30.01%
MLMG::mgVcycle() 36 0.1128 0.1128 0.1128 29.82%
Castro::construct_ctu_hydro_source() 5 0.0834 0.0834 0.0834 22.04%
MLCellLinOp::smooth() 720 0.05747 0.05747 0.05747 15.19%
Amr::writePlotFile() 1 0.05083 0.05083 0.05083 13.43%
Amr::init() 1 0.04728 0.04728 0.04728 12.50%
Amr::restart() 1 0.04728 0.04728 0.04728 12.50%
MLCellLinOp::applyBC() 1946 0.04076 0.04076 0.04076 10.77%
AmrLevel::restart() 1 0.04018 0.04018 0.04018 10.62%
StateData::restartDoit() 4 0.0401 0.0401 0.0401 10.60%
VisMF::Read() 3 0.03998 0.03998 0.03998 10.56%
MLMG::mgVcycle_bottom 36 0.03476 0.03476 0.03476 9.19%
MLMG::actualBottomSolve() 36 0.03474 0.03474 0.03474 9.18%
MLCGSolver::bicgstab 36 0.03439 0.03439 0.03439 9.09%
MLPoisson::Fsmooth() 1440 0.0269 0.0269 0.0269 7.11%
Castro::clean_state() 30 0.02608 0.02608 0.02608 6.89%
VisMF::Write(FabArray) 1 0.02289 0.02289 0.02289 6.05%
FillPatchIterator::Initialize 20 0.01943 0.01943 0.01943 5.13%
FillPatchSingleLevel 20 0.01865 0.01865 0.01865 4.93%
StateDataPhysBCFunct::() 20 0.01666 0.01666 0.01666 4.40%
MLCellLinOp::apply() 500 0.01567 0.01567 0.01567 4.14%
MLMG::mgVcycle_down::0 36 0.01505 0.01505 0.01505 3.98%
MLMG::mgVcycle_up::0 36 0.01269 0.01269 0.01269 3.35%
StateData::FillBoundary(geom) 160 0.0111 0.0111 0.0111 2.93%
Castro::initialize_do_advance() 5 0.01092 0.01092 0.01092 2.89%
MLPoisson::define() 6 0.009937 0.009937 0.009937 2.63%
Castro::computeTemp() 30 0.009842 0.009842 0.009842 2.60%
MultiFab::Dot() 484 0.009326 0.009326 0.009326 2.46%
Castro::normalize_species() 30 0.009285 0.009285 0.009285 2.45%
MLCellLinOp::correctionResidual() 216 0.009166 0.009166 0.009166 2.42%
MLMG:computeResOfCorrection() 180 0.007906 0.007906 0.007906 2.09%
MLMG::mgVcycle_down::1 36 0.007564 0.007564 0.007564 2.00%
MLMG::mgVcycle_down::2 36 0.007342 0.007342 0.007342 1.94%
Gravity::get_new_grav_vector() 5 0.007076 0.007076 0.007076 1.87%
Castro::construct_old_gravity() 5 0.007056 0.007056 0.007056 1.86%
Gravity::get_old_grav_vector() 5 0.007051 0.007051 0.007051 1.86%
MLMG::mgVcycle_down::3 36 0.00694 0.00694 0.00694 1.83%
MLCellLinOp::defineAuxData() 6 0.006914 0.006914 0.006914 1.83%
Castro::initialize_advance() 5 0.006878 0.006878 0.006878 1.82%
FabArray::FillBoundary() 1766 0.006843 0.006843 0.006843 1.81%
FabArray::setVal() 537 0.006673 0.006673 0.006673 1.76%
MLMG::mgVcycle_down::4 36 0.006655 0.006655 0.006655 1.76%
FillBoundary_nowait() 1766 0.006478 0.006478 0.006478 1.71%
FabArray::ParallelCopy() 380 0.006339 0.006339 0.006339 1.68%
Castro::do_new_sources() 5 0.006326 0.006326 0.006326 1.67%
Castro::enforce_min_density() 30 0.00628 0.00628 0.00628 1.66%
CGSolver::sxay() 690 0.006265 0.006265 0.006265 1.66%
FabArray::ParallelCopy_nowait() 380 0.006204 0.006204 0.006204 1.64%
MultiFab::LinComb() 690 0.00606 0.00606 0.00606 1.60%
Castro::expand_state() 5 0.005946 0.005946 0.005946 1.57%
MLCGSolver::ParallelAllReduce 659 0.005624 0.005624 0.005624 1.49%
MLMG::mgVcycle_up::2 36 0.005598 0.005598 0.005598 1.48%
MLMG::mgVcycle_up::1 36 0.005517 0.005517 0.005517 1.46%
Castro::do_old_sources() 5 0.0054 0.0054 0.0054 1.43%
MLMG::addInterpCorrection() 180 0.005322 0.005322 0.005322 1.41%
MLMG::mgVcycle_up::3 36 0.005308 0.005308 0.005308 1.40%
MLMG::mgVcycle_up::4 36 0.005241 0.005241 0.005241 1.38%
amrex::average_down 180 0.00515 0.00515 0.00515 1.36%
MLPoisson::Fapply() 500 0.005035 0.005035 0.005035 1.33%
Gravity::fill_multipole_BCs() 6 0.00487 0.00487 0.00487 1.29%
Castro::post_timestep() 5 0.004628 0.004628 0.004628 1.22%
Castro::post_restart() 1 0.003919 0.003919 0.003919 1.04%
Gravity::multilevel_solve_for_new_phi() 1 0.003794 0.003794 0.003794 1.00%
Gravity::actual_multilevel_solve() 1 0.003777 0.003777 0.003777 1.00%
Castro::estTimeStep() 10 0.00339 0.00339 0.00339 0.90%
MLCellLinOp::solutionResidual() 42 0.003203 0.003203 0.003203 0.85%
Castro::reset_internal_energy(MultiFab) 30 0.002951 0.002951 0.002951 0.78%
MultiFab::Xpay() 258 0.002877 0.002877 0.002877 0.76%
MLCellLinOp::defineBC() 6 0.00278 0.00278 0.00278 0.73%
MLMG::prepareForSolve() 6 0.002761 0.002761 0.002761 0.73%
MLMG::computeResidual() 36 0.002661 0.002661 0.002661 0.70%
BndryData::define() 6 0.00263 0.00263 0.00263 0.69%
Castro::computeNewDt() 5 0.00193 0.00193 0.00193 0.51%
Castro::construct_new_source() 25 0.001628 0.001628 0.001628 0.43%
Castro::construct_new_gravity_source() 5 0.001593 0.001593 0.001593 0.42%
Castro::construct_old_source() 25 0.001387 0.001387 0.001387 0.37%
Castro::construct_old_gravity_source() 5 0.001352 0.001352 0.001352 0.36%
Castro::apply_source_to_state() 10 0.0009226 0.0009226 0.0009226 0.24%
MultiFab::Saxpy() 10 0.0009171 0.0009171 0.0009171 0.24%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0008723 0.0008723 0.0008723 0.23%
MLMG::ResNormInf() 42 0.0008634 0.0008634 0.0008634 0.23%
MLCellLinOp::setLevelBC() 6 0.0008195 0.0008195 0.0008195 0.22%
FabArrayBase::getCPC() 632 0.0007917 0.0007917 0.0007917 0.21%
Castro::reset_internal_energy(Fab) 240 0.0007873 0.0007873 0.0007873 0.21%
MLMG::getGradSolution() 6 0.0007542 0.0007542 0.0007542 0.20%
MLCellLinOp::compGrad() 6 0.0007512 0.0007512 0.0007512 0.20%
FabArray::mult() 22 0.000647 0.000647 0.000647 0.17%
MLPoisson::prepareForSolve() 6 0.0006455 0.0006455 0.0006455 0.17%
MLCellLinOp::prepareForSolve() 6 0.0006411 0.0006411 0.0006411 0.17%
FabArray::setDomainBndry() 20 0.0006341 0.0006341 0.0006341 0.17%
Castro::enforce_speed_limit() 30 0.0006333 0.0006333 0.0006333 0.17%
Castro::check_for_nan() 10 0.0005862 0.0005862 0.0005862 0.15%
MultiFab::contains_nan() 10 0.0005793 0.0005793 0.0005793 0.15%
MLMG::computeMLResidual() 6 0.0005602 0.0005602 0.0005602 0.15%
Gravity::update_max_rhs() 6 0.0004317 0.0004317 0.0004317 0.11%
FabArrayBase::CPC::define() 244 0.0004083 0.0004083 0.0004083 0.11%
Amr::InitAmr() 1 0.0003806 0.0003806 0.0003806 0.10%
FabArrayBase::getFB() 1766 0.0003456 0.0003456 0.0003456 0.09%
MLLinOp::define() 6 0.0002295 0.0002295 0.0002295 0.06%
Gravity::swapTimeLevels() 5 0.0002234 0.0002234 0.0002234 0.06%
MLLinOp::defineGrids() 6 0.0002092 0.0002092 0.0002092 0.06%
Castro::buildMetrics() 1 0.000153 0.000153 0.000153 0.04%
MultiFab::Copy() 6 0.0001379 0.0001379 0.0001379 0.04%
MLMG::MLResNormInf() 6 0.0001348 0.0001348 0.0001348 0.04%
MultiFab::max() 6 0.0001346 0.0001346 0.0001346 0.04%
MLMG::MLRhsNormInf() 6 0.0001048 0.0001048 0.0001048 0.03%
Castro::create_source_corrector() 5 6.229e-05 6.229e-05 6.229e-05 0.02%
FabArrayBase::FB::FB() 26 6.01e-05 6.01e-05 6.01e-05 0.02%
Castro::initMFs() 1 3.09e-05 3.09e-05 3.09e-05 0.01%
Castro::swap_state_time_levels() 5 2.976e-05 2.976e-05 2.976e-05 0.01%
MLLinOp::makeAgglomeratedDMap 6 2.879e-05 2.879e-05 2.879e-05 0.01%
Castro::finalize_advance() 5 2.838e-05 2.838e-05 2.838e-05 0.01%
Amr::writeSmallPlotFile() 1 2.673e-05 2.673e-05 2.673e-05 0.01%
makeSFC 30 2.116e-05 2.116e-05 2.116e-05 0.01%
Castro::finalize_do_advance() 5 1.885e-05 1.885e-05 1.885e-05 0.00%
DistributionMapping::Distribute() 31 9.634e-06 9.634e-06 9.634e-06 0.00%
Amr::initSubcycle() 1 8.381e-06 8.381e-06 8.381e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.516e-06 4.516e-06 4.516e-06 0.00%
MLMG::buildFineMask() 6 3.21e-06 3.21e-06 3.21e-06 0.00%
Gravity::set_mass_offset() 6 2.105e-06 2.105e-06 2.105e-06 0.00%
Castro::retry_advance_ctu() 5 2.091e-06 2.091e-06 2.091e-06 0.00%
Castro::FluxRegCrseInit 5 1.831e-06 1.831e-06 1.831e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.278e-06 1.278e-06 1.278e-06 0.00%
Castro::FluxRegFineAdd() 5 1.215e-06 1.215e-06 1.215e-06 0.00%
AmrLevel::AmrLevel() 1 1.017e-06 1.017e-06 1.017e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 2465
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.05-5-gf29ddb5a82b2) finalized