Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.09-27-gd65e09e4a85d) initialized
Starting run at 08:36:14 UTC on 2022-09-30.
Successfully read inputs file ...
Castro git describe: 22.09-1-g65b273ad0
AMReX git describe: 22.09-27-gd65e09e4a
Microphysics git describe: 22.09-15-g9bf53bb5
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.053042429 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.030858577 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.050240642
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.053966495
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.050743507
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.086752043
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.071210425
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.049456055 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.069222628
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.068953146
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.049052137
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.051563668
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.056139917
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.079476605 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.030314318 seconds
Ending run at 08:36:15 UTC on 2022-09-30.
Run time = 0.901212002
Run time without initialization = 0.767686731
Average number of zones advanced per microsecond: 3.415
Average number of zones advanced per microsecond per rank: 3.415
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.9012 ... 0.9012 ... 0.9012
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.2181 0.2181 0.2181 24.19%
VisMF::Write(FabArray) 11 0.2056 0.2056 0.2056 22.82%
MLCellLinOp::applyBC() 4379 0.07925 0.07925 0.07925 8.79%
MLPoisson::Fsmooth() 3240 0.06322 0.06322 0.06322 7.01%
Amr::checkPoint() 3 0.03496 0.03496 0.03496 3.88%
StateData::FillBoundary(geom) 328 0.02433 0.02433 0.02433 2.70%
MLCGSolver::bicgstab 81 0.02395 0.02395 0.02395 2.66%
MultiFab::Dot() 1100 0.02217 0.02217 0.02217 2.46%
Castro::normalize_species() 62 0.0148 0.0148 0.0148 1.64%
MultiFab::LinComb() 1566 0.01442 0.01442 0.01442 1.60%
Castro::computeTemp() 63 0.01435 0.01435 0.01435 1.59%
FabArray::setVal() 1135 0.01427 0.01427 0.01427 1.58%
FillBoundary_nowait() 3974 0.01418 0.01418 0.01418 1.57%
FabArray::ParallelCopy_nowait() 851 0.01314 0.01314 0.01314 1.46%
StateDataPhysBCFunct::() 41 0.01273 0.01273 0.01273 1.41%
MLPoisson::Fapply() 1128 0.01165 0.01165 0.01165 1.29%
MLCellLinOp::defineAuxData() 11 0.01164 0.01164 0.01164 1.29%
Gravity::fill_multipole_BCs() 11 0.01031 0.01031 0.01031 1.14%
Castro::enforce_min_density() 62 0.009074 0.009074 0.009074 1.01%
MLMG::addInterpCorrection() 405 0.007603 0.007603 0.007603 0.84%
amrex::average_down 405 0.006775 0.006775 0.006775 0.75%
MultiFab::Xpay() 578 0.00654 0.00654 0.00654 0.73%
Castro::estTimeStep() 21 0.006475 0.006475 0.006475 0.72%
Castro::do_advance_ctu() 10 0.005114 0.005114 0.005114 0.57%
Castro::reset_internal_energy(MultiFab) 63 0.004176 0.004176 0.004176 0.46%
BndryData::define() 11 0.003962 0.003962 0.003962 0.44%
Castro::construct_new_gravity_source() 10 0.003346 0.003346 0.003346 0.37%
Castro::construct_old_gravity_source() 10 0.00274 0.00274 0.00274 0.30%
Amr::writePlotFile() 2 0.002593 0.002593 0.002593 0.29%
MLMG::ResNormInf() 92 0.002057 0.002057 0.002057 0.23%
Gravity::get_new_grav_vector() 11 0.001934 0.001934 0.001934 0.21%
MultiFab::Saxpy() 20 0.001819 0.001819 0.001819 0.20%
Castro::expand_state() 10 0.001742 0.001742 0.001742 0.19%
Gravity::get_old_grav_vector() 10 0.00173 0.00173 0.00173 0.19%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001678 0.001678 0.001678 0.19%
MultiFab::Add() 81 0.001636 0.001636 0.001636 0.18%
MLCellLinOp::setLevelBC() 11 0.001546 0.001546 0.001546 0.17%
Castro::reset_internal_energy(Fab) 504 0.001542 0.001542 0.001542 0.17%
Gravity::actual_solve_with_mlmg() 11 0.001405 0.001405 0.001405 0.16%
FabArray::mult() 43 0.001324 0.001324 0.001324 0.15%
FabArray::setDomainBndry() 41 0.001319 0.001319 0.001319 0.15%
Castro::initData() 1 0.001282 0.001282 0.001282 0.14%
MLMG::prepareForSolve() 11 0.001237 0.001237 0.001237 0.14%
MultiFab::contains_nan() 20 0.001184 0.001184 0.001184 0.13%
MLCellLinOp::prepareForSolve() 11 0.001163 0.001163 0.001163 0.13%
Castro::enforce_speed_limit() 62 0.001112 0.001112 0.001112 0.12%
MLCellLinOp::smooth() 1620 0.001082 0.001082 0.001082 0.12%
MLCellLinOp::compGrad() 11 0.0009154 0.0009154 0.0009154 0.10%
FabArray::FillBoundary() 3974 0.0008402 0.0008402 0.0008402 0.09%
FabArrayBase::getCPC() 1313 0.0007602 0.0007602 0.0007602 0.08%
Castro::subcycle_advance_ctu() 10 0.0007122 0.0007122 0.0007122 0.08%
FabArrayBase::CPC::define() 454 0.0006849 0.0006849 0.0006849 0.08%
FabArrayBase::getFB() 3974 0.0005833 0.0005833 0.0005833 0.06%
Amr::InitAmr() 1 0.0004848 0.0004848 0.0004848 0.05%
MLCellLinOp::apply() 1128 0.000438 0.000438 0.000438 0.05%
Gravity::solve_for_phi() 10 0.0004376 0.0004376 0.0004376 0.05%
Gravity::update_max_rhs() 11 0.0004264 0.0004264 0.0004264 0.05%
CGSolver::sxay() 1566 0.0003364 0.0003364 0.0003364 0.04%
MultiFab::Copy() 11 0.0003261 0.0003261 0.0003261 0.04%
Amr::coarseTimeStep() 10 0.0003155 0.0003155 0.0003155 0.04%
MLCGSolver::ParallelAllReduce 1495 0.0002966 0.0002966 0.0002966 0.03%
FillPatchIterator::Initialize 41 0.0002862 0.0002862 0.0002862 0.03%
MLCellLinOp::defineBC() 11 0.0002776 0.0002776 0.0002776 0.03%
FabArray::ParallelCopy() 851 0.0002755 0.0002755 0.0002755 0.03%
main() 1 0.0002605 0.0002605 0.0002605 0.03%
MultiFab::max() 11 0.0002559 0.0002559 0.0002559 0.03%
MLMG::mgVcycle() 81 0.0002398 0.0002398 0.0002398 0.03%
MLCellLinOp::correctionResidual() 486 0.0002279 0.0002279 0.0002279 0.03%
MLMG::MLRhsNormInf() 11 0.0002143 0.0002143 0.0002143 0.02%
Castro::construct_new_gravity() 10 0.0002039 0.0002039 0.0002039 0.02%
Amr::timeStep() 10 0.0001794 0.0001794 0.0001794 0.02%
Castro::create_source_corrector() 10 0.0001669 0.0001669 0.0001669 0.02%
MLLinOp::defineGrids() 11 0.0001635 0.0001635 0.0001635 0.02%
MLMG:computeResOfCorrection() 405 0.0001474 0.0001474 0.0001474 0.02%
StateData::checkPoint() 12 0.0001277 0.0001277 0.0001277 0.01%
MLMG::mgVcycle_down::0 81 0.000112 0.000112 0.000112 0.01%
MLMG::mgVcycle_down::1 81 9.759e-05 9.759e-05 9.759e-05 0.01%
Castro::Castro() 1 9.583e-05 9.583e-05 9.583e-05 0.01%
MLMG::mgVcycle_down::2 81 9.21e-05 9.21e-05 9.21e-05 0.01%
Castro::initialize_advance() 10 8.705e-05 8.705e-05 8.705e-05 0.01%
MLMG::mgVcycle_down::3 81 8.574e-05 8.574e-05 8.574e-05 0.01%
MLMG::mgVcycle_down::4 81 8.495e-05 8.495e-05 8.495e-05 0.01%
FabArrayBase::FB::FB() 56 8.491e-05 8.491e-05 8.491e-05 0.01%
MLMG::actualBottomSolve() 81 8.411e-05 8.411e-05 8.411e-05 0.01%
Castro::clean_state() 62 7.776e-05 7.776e-05 7.776e-05 0.01%
AmrLevel::checkPoint() 3 7.475e-05 7.475e-05 7.475e-05 0.01%
MLMG::mgVcycle_up::4 81 6.922e-05 6.922e-05 6.922e-05 0.01%
MLMG::solve() 11 6.73e-05 6.73e-05 6.73e-05 0.01%
Castro::construct_new_source() 50 6.599e-05 6.599e-05 6.599e-05 0.01%
MLMG::oneIter() 81 6.484e-05 6.484e-05 6.484e-05 0.01%
Castro::initialize_do_advance() 10 6.404e-05 6.404e-05 6.404e-05 0.01%
Castro::advance() 10 5.979e-05 5.979e-05 5.979e-05 0.01%
MLMG::mgVcycle_up::3 81 5.579e-05 5.579e-05 5.579e-05 0.01%
MLMG::mgVcycle_up::1 81 5.507e-05 5.507e-05 5.507e-05 0.01%
MLMG::mgVcycle_up::0 81 5.421e-05 5.421e-05 5.421e-05 0.01%
MLMG::mgVcycle_up::2 81 5.39e-05 5.39e-05 5.39e-05 0.01%
MLCellLinOp::solutionResidual() 92 5.226e-05 5.226e-05 5.226e-05 0.01%
Castro::finalize_advance() 10 4.764e-05 4.764e-05 4.764e-05 0.01%
StateData::define() 4 4.379e-05 4.379e-05 4.379e-05 0.00%
Castro::swap_state_time_levels() 10 3.833e-05 3.833e-05 3.833e-05 0.00%
MLMG::computeResidual() 81 3.776e-05 3.776e-05 3.776e-05 0.00%
Castro::enforce_consistent_e() 1 3.476e-05 3.476e-05 3.476e-05 0.00%
Gravity::actual_multilevel_solve() 1 3.226e-05 3.226e-05 3.226e-05 0.00%
MLMG::mgVcycle_bottom 81 3.195e-05 3.195e-05 3.195e-05 0.00%
Castro::finalize_do_advance() 10 3.096e-05 3.096e-05 3.096e-05 0.00%
FillPatchSingleLevel 41 2.734e-05 2.734e-05 2.734e-05 0.00%
Castro::initMFs() 1 2.64e-05 2.64e-05 2.64e-05 0.00%
makeSFC 55 2.508e-05 2.508e-05 2.508e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.472e-05 2.472e-05 2.472e-05 0.00%
Amr::writeSmallPlotFile() 1 2.461e-05 2.461e-05 2.461e-05 0.00%
MLLinOp::define() 11 2.363e-05 2.363e-05 2.363e-05 0.00%
Castro::buildMetrics() 1 2.218e-05 2.218e-05 2.218e-05 0.00%
Amr::defBaseLevel() 1 2.086e-05 2.086e-05 2.086e-05 0.00%
Amr::FinalizeInit() 1 1.95e-05 1.95e-05 1.95e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.815e-05 1.815e-05 1.815e-05 0.00%
Castro::construct_old_source() 50 1.776e-05 1.776e-05 1.776e-05 0.00%
MLPoisson::define() 11 1.766e-05 1.766e-05 1.766e-05 0.00%
Castro::do_new_sources() 10 1.712e-05 1.712e-05 1.712e-05 0.00%
Castro::do_old_sources() 10 1.625e-05 1.625e-05 1.625e-05 0.00%
DistributionMapping::Distribute() 56 1.461e-05 1.461e-05 1.461e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.3e-05 1.3e-05 1.3e-05 0.00%
Castro::check_for_nan() 20 1.277e-05 1.277e-05 1.277e-05 0.00%
Castro::apply_source_to_state() 20 1.146e-05 1.146e-05 1.146e-05 0.00%
MLPoisson::prepareForSolve() 11 1.069e-05 1.069e-05 1.069e-05 0.00%
MLMG::computeMLResidual() 11 9.684e-06 9.684e-06 9.684e-06 0.00%
Castro::construct_old_gravity() 10 9.479e-06 9.479e-06 9.479e-06 0.00%
Gravity::swapTimeLevels() 10 9.16e-06 9.16e-06 9.16e-06 0.00%
Castro::post_timestep() 10 8.667e-06 8.667e-06 8.667e-06 0.00%
Amr::initSubcycle() 1 7.941e-06 7.941e-06 7.941e-06 0.00%
AmrLevel::AmrLevel(dm) 1 6.882e-06 6.882e-06 6.882e-06 0.00%
Castro::computeNewDt() 9 6.846e-06 6.846e-06 6.846e-06 0.00%
MLMG::getGradSolution() 11 6.626e-06 6.626e-06 6.626e-06 0.00%
Amr::InitializeInit() 1 5.089e-06 5.089e-06 5.089e-06 0.00%
AmrLevel::checkPointPost() 3 4.54e-06 4.54e-06 4.54e-06 0.00%
Castro::retry_advance_ctu() 10 4.089e-06 4.089e-06 4.089e-06 0.00%
Castro::post_init() 1 3.825e-06 3.825e-06 3.825e-06 0.00%
Gravity::set_mass_offset() 11 3.755e-06 3.755e-06 3.755e-06 0.00%
MLMG::MLResNormInf() 11 3.478e-06 3.478e-06 3.478e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.19e-06 3.19e-06 3.19e-06 0.00%
Amr::init() 1 2.748e-06 2.748e-06 2.748e-06 0.00%
Castro::FluxRegCrseInit 10 2.553e-06 2.553e-06 2.553e-06 0.00%
Castro::computeInitialDt() 2 2.528e-06 2.528e-06 2.528e-06 0.00%
Castro::FluxRegFineAdd() 10 2.29e-06 2.29e-06 2.29e-06 0.00%
AmrLevel::checkPointPre() 3 2.13e-06 2.13e-06 2.13e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.046e-06 2.046e-06 2.046e-06 0.00%
Castro::post_regrid() 1 1.202e-06 1.202e-06 1.202e-06 0.00%
Amr::initialInit() 1 8.77e-07 8.77e-07 8.77e-07 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.9012 0.9012 0.9012 100.00%
Amr::coarseTimeStep() 10 0.7372 0.7372 0.7372 81.79%
Amr::timeStep() 10 0.6035 0.6035 0.6035 66.96%
Castro::advance() 10 0.5963 0.5963 0.5963 66.16%
Castro::subcycle_advance_ctu() 10 0.5855 0.5855 0.5855 64.97%
Castro::do_advance_ctu() 10 0.5848 0.5848 0.5848 64.89%
Gravity::solve_phi_with_mlmg() 11 0.3144 0.3144 0.3144 34.89%
Gravity::actual_solve_with_mlmg() 11 0.3039 0.3039 0.3039 33.72%
Castro::construct_new_gravity() 10 0.289 0.289 0.289 32.07%
MLMG::solve() 11 0.2811 0.2811 0.2811 31.19%
Gravity::solve_for_phi() 10 0.2734 0.2734 0.2734 30.34%
MLMG::oneIter() 81 0.2662 0.2662 0.2662 29.54%
MLMG::mgVcycle() 81 0.2645 0.2645 0.2645 29.35%
Castro::construct_ctu_hydro_source() 10 0.2181 0.2181 0.2181 24.19%
VisMF::Write(FabArray) 11 0.2056 0.2056 0.2056 22.82%
Amr::checkPoint() 3 0.1821 0.1821 0.1821 20.21%
AmrLevel::checkPoint() 3 0.1471 0.1471 0.1471 16.33%
StateData::checkPoint() 12 0.1471 0.1471 0.1471 16.32%
MLCellLinOp::smooth() 1620 0.1349 0.1349 0.1349 14.97%
Amr::init() 1 0.1329 0.1329 0.1329 14.75%
MLCellLinOp::applyBC() 4379 0.09494 0.09494 0.09494 10.53%
MLMG::mgVcycle_bottom 81 0.08167 0.08167 0.08167 9.06%
MLMG::actualBottomSolve() 81 0.08164 0.08164 0.08164 9.06%
MLCGSolver::bicgstab 81 0.08083 0.08083 0.08083 8.97%
MLPoisson::Fsmooth() 3240 0.06322 0.06322 0.06322 7.01%
Amr::writePlotFile() 2 0.06129 0.06129 0.06129 6.80%
Amr::initialInit() 1 0.04891 0.04891 0.04891 5.43%
Amr::FinalizeInit() 1 0.04467 0.04467 0.04467 4.96%
Castro::clean_state() 62 0.04425 0.04425 0.04425 4.91%
Castro::post_init() 1 0.04331 0.04331 0.04331 4.81%
FillPatchIterator::Initialize 41 0.04269 0.04269 0.04269 4.74%
Gravity::multilevel_solve_for_new_phi() 1 0.04149 0.04149 0.04149 4.60%
Gravity::actual_multilevel_solve() 1 0.04147 0.04147 0.04147 4.60%
FillPatchSingleLevel 41 0.04108 0.04108 0.04108 4.56%
StateDataPhysBCFunct::() 41 0.03706 0.03706 0.03706 4.11%
MLCellLinOp::apply() 1128 0.03593 0.03593 0.03593 3.99%
MLMG::mgVcycle_down::0 81 0.03503 0.03503 0.03503 3.89%
MLMG::mgVcycle_up::0 81 0.03 0.03 0.03 3.33%
StateData::FillBoundary(geom) 328 0.02433 0.02433 0.02433 2.70%
MultiFab::Dot() 1100 0.02217 0.02217 0.02217 2.46%
MLCellLinOp::correctionResidual() 486 0.02109 0.02109 0.02109 2.34%
Castro::computeTemp() 63 0.02007 0.02007 0.02007 2.23%
Castro::initialize_do_advance() 10 0.01974 0.01974 0.01974 2.19%
MLPoisson::define() 11 0.01843 0.01843 0.01843 2.04%
MLMG:computeResOfCorrection() 405 0.0182 0.0182 0.0182 2.02%
MLMG::mgVcycle_down::1 81 0.01758 0.01758 0.01758 1.95%
MLMG::mgVcycle_down::2 81 0.01719 0.01719 0.01719 1.91%
Gravity::get_new_grav_vector() 11 0.01711 0.01711 0.01711 1.90%
MLMG::mgVcycle_down::3 81 0.01623 0.01623 0.01623 1.80%
FabArray::FillBoundary() 3974 0.01569 0.01569 0.01569 1.74%
MLMG::mgVcycle_down::4 81 0.01558 0.01558 0.01558 1.73%
Castro::construct_old_gravity() 10 0.01505 0.01505 0.01505 1.67%
Gravity::get_old_grav_vector() 10 0.01504 0.01504 0.01504 1.67%
FillBoundary_nowait() 3974 0.01485 0.01485 0.01485 1.65%
Castro::normalize_species() 62 0.0148 0.0148 0.0148 1.64%
CGSolver::sxay() 1566 0.01475 0.01475 0.01475 1.64%
MultiFab::LinComb() 1566 0.01442 0.01442 0.01442 1.60%
FabArray::setVal() 1135 0.01427 0.01427 0.01427 1.58%
FabArray::ParallelCopy() 851 0.01424 0.01424 0.01424 1.58%
FabArray::ParallelCopy_nowait() 851 0.01397 0.01397 0.01397 1.55%
MLCGSolver::ParallelAllReduce 1495 0.01322 0.01322 0.01322 1.47%
MLMG::mgVcycle_up::2 81 0.01318 0.01318 0.01318 1.46%
MLCellLinOp::defineAuxData() 11 0.01299 0.01299 0.01299 1.44%
MLMG::mgVcycle_up::1 81 0.01295 0.01295 0.01295 1.44%
MLMG::addInterpCorrection() 405 0.01269 0.01269 0.01269 1.41%
MLMG::mgVcycle_up::3 81 0.01252 0.01252 0.01252 1.39%
MLMG::mgVcycle_up::4 81 0.01236 0.01236 0.01236 1.37%
Castro::do_new_sources() 10 0.01209 0.01209 0.01209 1.34%
amrex::average_down 405 0.01193 0.01193 0.01193 1.32%
Castro::expand_state() 10 0.01166 0.01166 0.01166 1.29%
MLPoisson::Fapply() 1128 0.01165 0.01165 0.01165 1.29%
Castro::initialize_advance() 10 0.01066 0.01066 0.01066 1.18%
Gravity::fill_multipole_BCs() 11 0.01031 0.01031 0.01031 1.14%
Castro::do_old_sources() 10 0.01025 0.01025 0.01025 1.14%
Castro::enforce_min_density() 62 0.009074 0.009074 0.009074 1.01%
MLCellLinOp::solutionResidual() 92 0.007049 0.007049 0.007049 0.78%
Castro::post_timestep() 10 0.007007 0.007007 0.007007 0.78%
MultiFab::Xpay() 578 0.00654 0.00654 0.00654 0.73%
Castro::estTimeStep() 21 0.006475 0.006475 0.006475 0.72%
MLMG::computeResidual() 81 0.006071 0.006071 0.006071 0.67%
Castro::reset_internal_energy(MultiFab) 63 0.005718 0.005718 0.005718 0.63%
MLMG::prepareForSolve() 11 0.005394 0.005394 0.005394 0.60%
MLCellLinOp::defineBC() 11 0.005184 0.005184 0.005184 0.58%
BndryData::define() 11 0.004907 0.004907 0.004907 0.54%
Amr::InitializeInit() 1 0.004236 0.004236 0.004236 0.47%
Amr::defBaseLevel() 1 0.004231 0.004231 0.004231 0.47%
Castro::computeNewDt() 9 0.003877 0.003877 0.003877 0.43%
Castro::initData() 1 0.003693 0.003693 0.003693 0.41%
Castro::construct_new_source() 50 0.003412 0.003412 0.003412 0.38%
Castro::construct_new_gravity_source() 10 0.003346 0.003346 0.003346 0.37%
Castro::construct_old_source() 50 0.002757 0.002757 0.002757 0.31%
Castro::construct_old_gravity_source() 10 0.00274 0.00274 0.00274 0.30%
MLMG::ResNormInf() 92 0.002057 0.002057 0.002057 0.23%
Castro::apply_source_to_state() 20 0.00183 0.00183 0.00183 0.20%
MultiFab::Saxpy() 20 0.001819 0.001819 0.001819 0.20%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001678 0.001678 0.001678 0.19%
MultiFab::Add() 81 0.001636 0.001636 0.001636 0.18%
MLCellLinOp::setLevelBC() 11 0.001546 0.001546 0.001546 0.17%
Castro::reset_internal_energy(Fab) 504 0.001542 0.001542 0.001542 0.17%
FabArrayBase::getCPC() 1313 0.001445 0.001445 0.001445 0.16%
MLMG::getGradSolution() 11 0.001423 0.001423 0.001423 0.16%
MLCellLinOp::compGrad() 11 0.001416 0.001416 0.001416 0.16%
FabArray::mult() 43 0.001324 0.001324 0.001324 0.15%
FabArray::setDomainBndry() 41 0.001319 0.001319 0.001319 0.15%
Castro::check_for_nan() 20 0.001196 0.001196 0.001196 0.13%
MultiFab::contains_nan() 20 0.001184 0.001184 0.001184 0.13%
MLPoisson::prepareForSolve() 11 0.001174 0.001174 0.001174 0.13%
MLCellLinOp::prepareForSolve() 11 0.001163 0.001163 0.001163 0.13%
Castro::post_regrid() 1 0.001154 0.001154 0.001154 0.13%
Castro::enforce_speed_limit() 62 0.001112 0.001112 0.001112 0.12%
MLMG::computeMLResidual() 11 0.001025 0.001025 0.001025 0.11%
Gravity::update_max_rhs() 11 0.0008347 0.0008347 0.0008347 0.09%
FabArrayBase::CPC::define() 454 0.0006849 0.0006849 0.0006849 0.08%
Castro::computeInitialDt() 2 0.000669 0.000669 0.000669 0.07%
FabArrayBase::getFB() 3974 0.0006682 0.0006682 0.0006682 0.07%
Amr::InitAmr() 1 0.0004928 0.0004928 0.0004928 0.05%
Castro::Castro() 1 0.0004629 0.0004629 0.0004629 0.05%
Gravity::swapTimeLevels() 10 0.0004415 0.0004415 0.0004415 0.05%
MultiFab::Copy() 11 0.0003261 0.0003261 0.0003261 0.04%
MLMG::MLResNormInf() 11 0.0002766 0.0002766 0.0002766 0.03%
MultiFab::max() 11 0.0002559 0.0002559 0.0002559 0.03%
MLLinOp::define() 11 0.0002409 0.0002409 0.0002409 0.03%
MLLinOp::defineGrids() 11 0.0002172 0.0002172 0.0002172 0.02%
MLMG::MLRhsNormInf() 11 0.0002143 0.0002143 0.0002143 0.02%
Castro::create_source_corrector() 10 0.0001669 0.0001669 0.0001669 0.02%
Castro::buildMetrics() 1 0.0001593 0.0001593 0.0001593 0.02%
FabArrayBase::FB::FB() 56 8.491e-05 8.491e-05 8.491e-05 0.01%
Castro::finalize_advance() 10 5.249e-05 5.249e-05 5.249e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.165e-05 5.165e-05 5.165e-05 0.01%
AmrLevel::AmrLevel(dm) 1 5.068e-05 5.068e-05 5.068e-05 0.01%
StateData::define() 4 4.379e-05 4.379e-05 4.379e-05 0.00%
makeSFC 55 3.865e-05 3.865e-05 3.865e-05 0.00%
Castro::swap_state_time_levels() 10 3.833e-05 3.833e-05 3.833e-05 0.00%
Castro::enforce_consistent_e() 1 3.476e-05 3.476e-05 3.476e-05 0.00%
Castro::finalize_do_advance() 10 3.096e-05 3.096e-05 3.096e-05 0.00%
Castro::initMFs() 1 2.64e-05 2.64e-05 2.64e-05 0.00%
Amr::writeSmallPlotFile() 1 2.461e-05 2.461e-05 2.461e-05 0.00%
DistributionMapping::Distribute() 56 1.461e-05 1.461e-05 1.461e-05 0.00%
Amr::initSubcycle() 1 7.941e-06 7.941e-06 7.941e-06 0.00%
AmrLevel::checkPointPost() 3 4.54e-06 4.54e-06 4.54e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.228e-06 4.228e-06 4.228e-06 0.00%
Castro::retry_advance_ctu() 10 4.089e-06 4.089e-06 4.089e-06 0.00%
Gravity::set_mass_offset() 11 3.755e-06 3.755e-06 3.755e-06 0.00%
Castro::FluxRegCrseInit 10 2.553e-06 2.553e-06 2.553e-06 0.00%
Castro::FluxRegFineAdd() 10 2.29e-06 2.29e-06 2.29e-06 0.00%
AmrLevel::checkPointPre() 3 2.13e-06 2.13e-06 2.13e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.046e-06 2.046e-06 2.046e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 2465
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.09-27-gd65e09e4a85d) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.09-27-gd65e09e4a85d) initialized
Starting run at 08:36:16 UTC on 2022-09-30.
Successfully read inputs file ...
Castro git describe: 22.09-1-g65b273ad0
AMReX git describe: 22.09-27-gd65e09e4a
Microphysics git describe: 22.09-15-g9bf53bb5
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.495909337
Restart time = 0.049298036 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.051269313
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.053968659
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.061926305
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.068431761
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.071587039
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.032295805 seconds
Ending run at 08:36:16 UTC on 2022-09-30.
Run time = 0.389785125
Run time without initialization = 0.339891282
Average number of zones advanced per microsecond: 3.856
Average number of zones advanced per microsecond per rank: 3.856
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.3898 ... 0.3898 ... 0.3898
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.1102 0.1102 0.1102 28.28%
VisMF::Read() 3 0.0413 0.0413 0.0413 10.60%
MLCellLinOp::applyBC() 1946 0.03465 0.03465 0.03465 8.89%
VisMF::Write(FabArray) 1 0.03077 0.03077 0.03077 7.89%
MLPoisson::Fsmooth() 1440 0.02709 0.02709 0.02709 6.95%
StateData::FillBoundary(geom) 160 0.01193 0.01193 0.01193 3.06%
MLCGSolver::bicgstab 36 0.01024 0.01024 0.01024 2.63%
MultiFab::Dot() 484 0.009547 0.009547 0.009547 2.45%
Castro::normalize_species() 30 0.007833 0.007833 0.007833 2.01%
Castro::enforce_min_density() 30 0.007829 0.007829 0.007829 2.01%
FabArray::setVal() 537 0.00673 0.00673 0.00673 1.73%
MLCellLinOp::defineAuxData() 6 0.006254 0.006254 0.006254 1.60%
FillBoundary_nowait() 1766 0.006252 0.006252 0.006252 1.60%
Castro::computeTemp() 30 0.006118 0.006118 0.006118 1.57%
MultiFab::LinComb() 690 0.006116 0.006116 0.006116 1.57%
FabArray::ParallelCopy_nowait() 380 0.005942 0.005942 0.005942 1.52%
MLPoisson::Fapply() 500 0.005046 0.005046 0.005046 1.29%
StateDataPhysBCFunct::() 20 0.004542 0.004542 0.004542 1.17%
Gravity::fill_multipole_BCs() 6 0.004514 0.004514 0.004514 1.16%
Amr::restart() 1 0.003664 0.003664 0.003664 0.94%
MLMG::addInterpCorrection() 180 0.00331 0.00331 0.00331 0.85%
amrex::average_down 180 0.00296 0.00296 0.00296 0.76%
Castro::estTimeStep() 10 0.002857 0.002857 0.002857 0.73%
MultiFab::Xpay() 258 0.002854 0.002854 0.002854 0.73%
Castro::do_advance_ctu() 5 0.002438 0.002438 0.002438 0.63%
BndryData::define() 6 0.002127 0.002127 0.002127 0.55%
Castro::reset_internal_energy(MultiFab) 30 0.001896 0.001896 0.001896 0.49%
Castro::construct_new_gravity_source() 5 0.001803 0.001803 0.001803 0.46%
Amr::writePlotFile() 1 0.001624 0.001624 0.001624 0.42%
Castro::construct_old_gravity_source() 5 0.001483 0.001483 0.001483 0.38%
Gravity::get_old_grav_vector() 5 0.0009271 0.0009271 0.0009271 0.24%
MultiFab::Saxpy() 10 0.0009205 0.0009205 0.0009205 0.24%
MLMG::ResNormInf() 42 0.0009122 0.0009122 0.0009122 0.23%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0008966 0.0008966 0.0008966 0.23%
Castro::expand_state() 5 0.0008736 0.0008736 0.0008736 0.22%
Gravity::get_new_grav_vector() 5 0.0008691 0.0008691 0.0008691 0.22%
MLCellLinOp::setLevelBC() 6 0.0008134 0.0008134 0.0008134 0.21%
Gravity::actual_solve_with_mlmg() 6 0.0008068 0.0008068 0.0008068 0.21%
Castro::reset_internal_energy(Fab) 240 0.0007721 0.0007721 0.0007721 0.20%
MultiFab::Add() 36 0.0007061 0.0007061 0.0007061 0.18%
MLMG::prepareForSolve() 6 0.0006617 0.0006617 0.0006617 0.17%
MLCellLinOp::prepareForSolve() 6 0.0006519 0.0006519 0.0006519 0.17%
FabArray::setDomainBndry() 20 0.0006511 0.0006511 0.0006511 0.17%
FabArray::mult() 22 0.0006496 0.0006496 0.0006496 0.17%
MultiFab::contains_nan() 10 0.0005831 0.0005831 0.0005831 0.15%
Castro::enforce_speed_limit() 30 0.0004955 0.0004955 0.0004955 0.13%
MLCellLinOp::compGrad() 6 0.0004815 0.0004815 0.0004815 0.12%
MLCellLinOp::smooth() 720 0.000467 0.000467 0.000467 0.12%
Amr::InitAmr() 1 0.0004324 0.0004324 0.0004324 0.11%
FabArrayBase::CPC::define() 244 0.0003912 0.0003912 0.0003912 0.10%
FabArrayBase::getCPC() 632 0.0003904 0.0003904 0.0003904 0.10%
FabArray::FillBoundary() 1766 0.0003812 0.0003812 0.0003812 0.10%
main() 1 0.0002574 0.0002574 0.0002574 0.07%
FabArrayBase::getFB() 1766 0.0002569 0.0002569 0.0002569 0.07%
Gravity::update_max_rhs() 6 0.0002529 0.0002529 0.0002529 0.06%
Gravity::solve_for_phi() 5 0.0002063 0.0002063 0.0002063 0.05%
MLCellLinOp::apply() 500 0.0001977 0.0001977 0.0001977 0.05%
MultiFab::Copy() 6 0.0001756 0.0001756 0.0001756 0.05%
CGSolver::sxay() 690 0.0001671 0.0001671 0.0001671 0.04%
Castro::construct_new_gravity() 5 0.000165 0.000165 0.000165 0.04%
Amr::coarseTimeStep() 5 0.0001611 0.0001611 0.0001611 0.04%
Castro::construct_new_source() 25 0.0001526 0.0001526 0.0001526 0.04%
MLCellLinOp::defineBC() 6 0.0001502 0.0001502 0.0001502 0.04%
Castro::create_source_corrector() 5 0.0001461 0.0001461 0.0001461 0.04%
FillPatchIterator::Initialize 20 0.0001392 0.0001392 0.0001392 0.04%
Castro::initialize_advance() 5 0.0001383 0.0001383 0.0001383 0.04%
MultiFab::max() 6 0.0001354 0.0001354 0.0001354 0.03%
MLCGSolver::ParallelAllReduce 659 0.0001284 0.0001284 0.0001284 0.03%
FabArray::ParallelCopy() 380 0.0001231 0.0001231 0.0001231 0.03%
MLMG::MLRhsNormInf() 6 0.0001125 0.0001125 0.0001125 0.03%
MLMG::mgVcycle() 36 0.0001045 0.0001045 0.0001045 0.03%
Castro::initialize_do_advance() 5 0.0001031 0.0001031 0.0001031 0.03%
Castro::post_timestep() 5 0.000103 0.000103 0.000103 0.03%
MLCellLinOp::correctionResidual() 216 9.991e-05 9.991e-05 9.991e-05 0.03%
StateData::restartDoit() 4 9.428e-05 9.428e-05 9.428e-05 0.02%
MLLinOp::defineGrids() 6 8.973e-05 8.973e-05 8.973e-05 0.02%
Amr::timeStep() 5 7.769e-05 7.769e-05 7.769e-05 0.02%
AmrLevel::restart() 1 7.516e-05 7.516e-05 7.516e-05 0.02%
Castro::subcycle_advance_ctu() 5 6.605e-05 6.605e-05 6.605e-05 0.02%
MLMG:computeResOfCorrection() 180 6.551e-05 6.551e-05 6.551e-05 0.02%
Castro::computeNewDt() 5 6.198e-05 6.198e-05 6.198e-05 0.02%
Castro::construct_old_gravity() 5 5.497e-05 5.497e-05 5.497e-05 0.01%
FabArrayBase::FB::FB() 26 5.471e-05 5.471e-05 5.471e-05 0.01%
MLMG::mgVcycle_down::0 36 4.939e-05 4.939e-05 4.939e-05 0.01%
MLMG::mgVcycle_down::1 36 4.382e-05 4.382e-05 4.382e-05 0.01%
MLMG::mgVcycle_down::2 36 4.025e-05 4.025e-05 4.025e-05 0.01%
Castro::clean_state() 30 3.976e-05 3.976e-05 3.976e-05 0.01%
MLMG::mgVcycle_down::4 36 3.863e-05 3.863e-05 3.863e-05 0.01%
MLMG::mgVcycle_down::3 36 3.773e-05 3.773e-05 3.773e-05 0.01%
MLMG::actualBottomSolve() 36 3.683e-05 3.683e-05 3.683e-05 0.01%
MLMG::solve() 6 3.279e-05 3.279e-05 3.279e-05 0.01%
Castro::buildMetrics() 1 3.21e-05 3.21e-05 3.21e-05 0.01%
MLMG::mgVcycle_up::4 36 3.19e-05 3.19e-05 3.19e-05 0.01%
Castro::construct_old_source() 25 3.124e-05 3.124e-05 3.124e-05 0.01%
Castro::post_restart() 1 3.091e-05 3.091e-05 3.091e-05 0.01%
Castro::advance() 5 3.081e-05 3.081e-05 3.081e-05 0.01%
Gravity::actual_multilevel_solve() 1 3.054e-05 3.054e-05 3.054e-05 0.01%
Castro::initMFs() 1 2.899e-05 2.899e-05 2.899e-05 0.01%
MLMG::oneIter() 36 2.854e-05 2.854e-05 2.854e-05 0.01%
Castro::swap_state_time_levels() 5 2.826e-05 2.826e-05 2.826e-05 0.01%
MLMG::mgVcycle_up::0 36 2.661e-05 2.661e-05 2.661e-05 0.01%
Amr::writeSmallPlotFile() 1 2.609e-05 2.609e-05 2.609e-05 0.01%
MLMG::mgVcycle_up::3 36 2.536e-05 2.536e-05 2.536e-05 0.01%
MLMG::mgVcycle_up::2 36 2.425e-05 2.425e-05 2.425e-05 0.01%
Castro::finalize_advance() 5 2.316e-05 2.316e-05 2.316e-05 0.01%
MLCellLinOp::solutionResidual() 42 2.308e-05 2.308e-05 2.308e-05 0.01%
MLMG::mgVcycle_up::1 36 2.282e-05 2.282e-05 2.282e-05 0.01%
MLLinOp::define() 6 2.053e-05 2.053e-05 2.053e-05 0.01%
Castro::finalize_do_advance() 5 1.789e-05 1.789e-05 1.789e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.739e-05 1.739e-05 1.739e-05 0.00%
MLMG::computeResidual() 36 1.726e-05 1.726e-05 1.726e-05 0.00%
MLMG::mgVcycle_bottom 36 1.41e-05 1.41e-05 1.41e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.401e-05 1.401e-05 1.401e-05 0.00%
makeSFC 30 1.368e-05 1.368e-05 1.368e-05 0.00%
FillPatchSingleLevel 20 1.349e-05 1.349e-05 1.349e-05 0.00%
MLPoisson::define() 6 9.82e-06 9.82e-06 9.82e-06 0.00%
Castro::do_new_sources() 5 9.729e-06 9.729e-06 9.729e-06 0.00%
Amr::initSubcycle() 1 9.372e-06 9.372e-06 9.372e-06 0.00%
DistributionMapping::Distribute() 31 8.41e-06 8.41e-06 8.41e-06 0.00%
Castro::do_old_sources() 5 8.39e-06 8.39e-06 8.39e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 7.132e-06 7.132e-06 7.132e-06 0.00%
Castro::check_for_nan() 10 6.632e-06 6.632e-06 6.632e-06 0.00%
Castro::apply_source_to_state() 10 5.661e-06 5.661e-06 5.661e-06 0.00%
MLPoisson::prepareForSolve() 6 5.316e-06 5.316e-06 5.316e-06 0.00%
MLMG::computeMLResidual() 6 5.043e-06 5.043e-06 5.043e-06 0.00%
Gravity::swapTimeLevels() 5 3.947e-06 3.947e-06 3.947e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.211e-06 3.211e-06 3.211e-06 0.00%
MLMG::getGradSolution() 6 3.061e-06 3.061e-06 3.061e-06 0.00%
Gravity::set_mass_offset() 6 2.205e-06 2.205e-06 2.205e-06 0.00%
Castro::retry_advance_ctu() 5 2.121e-06 2.121e-06 2.121e-06 0.00%
MLMG::MLResNormInf() 6 2.032e-06 2.032e-06 2.032e-06 0.00%
Castro::FluxRegCrseInit 5 1.612e-06 1.612e-06 1.612e-06 0.00%
Castro::FluxRegFineAdd() 5 1.223e-06 1.223e-06 1.223e-06 0.00%
Amr::init() 1 1.108e-06 1.108e-06 1.108e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.091e-06 1.091e-06 1.091e-06 0.00%
AmrLevel::AmrLevel() 1 8.5e-07 8.5e-07 8.5e-07 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.3898 0.3898 0.3898 100.00%
Amr::coarseTimeStep() 5 0.3073 0.3073 0.3073 78.84%
Amr::timeStep() 5 0.3052 0.3052 0.3052 78.30%
Castro::advance() 5 0.3003 0.3003 0.3003 77.03%
Castro::subcycle_advance_ctu() 5 0.2936 0.2936 0.2936 75.32%
Castro::do_advance_ctu() 5 0.2935 0.2935 0.2935 75.30%
Castro::construct_new_gravity() 5 0.1435 0.1435 0.1435 36.82%
Gravity::solve_phi_with_mlmg() 6 0.1393 0.1393 0.1393 35.73%
Gravity::solve_for_phi() 5 0.1357 0.1357 0.1357 34.81%
Gravity::actual_solve_with_mlmg() 6 0.1346 0.1346 0.1346 34.54%
MLMG::solve() 6 0.1223 0.1223 0.1223 31.37%
MLMG::oneIter() 36 0.1151 0.1151 0.1151 29.53%
MLMG::mgVcycle() 36 0.1144 0.1144 0.1144 29.34%
Castro::construct_ctu_hydro_source() 5 0.1102 0.1102 0.1102 28.28%
MLCellLinOp::smooth() 720 0.05841 0.05841 0.05841 14.99%
Amr::init() 1 0.04934 0.04934 0.04934 12.66%
Amr::restart() 1 0.04934 0.04934 0.04934 12.66%
MLCellLinOp::applyBC() 1946 0.0416 0.0416 0.0416 10.67%
AmrLevel::restart() 1 0.04153 0.04153 0.04153 10.65%
StateData::restartDoit() 4 0.04145 0.04145 0.04145 10.63%
VisMF::Read() 3 0.0413 0.0413 0.0413 10.60%
MLMG::mgVcycle_bottom 36 0.03507 0.03507 0.03507 9.00%
MLMG::actualBottomSolve() 36 0.03505 0.03505 0.03505 8.99%
MLCGSolver::bicgstab 36 0.0347 0.0347 0.0347 8.90%
Amr::writePlotFile() 1 0.03239 0.03239 0.03239 8.31%
VisMF::Write(FabArray) 1 0.03077 0.03077 0.03077 7.89%
MLPoisson::Fsmooth() 1440 0.02709 0.02709 0.02709 6.95%
Castro::clean_state() 30 0.02498 0.02498 0.02498 6.41%
FillPatchIterator::Initialize 20 0.01927 0.01927 0.01927 4.94%
FillPatchSingleLevel 20 0.01848 0.01848 0.01848 4.74%
StateDataPhysBCFunct::() 20 0.01647 0.01647 0.01647 4.22%
MLCellLinOp::apply() 500 0.01571 0.01571 0.01571 4.03%
MLMG::mgVcycle_down::0 36 0.01529 0.01529 0.01529 3.92%
MLMG::mgVcycle_up::0 36 0.0131 0.0131 0.0131 3.36%
StateData::FillBoundary(geom) 160 0.01193 0.01193 0.01193 3.06%
MLPoisson::define() 6 0.009957 0.009957 0.009957 2.55%
MultiFab::Dot() 484 0.009547 0.009547 0.009547 2.45%
Castro::initialize_do_advance() 5 0.009318 0.009318 0.009318 2.39%
MLCellLinOp::correctionResidual() 216 0.009175 0.009175 0.009175 2.35%
Castro::computeTemp() 30 0.008786 0.008786 0.008786 2.25%
MLMG:computeResOfCorrection() 180 0.007919 0.007919 0.007919 2.03%
Castro::normalize_species() 30 0.007833 0.007833 0.007833 2.01%
Castro::enforce_min_density() 30 0.007829 0.007829 0.007829 2.01%
Gravity::get_new_grav_vector() 5 0.007696 0.007696 0.007696 1.97%
Castro::construct_old_gravity() 5 0.00767 0.00767 0.00767 1.97%
Gravity::get_old_grav_vector() 5 0.007615 0.007615 0.007615 1.95%
MLMG::mgVcycle_down::1 36 0.00761 0.00761 0.00761 1.95%
MLMG::mgVcycle_down::2 36 0.007359 0.007359 0.007359 1.89%
MLMG::mgVcycle_down::3 36 0.007029 0.007029 0.007029 1.80%
MLCellLinOp::defineAuxData() 6 0.006992 0.006992 0.006992 1.79%
FabArray::FillBoundary() 1766 0.006945 0.006945 0.006945 1.78%
FabArray::setVal() 537 0.00673 0.00673 0.00673 1.73%
MLMG::mgVcycle_down::4 36 0.00671 0.00671 0.00671 1.72%
Castro::initialize_advance() 5 0.006632 0.006632 0.006632 1.70%
FillBoundary_nowait() 1766 0.006563 0.006563 0.006563 1.68%
FabArray::ParallelCopy() 380 0.006465 0.006465 0.006465 1.66%
Castro::do_new_sources() 5 0.00645 0.00645 0.00645 1.65%
FabArray::ParallelCopy_nowait() 380 0.006342 0.006342 0.006342 1.63%
CGSolver::sxay() 690 0.006284 0.006284 0.006284 1.61%
MultiFab::LinComb() 690 0.006116 0.006116 0.006116 1.57%
Castro::do_old_sources() 5 0.005811 0.005811 0.005811 1.49%
MLCGSolver::ParallelAllReduce 659 0.005732 0.005732 0.005732 1.47%
MLMG::mgVcycle_up::2 36 0.005687 0.005687 0.005687 1.46%
MLMG::mgVcycle_up::1 36 0.005613 0.005613 0.005613 1.44%
MLMG::addInterpCorrection() 180 0.005537 0.005537 0.005537 1.42%
MLMG::mgVcycle_up::3 36 0.005415 0.005415 0.005415 1.39%
MLMG::mgVcycle_up::4 36 0.005381 0.005381 0.005381 1.38%
amrex::average_down 180 0.005205 0.005205 0.005205 1.34%
MLPoisson::Fapply() 500 0.005046 0.005046 0.005046 1.29%
Castro::post_timestep() 5 0.004855 0.004855 0.004855 1.25%
Castro::expand_state() 5 0.004785 0.004785 0.004785 1.23%
Gravity::fill_multipole_BCs() 6 0.004514 0.004514 0.004514 1.16%
Castro::post_restart() 1 0.003967 0.003967 0.003967 1.02%
Gravity::multilevel_solve_for_new_phi() 1 0.00384 0.00384 0.00384 0.99%
Gravity::actual_multilevel_solve() 1 0.003823 0.003823 0.003823 0.98%
MLCellLinOp::solutionResidual() 42 0.003196 0.003196 0.003196 0.82%
MLMG::prepareForSolve() 6 0.002913 0.002913 0.002913 0.75%
Castro::estTimeStep() 10 0.002857 0.002857 0.002857 0.73%
MultiFab::Xpay() 258 0.002854 0.002854 0.002854 0.73%
MLCellLinOp::defineBC() 6 0.002816 0.002816 0.002816 0.72%
Castro::reset_internal_energy(MultiFab) 30 0.002668 0.002668 0.002668 0.68%
BndryData::define() 6 0.002666 0.002666 0.002666 0.68%
MLMG::computeResidual() 36 0.002646 0.002646 0.002646 0.68%
Castro::computeNewDt() 5 0.001962 0.001962 0.001962 0.50%
Castro::construct_new_source() 25 0.001955 0.001955 0.001955 0.50%
Castro::construct_new_gravity_source() 5 0.001803 0.001803 0.001803 0.46%
Castro::construct_old_source() 25 0.001514 0.001514 0.001514 0.39%
Castro::construct_old_gravity_source() 5 0.001483 0.001483 0.001483 0.38%
Castro::apply_source_to_state() 10 0.0009261 0.0009261 0.0009261 0.24%
MultiFab::Saxpy() 10 0.0009205 0.0009205 0.0009205 0.24%
MLMG::ResNormInf() 42 0.0009122 0.0009122 0.0009122 0.23%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0008966 0.0008966 0.0008966 0.23%
MLCellLinOp::setLevelBC() 6 0.0008134 0.0008134 0.0008134 0.21%
FabArrayBase::getCPC() 632 0.0007815 0.0007815 0.0007815 0.20%
Castro::reset_internal_energy(Fab) 240 0.0007721 0.0007721 0.0007721 0.20%
MLMG::getGradSolution() 6 0.0007583 0.0007583 0.0007583 0.19%
MLCellLinOp::compGrad() 6 0.0007552 0.0007552 0.0007552 0.19%
MultiFab::Add() 36 0.0007061 0.0007061 0.0007061 0.18%
MLPoisson::prepareForSolve() 6 0.0006572 0.0006572 0.0006572 0.17%
MLCellLinOp::prepareForSolve() 6 0.0006519 0.0006519 0.0006519 0.17%
FabArray::setDomainBndry() 20 0.0006511 0.0006511 0.0006511 0.17%
FabArray::mult() 22 0.0006496 0.0006496 0.0006496 0.17%
Castro::check_for_nan() 10 0.0005897 0.0005897 0.0005897 0.15%
MultiFab::contains_nan() 10 0.0005831 0.0005831 0.0005831 0.15%
MLMG::computeMLResidual() 6 0.0005721 0.0005721 0.0005721 0.15%
Castro::enforce_speed_limit() 30 0.0004955 0.0004955 0.0004955 0.13%
Gravity::update_max_rhs() 6 0.0004662 0.0004662 0.0004662 0.12%
Amr::InitAmr() 1 0.0004418 0.0004418 0.0004418 0.11%
FabArrayBase::CPC::define() 244 0.0003912 0.0003912 0.0003912 0.10%
FabArrayBase::getFB() 1766 0.0003116 0.0003116 0.0003116 0.08%
Gravity::swapTimeLevels() 5 0.0002288 0.0002288 0.0002288 0.06%
MultiFab::Copy() 6 0.0001756 0.0001756 0.0001756 0.05%
Castro::buildMetrics() 1 0.0001517 0.0001517 0.0001517 0.04%
MLMG::MLResNormInf() 6 0.0001465 0.0001465 0.0001465 0.04%
Castro::create_source_corrector() 5 0.0001461 0.0001461 0.0001461 0.04%
MLLinOp::define() 6 0.0001391 0.0001391 0.0001391 0.04%
MultiFab::max() 6 0.0001354 0.0001354 0.0001354 0.03%
MLLinOp::defineGrids() 6 0.0001186 0.0001186 0.0001186 0.03%
MLMG::MLRhsNormInf() 6 0.0001125 0.0001125 0.0001125 0.03%
FabArrayBase::FB::FB() 26 5.471e-05 5.471e-05 5.471e-05 0.01%
Castro::initMFs() 1 2.899e-05 2.899e-05 2.899e-05 0.01%
Castro::swap_state_time_levels() 5 2.826e-05 2.826e-05 2.826e-05 0.01%
MLLinOp::makeAgglomeratedDMap 6 2.775e-05 2.775e-05 2.775e-05 0.01%
Amr::writeSmallPlotFile() 1 2.609e-05 2.609e-05 2.609e-05 0.01%
Castro::finalize_advance() 5 2.6e-05 2.6e-05 2.6e-05 0.01%
makeSFC 30 2.062e-05 2.062e-05 2.062e-05 0.01%
Castro::finalize_do_advance() 5 1.789e-05 1.789e-05 1.789e-05 0.00%
Amr::initSubcycle() 1 9.372e-06 9.372e-06 9.372e-06 0.00%
DistributionMapping::Distribute() 31 8.41e-06 8.41e-06 8.41e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.686e-06 4.686e-06 4.686e-06 0.00%
Gravity::set_mass_offset() 6 2.205e-06 2.205e-06 2.205e-06 0.00%
Castro::retry_advance_ctu() 5 2.121e-06 2.121e-06 2.121e-06 0.00%
Castro::FluxRegCrseInit 5 1.612e-06 1.612e-06 1.612e-06 0.00%
Castro::FluxRegFineAdd() 5 1.223e-06 1.223e-06 1.223e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.091e-06 1.091e-06 1.091e-06 0.00%
AmrLevel::AmrLevel() 1 8.5e-07 8.5e-07 8.5e-07 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 2465
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.09-27-gd65e09e4a85d) finalized