Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.12-17-gc2decc83820a) initialized
Starting run at 10:02:00 UTC on 2022-12-20.
Successfully read inputs file ...
Castro git describe: 22.12-5-g3ad2ef0c0
AMReX git describe: 22.12-17-gc2decc838
Microphysics git describe: 22.12-23-gacd17b4d
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.058118715 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.033500323 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.047641316
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.049632341
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.052950521
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.065683237
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.057519116
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.057287179 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.080855526
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.074749642
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.059456544
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.05828829
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.062804106
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.057333255 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.033695494 seconds
Ending run at 10:02:01 UTC on 2022-12-20.
Run time = 0.899700782
Run time without initialization = 0.75858242
Average number of zones advanced per microsecond: 3.456
Average number of zones advanced per microsecond per rank: 3.456
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9481961472
TinyProfiler total time across processes [min...avg...max]: 0.8997 ... 0.8997 ... 0.8997
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.2354 0.2354 0.2354 26.16%
VisMF::Write(FabArray) 11 0.2331 0.2331 0.2331 25.90%
MLCellLinOp::applyBC() 4433 0.07507 0.07507 0.07507 8.34%
MLPoisson::Fsmooth() 3280 0.06037 0.06037 0.06037 6.71%
StateData::FillBoundary(geom) 328 0.02381 0.02381 0.02381 2.65%
StateDataPhysBCFunct::() 41 0.02121 0.02121 0.02121 2.36%
amrex::Dot() 1114 0.02046 0.02046 0.02046 2.27%
amrex::Copy() 1029 0.01483 0.01483 0.01483 1.65%
Castro::computeTemp() 63 0.01445 0.01445 0.01445 1.61%
FabArray::norminf() 743 0.01445 0.01445 0.01445 1.61%
FillBoundary_nowait() 4023 0.01399 0.01399 0.01399 1.55%
Castro::normalize_species() 62 0.01349 0.01349 0.01349 1.50%
FabArray::setVal() 1144 0.01316 0.01316 0.01316 1.46%
FabArray::ParallelCopy_nowait() 861 0.01315 0.01315 0.01315 1.46%
MLPoisson::Fapply() 1142 0.01049 0.01049 0.01049 1.17%
MLCellLinOp::defineAuxData() 11 0.009977 0.009977 0.009977 1.11%
Castro::enforce_min_density() 62 0.009769 0.009769 0.009769 1.09%
FabArray::Saxpy() 813 0.008212 0.008212 0.008212 0.91%
FabArray::Xpay() 821 0.008181 0.008181 0.008181 0.91%
Gravity::fill_multipole_BCs() 11 0.006907 0.006907 0.006907 0.77%
MLMG::addInterpCorrection() 410 0.006573 0.006573 0.006573 0.73%
amrex::average_down 410 0.005914 0.005914 0.005914 0.66%
Castro::estTimeStep() 21 0.005363 0.005363 0.005363 0.60%
FabArray::LinComb() 557 0.004529 0.004529 0.004529 0.50%
Castro::reset_internal_energy(MultiFab) 63 0.004289 0.004289 0.004289 0.48%
Amr::checkPoint() 3 0.004018 0.004018 0.004018 0.45%
BndryData::define() 11 0.003692 0.003692 0.003692 0.41%
Castro::construct_new_gravity_source() 10 0.0033 0.0033 0.0033 0.37%
Castro::do_advance_ctu() 10 0.003181 0.003181 0.003181 0.35%
Castro::construct_old_gravity_source() 10 0.002706 0.002706 0.002706 0.30%
Amr::writePlotFile() 2 0.002403 0.002403 0.002403 0.27%
amrex::Add() 164 0.002395 0.002395 0.002395 0.27%
MLCGSolver::bicgstab 82 0.002094 0.002094 0.002094 0.23%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001667 0.001667 0.001667 0.19%
Castro::reset_internal_energy(Fab) 504 0.00161 0.00161 0.00161 0.18%
Castro::enforce_speed_limit() 62 0.001447 0.001447 0.001447 0.16%
MLCellLinOp::setLevelBC() 11 0.001405 0.001405 0.001405 0.16%
FabArray::mult() 43 0.001321 0.001321 0.001321 0.15%
FabArray::setDomainBndry() 41 0.00131 0.00131 0.00131 0.15%
Gravity::actual_solve_with_mlmg() 11 0.001288 0.001288 0.001288 0.14%
Castro::initData() 1 0.001281 0.001281 0.001281 0.14%
MLCellLinOp::smooth() 1640 0.001182 0.001182 0.001182 0.13%
MultiFab::contains_nan() 20 0.001178 0.001178 0.001178 0.13%
MLCellLinOp::prepareForSolve() 11 0.001124 0.001124 0.001124 0.12%
MLCellLinOp::compGrad() 11 0.0009079 0.0009079 0.0009079 0.10%
Castro::finalize_advance() 10 0.0008824 0.0008824 0.0008824 0.10%
FabArray::FillBoundary() 4023 0.0008329 0.0008329 0.0008329 0.09%
MLMG::prepareForSolve() 11 0.0008277 0.0008277 0.0008277 0.09%
Castro::subcycle_advance_ctu() 10 0.0007663 0.0007663 0.0007663 0.09%
FabArrayBase::getCPC() 1323 0.0007203 0.0007203 0.0007203 0.08%
FabArrayBase::CPC::define() 454 0.0006967 0.0006967 0.0006967 0.08%
Gravity::get_new_grav_vector() 11 0.0006103 0.0006103 0.0006103 0.07%
FabArrayBase::getFB() 4023 0.0006038 0.0006038 0.0006038 0.07%
Gravity::get_old_grav_vector() 10 0.0005303 0.0005303 0.0005303 0.06%
Amr::InitAmr() 1 0.0004969 0.0004969 0.0004969 0.06%
MLCellLinOp::apply() 1142 0.0004734 0.0004734 0.0004734 0.05%
Amr::coarseTimeStep() 10 0.000396 0.000396 0.000396 0.04%
MLMG::mgVcycle() 82 0.000394 0.000394 0.000394 0.04%
MLLinOp::defineGrids() 11 0.0003297 0.0003297 0.0003297 0.04%
main() 1 0.0002796 0.0002796 0.0002796 0.03%
MLCGSolver::ParallelAllReduce 1514 0.0002763 0.0002763 0.0002763 0.03%
MultiFab::max() 11 0.0002575 0.0002575 0.0002575 0.03%
MLCellLinOp::correctionResidual() 492 0.0002381 0.0002381 0.0002381 0.03%
FabArray::ParallelCopy() 861 0.0002325 0.0002325 0.0002325 0.03%
FillPatchIterator::Initialize 41 0.0002058 0.0002058 0.0002058 0.02%
MLCellLinOp::defineBC() 11 0.0001952 0.0001952 0.0001952 0.02%
Amr::timeStep() 10 0.0001536 0.0001536 0.0001536 0.02%
StateData::checkPoint() 12 0.0001335 0.0001335 0.0001335 0.01%
Gravity::update_max_rhs() 11 0.0001156 0.0001156 0.0001156 0.01%
MLMG:computeResOfCorrection() 410 0.0001109 0.0001109 0.0001109 0.01%
Gravity::solve_for_phi() 10 0.0001039 0.0001039 0.0001039 0.01%
MLMG::mgVcycle_down::0 82 9.129e-05 9.129e-05 9.129e-05 0.01%
MLMG::actualBottomSolve() 82 9.071e-05 9.071e-05 9.071e-05 0.01%
FabArrayBase::FB::FB() 56 8.561e-05 8.561e-05 8.561e-05 0.01%
Castro::expand_state() 10 8.303e-05 8.303e-05 8.303e-05 0.01%
MLMG::mgVcycle_down::1 82 8.29e-05 8.29e-05 8.29e-05 0.01%
Castro::clean_state() 62 8.03e-05 8.03e-05 8.03e-05 0.01%
MLMG::mgVcycle_down::2 82 8.008e-05 8.008e-05 8.008e-05 0.01%
AmrLevel::checkPoint() 3 7.859e-05 7.859e-05 7.859e-05 0.01%
Castro::Castro() 1 7.594e-05 7.594e-05 7.594e-05 0.01%
MLMG::mgVcycle_down::3 82 7.357e-05 7.357e-05 7.357e-05 0.01%
MLMG::solve() 11 7.336e-05 7.336e-05 7.336e-05 0.01%
MLMG::mgVcycle_down::4 82 7.266e-05 7.266e-05 7.266e-05 0.01%
Castro::initialize_advance() 10 6.718e-05 6.718e-05 6.718e-05 0.01%
MLMG::mgVcycle_up::4 82 5.919e-05 5.919e-05 5.919e-05 0.01%
MLMG::mgVcycle_up::0 82 5.481e-05 5.481e-05 5.481e-05 0.01%
MLMG::oneIter() 82 5.479e-05 5.479e-05 5.479e-05 0.01%
MLMG::mgVcycle_up::3 82 5.319e-05 5.319e-05 5.319e-05 0.01%
MLMG::mgVcycle_up::1 82 5.315e-05 5.315e-05 5.315e-05 0.01%
MLCellLinOp::solutionResidual() 93 5.251e-05 5.251e-05 5.251e-05 0.01%
MLMG::mgVcycle_up::2 82 5.138e-05 5.138e-05 5.138e-05 0.01%
Castro::initialize_do_advance() 10 4.406e-05 4.406e-05 4.406e-05 0.00%
Castro::advance() 10 4.226e-05 4.226e-05 4.226e-05 0.00%
Castro::swap_state_time_levels() 10 3.682e-05 3.682e-05 3.682e-05 0.00%
Castro::enforce_consistent_e() 1 3.525e-05 3.525e-05 3.525e-05 0.00%
Castro::finalize_do_advance() 10 3.513e-05 3.513e-05 3.513e-05 0.00%
MLMG::ResNormInf() 93 3.447e-05 3.447e-05 3.447e-05 0.00%
MLMG::computeResidual() 82 3.233e-05 3.233e-05 3.233e-05 0.00%
StateData::define() 4 3.221e-05 3.221e-05 3.221e-05 0.00%
MLMG::mgVcycle_bottom 82 3.113e-05 3.113e-05 3.113e-05 0.00%
FillPatchSingleLevel 41 3.021e-05 3.021e-05 3.021e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.821e-05 2.821e-05 2.821e-05 0.00%
makeSFC 55 2.641e-05 2.641e-05 2.641e-05 0.00%
Amr::writeSmallPlotFile() 1 2.583e-05 2.583e-05 2.583e-05 0.00%
Castro::construct_new_gravity() 10 2.404e-05 2.404e-05 2.404e-05 0.00%
MLPoisson::define() 11 2.095e-05 2.095e-05 2.095e-05 0.00%
Amr::FinalizeInit() 1 2.032e-05 2.032e-05 2.032e-05 0.00%
Castro::initMFs() 1 1.973e-05 1.973e-05 1.973e-05 0.00%
Castro::construct_new_source() 50 1.794e-05 1.794e-05 1.794e-05 0.00%
Castro::do_new_sources() 10 1.759e-05 1.759e-05 1.759e-05 0.00%
Castro::construct_old_source() 50 1.68e-05 1.68e-05 1.68e-05 0.00%
DistributionMapping::Distribute() 56 1.664e-05 1.664e-05 1.664e-05 0.00%
Castro::buildMetrics() 1 1.546e-05 1.546e-05 1.546e-05 0.00%
Castro::do_old_sources() 10 1.539e-05 1.539e-05 1.539e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.49e-05 1.49e-05 1.49e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.427e-05 1.427e-05 1.427e-05 0.00%
Amr::defBaseLevel() 1 1.426e-05 1.426e-05 1.426e-05 0.00%
Castro::check_for_nan() 20 1.123e-05 1.123e-05 1.123e-05 0.00%
Castro::post_timestep() 10 1.101e-05 1.101e-05 1.101e-05 0.00%
MLLinOp::define() 11 1.099e-05 1.099e-05 1.099e-05 0.00%
Castro::apply_source_to_state() 20 1.067e-05 1.067e-05 1.067e-05 0.00%
Castro::post_init() 1 1.014e-05 1.014e-05 1.014e-05 0.00%
Gravity::swapTimeLevels() 10 9.692e-06 9.692e-06 9.692e-06 0.00%
MLMG::getGradSolution() 11 9.686e-06 9.686e-06 9.686e-06 0.00%
Castro::construct_old_gravity() 10 9.614e-06 9.614e-06 9.614e-06 0.00%
Amr::initSubcycle() 1 8.717e-06 8.717e-06 8.717e-06 0.00%
AmrLevel::AmrLevel(dm) 1 8.359e-06 8.359e-06 8.359e-06 0.00%
MLPoisson::prepareForSolve() 11 8.254e-06 8.254e-06 8.254e-06 0.00%
Castro::computeNewDt() 9 7.416e-06 7.416e-06 7.416e-06 0.00%
MLMG::computeMLResidual() 11 7.156e-06 7.156e-06 7.156e-06 0.00%
Gravity::actual_multilevel_solve() 1 6.808e-06 6.808e-06 6.808e-06 0.00%
Castro::retry_advance_ctu() 10 5.005e-06 5.005e-06 5.005e-06 0.00%
Amr::InitializeInit() 1 4.712e-06 4.712e-06 4.712e-06 0.00%
Gravity::set_mass_offset() 11 4.394e-06 4.394e-06 4.394e-06 0.00%
MLMG::MLRhsNormInf() 11 4.174e-06 4.174e-06 4.174e-06 0.00%
AmrLevel::checkPointPost() 3 3.993e-06 3.993e-06 3.993e-06 0.00%
Castro::create_source_corrector() 10 3.942e-06 3.942e-06 3.942e-06 0.00%
Castro::FluxRegCrseInit 10 3.743e-06 3.743e-06 3.743e-06 0.00%
MLMG::MLResNormInf() 11 3.522e-06 3.522e-06 3.522e-06 0.00%
Castro::FluxRegFineAdd() 10 2.631e-06 2.631e-06 2.631e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.578e-06 2.578e-06 2.578e-06 0.00%
Castro::computeInitialDt() 2 2.575e-06 2.575e-06 2.575e-06 0.00%
Amr::init() 1 2.207e-06 2.207e-06 2.207e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.143e-06 2.143e-06 2.143e-06 0.00%
AmrLevel::checkPointPre() 3 1.843e-06 1.843e-06 1.843e-06 0.00%
Castro::post_regrid() 1 1.269e-06 1.269e-06 1.269e-06 0.00%
Amr::initialInit() 1 1.071e-06 1.071e-06 1.071e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.8997 0.8997 0.8997 100.00%
Amr::coarseTimeStep() 10 0.7247 0.7247 0.7247 80.54%
Amr::timeStep() 10 0.606 0.606 0.606 67.36%
Castro::advance() 10 0.5995 0.5995 0.5995 66.63%
Castro::subcycle_advance_ctu() 10 0.587 0.587 0.587 65.25%
Castro::do_advance_ctu() 10 0.5863 0.5863 0.5863 65.16%
Gravity::solve_phi_with_mlmg() 11 0.291 0.291 0.291 32.34%
Gravity::actual_solve_with_mlmg() 11 0.2836 0.2836 0.2836 31.52%
Castro::construct_new_gravity() 10 0.2661 0.2661 0.2661 29.58%
MLMG::solve() 11 0.263 0.263 0.263 29.23%
Gravity::solve_for_phi() 10 0.25 0.25 0.25 27.79%
MLMG::oneIter() 82 0.2488 0.2488 0.2488 27.65%
MLMG::mgVcycle() 82 0.2471 0.2471 0.2471 27.46%
Castro::construct_ctu_hydro_source() 10 0.2354 0.2354 0.2354 26.16%
VisMF::Write(FabArray) 11 0.2331 0.2331 0.2331 25.90%
Amr::checkPoint() 3 0.1729 0.1729 0.1729 19.22%
AmrLevel::checkPoint() 3 0.1689 0.1689 0.1689 18.77%
StateData::checkPoint() 12 0.1688 0.1688 0.1688 18.76%
Amr::init() 1 0.1405 0.1405 0.1405 15.61%
MLCellLinOp::smooth() 1640 0.129 0.129 0.129 14.33%
MLCellLinOp::applyBC() 4433 0.09057 0.09057 0.09057 10.07%
MLMG::mgVcycle_bottom 82 0.07365 0.07365 0.07365 8.19%
MLMG::actualBottomSolve() 82 0.07362 0.07362 0.07362 8.18%
MLCGSolver::bicgstab 82 0.07292 0.07292 0.07292 8.10%
Amr::writePlotFile() 2 0.06732 0.06732 0.06732 7.48%
MLPoisson::Fsmooth() 3280 0.06037 0.06037 0.06037 6.71%
FillPatchIterator::Initialize 41 0.05057 0.05057 0.05057 5.62%
FillPatchSingleLevel 41 0.04905 0.04905 0.04905 5.45%
Amr::initialInit() 1 0.04873 0.04873 0.04873 5.42%
StateDataPhysBCFunct::() 41 0.04502 0.04502 0.04502 5.00%
Amr::FinalizeInit() 1 0.04459 0.04459 0.04459 4.96%
Castro::clean_state() 62 0.04435 0.04435 0.04435 4.93%
Castro::post_init() 1 0.04329 0.04329 0.04329 4.81%
Gravity::multilevel_solve_for_new_phi() 1 0.04142 0.04142 0.04142 4.60%
Gravity::actual_multilevel_solve() 1 0.0414 0.0414 0.0414 4.60%
MLMG::mgVcycle_down::0 82 0.03493 0.03493 0.03493 3.88%
MLCellLinOp::apply() 1142 0.03365 0.03365 0.03365 3.74%
MLMG::mgVcycle_up::0 82 0.02988 0.02988 0.02988 3.32%
Castro::initialize_do_advance() 10 0.02708 0.02708 0.02708 3.01%
StateData::FillBoundary(geom) 328 0.02381 0.02381 0.02381 2.65%
amrex::Dot() 1114 0.02046 0.02046 0.02046 2.27%
Castro::computeTemp() 63 0.02035 0.02035 0.02035 2.26%
MLCellLinOp::correctionResidual() 492 0.01973 0.01973 0.01973 2.19%
Castro::expand_state() 10 0.0188 0.0188 0.0188 2.09%
Gravity::get_new_grav_vector() 11 0.0177 0.0177 0.0177 1.97%
MLMG:computeResOfCorrection() 410 0.01711 0.01711 0.01711 1.90%
MLPoisson::define() 11 0.01657 0.01657 0.01657 1.84%
MLMG::mgVcycle_down::1 82 0.0163 0.0163 0.0163 1.81%
MLMG::mgVcycle_down::2 82 0.01572 0.01572 0.01572 1.75%
FabArray::FillBoundary() 4023 0.01551 0.01551 0.01551 1.72%
Castro::construct_old_gravity() 10 0.01513 0.01513 0.01513 1.68%
Gravity::get_old_grav_vector() 10 0.01512 0.01512 0.01512 1.68%
MLMG::mgVcycle_down::3 82 0.01496 0.01496 0.01496 1.66%
amrex::Copy() 1029 0.01483 0.01483 0.01483 1.65%
FillBoundary_nowait() 4023 0.01468 0.01468 0.01468 1.63%
FabArray::norminf() 743 0.01445 0.01445 0.01445 1.61%
FabArray::ParallelCopy() 861 0.01419 0.01419 0.01419 1.58%
MLMG::mgVcycle_down::4 82 0.01407 0.01407 0.01407 1.56%
FabArray::ParallelCopy_nowait() 861 0.01396 0.01396 0.01396 1.55%
Castro::normalize_species() 62 0.01349 0.01349 0.01349 1.50%
FabArray::setVal() 1144 0.01316 0.01316 0.01316 1.46%
MLCGSolver::ParallelAllReduce 1514 0.01225 0.01225 0.01225 1.36%
MLMG::mgVcycle_up::2 82 0.01221 0.01221 0.01221 1.36%
MLMG::mgVcycle_up::1 82 0.01196 0.01196 0.01196 1.33%
Castro::do_new_sources() 10 0.01194 0.01194 0.01194 1.33%
MLMG::addInterpCorrection() 410 0.01168 0.01168 0.01168 1.30%
MLMG::mgVcycle_up::3 82 0.01153 0.01153 0.01153 1.28%
MLMG::mgVcycle_up::4 82 0.0115 0.0115 0.0115 1.28%
MLCellLinOp::defineAuxData() 11 0.01132 0.01132 0.01132 1.26%
Castro::initialize_advance() 10 0.01101 0.01101 0.01101 1.22%
amrex::average_down 410 0.011 0.011 0.011 1.22%
Castro::do_old_sources() 10 0.01091 0.01091 0.01091 1.21%
MLPoisson::Fapply() 1142 0.01049 0.01049 0.01049 1.17%
Castro::enforce_min_density() 62 0.009769 0.009769 0.009769 1.09%
FabArray::Saxpy() 813 0.008212 0.008212 0.008212 0.91%
FabArray::Xpay() 821 0.008181 0.008181 0.008181 0.91%
Gravity::fill_multipole_BCs() 11 0.007141 0.007141 0.007141 0.79%
MLCellLinOp::solutionResidual() 93 0.007141 0.007141 0.007141 0.79%
Castro::post_timestep() 10 0.006379 0.006379 0.006379 0.71%
MLMG::computeResidual() 82 0.006174 0.006174 0.006174 0.69%
Castro::reset_internal_energy(MultiFab) 63 0.005899 0.005899 0.005899 0.66%
Castro::estTimeStep() 21 0.005363 0.005363 0.005363 0.60%
MLCellLinOp::defineBC() 11 0.004826 0.004826 0.004826 0.54%
BndryData::define() 11 0.004631 0.004631 0.004631 0.51%
MLMG::prepareForSolve() 11 0.004619 0.004619 0.004619 0.51%
FabArray::LinComb() 557 0.004529 0.004529 0.004529 0.50%
Amr::InitializeInit() 1 0.004136 0.004136 0.004136 0.46%
Amr::defBaseLevel() 1 0.004131 0.004131 0.004131 0.46%
Castro::initData() 1 0.003661 0.003661 0.003661 0.41%
Castro::construct_new_source() 50 0.003318 0.003318 0.003318 0.37%
Castro::construct_new_gravity_source() 10 0.0033 0.0033 0.0033 0.37%
Castro::computeNewDt() 9 0.002769 0.002769 0.002769 0.31%
Castro::construct_old_source() 50 0.002723 0.002723 0.002723 0.30%
Castro::construct_old_gravity_source() 10 0.002706 0.002706 0.002706 0.30%
amrex::Add() 164 0.002395 0.002395 0.002395 0.27%
MLMG::ResNormInf() 93 0.002118 0.002118 0.002118 0.24%
Castro::apply_source_to_state() 20 0.001819 0.001819 0.001819 0.20%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001667 0.001667 0.001667 0.19%
Castro::reset_internal_energy(Fab) 504 0.00161 0.00161 0.00161 0.18%
Castro::enforce_speed_limit() 62 0.001447 0.001447 0.001447 0.16%
Castro::finalize_advance() 10 0.001425 0.001425 0.001425 0.16%
FabArrayBase::getCPC() 1323 0.001417 0.001417 0.001417 0.16%
MLCellLinOp::setLevelBC() 11 0.001405 0.001405 0.001405 0.16%
MLMG::getGradSolution() 11 0.001381 0.001381 0.001381 0.15%
MLCellLinOp::compGrad() 11 0.001371 0.001371 0.001371 0.15%
FabArray::mult() 43 0.001321 0.001321 0.001321 0.15%
FabArray::setDomainBndry() 41 0.00131 0.00131 0.00131 0.15%
Castro::check_for_nan() 20 0.001189 0.001189 0.001189 0.13%
MultiFab::contains_nan() 20 0.001178 0.001178 0.001178 0.13%
MLPoisson::prepareForSolve() 11 0.001132 0.001132 0.001132 0.13%
MLCellLinOp::prepareForSolve() 11 0.001124 0.001124 0.001124 0.12%
Castro::post_regrid() 1 0.001117 0.001117 0.001117 0.12%
MLMG::computeMLResidual() 11 0.001006 0.001006 0.001006 0.11%
Castro::computeInitialDt() 2 0.0009164 0.0009164 0.0009164 0.10%
Gravity::update_max_rhs() 11 0.0008124 0.0008124 0.0008124 0.09%
FabArrayBase::CPC::define() 454 0.0006967 0.0006967 0.0006967 0.08%
FabArrayBase::getFB() 4023 0.0006894 0.0006894 0.0006894 0.08%
Amr::InitAmr() 1 0.0005057 0.0005057 0.0005057 0.06%
Gravity::swapTimeLevels() 10 0.0004395 0.0004395 0.0004395 0.05%
Castro::Castro() 1 0.0004111 0.0004111 0.0004111 0.05%
MLLinOp::define() 11 0.0003991 0.0003991 0.0003991 0.04%
MLLinOp::defineGrids() 11 0.0003881 0.0003881 0.0003881 0.04%
MLMG::MLResNormInf() 11 0.0002825 0.0002825 0.0002825 0.03%
MultiFab::max() 11 0.0002575 0.0002575 0.0002575 0.03%
MLMG::MLRhsNormInf() 11 0.0002252 0.0002252 0.0002252 0.03%
Castro::buildMetrics() 1 0.0001545 0.0001545 0.0001545 0.02%
FabArrayBase::FB::FB() 56 8.561e-05 8.561e-05 8.561e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.624e-05 5.624e-05 5.624e-05 0.01%
makeSFC 55 4.134e-05 4.134e-05 4.134e-05 0.00%
AmrLevel::AmrLevel(dm) 1 4.057e-05 4.057e-05 4.057e-05 0.00%
Castro::swap_state_time_levels() 10 3.682e-05 3.682e-05 3.682e-05 0.00%
Castro::enforce_consistent_e() 1 3.525e-05 3.525e-05 3.525e-05 0.00%
Castro::finalize_do_advance() 10 3.513e-05 3.513e-05 3.513e-05 0.00%
StateData::define() 4 3.221e-05 3.221e-05 3.221e-05 0.00%
Amr::writeSmallPlotFile() 1 2.583e-05 2.583e-05 2.583e-05 0.00%
Castro::initMFs() 1 1.973e-05 1.973e-05 1.973e-05 0.00%
DistributionMapping::Distribute() 56 1.664e-05 1.664e-05 1.664e-05 0.00%
Amr::initSubcycle() 1 8.717e-06 8.717e-06 8.717e-06 0.00%
Castro::retry_advance_ctu() 10 5.005e-06 5.005e-06 5.005e-06 0.00%
Gravity::set_mass_offset() 11 4.394e-06 4.394e-06 4.394e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.285e-06 4.285e-06 4.285e-06 0.00%
AmrLevel::checkPointPost() 3 3.993e-06 3.993e-06 3.993e-06 0.00%
Castro::create_source_corrector() 10 3.942e-06 3.942e-06 3.942e-06 0.00%
Castro::FluxRegCrseInit 10 3.743e-06 3.743e-06 3.743e-06 0.00%
Castro::FluxRegFineAdd() 10 2.631e-06 2.631e-06 2.631e-06 0.00%
MLLinOp::makeSubCommunicator() 11 2.143e-06 2.143e-06 2.143e-06 0.00%
AmrLevel::checkPointPre() 3 1.843e-06 1.843e-06 1.843e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12056
Free GPU global memory (MB): 2459
[The Arena] space allocated (MB): 9042
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.12-17-gc2decc83820a) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.12-17-gc2decc83820a) initialized
Starting run at 10:02:02 UTC on 2022-12-20.
Successfully read inputs file ...
Castro git describe: 22.12-5-g3ad2ef0c0
AMReX git describe: 22.12-17-gc2decc838
Microphysics git describe: 22.12-23-gacd17b4d
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.471838414
Restart time = 0.058876081 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.049823011
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.046605111
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.058612289
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.056340391
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.060694566
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.033058501 seconds
Ending run at 10:02:02 UTC on 2022-12-20.
Run time = 0.36509355
Run time without initialization = 0.305584849
Average number of zones advanced per microsecond: 4.289
Average number of zones advanced per microsecond per rank: 4.289
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9481961472
TinyProfiler total time across processes [min...avg...max]: 0.3651 ... 0.3651 ... 0.3651
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 5 0.0933 0.0933 0.0933 25.56%
VisMF::Read() 3 0.04049 0.04049 0.04049 11.09%
MLCellLinOp::applyBC() 1946 0.0324 0.0324 0.0324 8.87%
VisMF::Write(FabArray) 1 0.03152 0.03152 0.03152 8.63%
MLPoisson::Fsmooth() 1440 0.02576 0.02576 0.02576 7.06%
Amr::restart() 1 0.01469 0.01469 0.01469 4.02%
StateData::FillBoundary(geom) 160 0.01117 0.01117 0.01117 3.06%
amrex::Dot() 484 0.008691 0.008691 0.008691 2.38%
amrex::Copy() 463 0.006907 0.006907 0.006907 1.89%
FabArray::setVal() 537 0.006239 0.006239 0.006239 1.71%
FabArray::norminf() 326 0.006169 0.006169 0.006169 1.69%
Castro::normalize_species() 30 0.006113 0.006113 0.006113 1.67%
FillBoundary_nowait() 1766 0.006104 0.006104 0.006104 1.67%
FabArray::ParallelCopy_nowait() 380 0.005956 0.005956 0.005956 1.63%
StateDataPhysBCFunct::() 20 0.005853 0.005853 0.005853 1.60%
Castro::computeTemp() 30 0.005669 0.005669 0.005669 1.55%
MLCellLinOp::defineAuxData() 6 0.005366 0.005366 0.005366 1.47%
MLPoisson::Fapply() 500 0.004494 0.004494 0.004494 1.23%
FabArray::Saxpy() 355 0.003651 0.003651 0.003651 1.00%
FabArray::Xpay() 361 0.00355 0.00355 0.00355 0.97%
Castro::enforce_min_density() 30 0.003218 0.003218 0.003218 0.88%
Gravity::fill_multipole_BCs() 6 0.002987 0.002987 0.002987 0.82%
MLMG::addInterpCorrection() 180 0.002789 0.002789 0.002789 0.76%
amrex::average_down 180 0.002533 0.002533 0.002533 0.69%
Castro::estTimeStep() 10 0.002169 0.002169 0.002169 0.59%
BndryData::define() 6 0.00202 0.00202 0.00202 0.55%
FabArray::LinComb() 242 0.001913 0.001913 0.001913 0.52%
Castro::reset_internal_energy(MultiFab) 30 0.001814 0.001814 0.001814 0.50%
Castro::construct_new_gravity_source() 5 0.001613 0.001613 0.001613 0.44%
Castro::reset_internal_energy(Fab) 240 0.001433 0.001433 0.001433 0.39%
Amr::writePlotFile() 1 0.001385 0.001385 0.001385 0.38%
Castro::do_advance_ctu() 5 0.001314 0.001314 0.001314 0.36%
Castro::construct_old_gravity_source() 5 0.001269 0.001269 0.001269 0.35%
amrex::Add() 72 0.001036 0.001036 0.001036 0.28%
MLCGSolver::bicgstab 36 0.0009542 0.0009542 0.0009542 0.26%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0008869 0.0008869 0.0008869 0.24%
Castro::enforce_speed_limit() 30 0.0007427 0.0007427 0.0007427 0.20%
MLCellLinOp::setLevelBC() 6 0.0007316 0.0007316 0.0007316 0.20%
Gravity::actual_solve_with_mlmg() 6 0.0006907 0.0006907 0.0006907 0.19%
FabArray::mult() 22 0.0006529 0.0006529 0.0006529 0.18%
FabArray::setDomainBndry() 20 0.0006399 0.0006399 0.0006399 0.18%
MLCellLinOp::prepareForSolve() 6 0.0006003 0.0006003 0.0006003 0.16%
MultiFab::contains_nan() 10 0.0005857 0.0005857 0.0005857 0.16%
MLCellLinOp::compGrad() 6 0.0004927 0.0004927 0.0004927 0.13%
MLCellLinOp::smooth() 720 0.0004777 0.0004777 0.0004777 0.13%
MLMG::prepareForSolve() 6 0.0004589 0.0004589 0.0004589 0.13%
Amr::InitAmr() 1 0.000429 0.000429 0.000429 0.12%
FabArrayBase::CPC::define() 244 0.0003987 0.0003987 0.0003987 0.11%
FabArrayBase::getCPC() 632 0.0003508 0.0003508 0.0003508 0.10%
FabArray::FillBoundary() 1766 0.0003492 0.0003492 0.0003492 0.10%
main() 1 0.0002967 0.0002967 0.0002967 0.08%
Gravity::get_old_grav_vector() 5 0.0002886 0.0002886 0.0002886 0.08%
Gravity::get_new_grav_vector() 5 0.0002681 0.0002681 0.0002681 0.07%
FabArrayBase::getFB() 1766 0.0002478 0.0002478 0.0002478 0.07%
MLCellLinOp::apply() 500 0.0002053 0.0002053 0.0002053 0.06%
Amr::coarseTimeStep() 5 0.0002008 0.0002008 0.0002008 0.05%
MLMG::mgVcycle() 36 0.0001713 0.0001713 0.0001713 0.05%
MultiFab::max() 6 0.0001355 0.0001355 0.0001355 0.04%
MLCGSolver::ParallelAllReduce 659 0.0001213 0.0001213 0.0001213 0.03%
MLCellLinOp::defineBC() 6 0.0001063 0.0001063 0.0001063 0.03%
FabArray::ParallelCopy() 380 0.0001062 0.0001062 0.0001062 0.03%
FillPatchIterator::Initialize 20 0.0001033 0.0001033 0.0001033 0.03%
MLCellLinOp::correctionResidual() 216 9.825e-05 9.825e-05 9.825e-05 0.03%
MLLinOp::defineGrids() 6 8.885e-05 8.885e-05 8.885e-05 0.02%
Gravity::solve_for_phi() 5 8.883e-05 8.883e-05 8.883e-05 0.02%
AmrLevel::restart() 1 8.657e-05 8.657e-05 8.657e-05 0.02%
Amr::timeStep() 5 7.999e-05 7.999e-05 7.999e-05 0.02%
Castro::subcycle_advance_ctu() 5 7.085e-05 7.085e-05 7.085e-05 0.02%
FabArrayBase::FB::FB() 26 6.16e-05 6.16e-05 6.16e-05 0.02%
Castro::finalize_advance() 5 6.121e-05 6.121e-05 6.121e-05 0.02%
Gravity::update_max_rhs() 6 6.097e-05 6.097e-05 6.097e-05 0.02%
StateData::restartDoit() 4 5.798e-05 5.798e-05 5.798e-05 0.02%
MLMG:computeResOfCorrection() 180 5.088e-05 5.088e-05 5.088e-05 0.01%
MLMG::mgVcycle_down::0 36 4.224e-05 4.224e-05 4.224e-05 0.01%
MLMG::actualBottomSolve() 36 4.02e-05 4.02e-05 4.02e-05 0.01%
Castro::clean_state() 30 4e-05 4e-05 4e-05 0.01%
MLMG::mgVcycle_down::1 36 3.843e-05 3.843e-05 3.843e-05 0.01%
Castro::expand_state() 5 3.777e-05 3.777e-05 3.777e-05 0.01%
MLMG::solve() 6 3.517e-05 3.517e-05 3.517e-05 0.01%
MLMG::mgVcycle_down::2 36 3.506e-05 3.506e-05 3.506e-05 0.01%
MLMG::mgVcycle_down::4 36 3.502e-05 3.502e-05 3.502e-05 0.01%
MLMG::mgVcycle_down::3 36 3.369e-05 3.369e-05 3.369e-05 0.01%
Castro::advance() 5 3.297e-05 3.297e-05 3.297e-05 0.01%
Castro::initialize_advance() 5 3.225e-05 3.225e-05 3.225e-05 0.01%
MLMG::mgVcycle_up::4 36 2.878e-05 2.878e-05 2.878e-05 0.01%
Amr::writeSmallPlotFile() 1 2.594e-05 2.594e-05 2.594e-05 0.01%
MLMG::mgVcycle_up::0 36 2.491e-05 2.491e-05 2.491e-05 0.01%
MLMG::oneIter() 36 2.485e-05 2.485e-05 2.485e-05 0.01%
Castro::buildMetrics() 1 2.461e-05 2.461e-05 2.461e-05 0.01%
MLMG::mgVcycle_up::3 36 2.343e-05 2.343e-05 2.343e-05 0.01%
MLMG::mgVcycle_up::2 36 2.34e-05 2.34e-05 2.34e-05 0.01%
Castro::initialize_do_advance() 5 2.33e-05 2.33e-05 2.33e-05 0.01%
Castro::post_restart() 1 2.291e-05 2.291e-05 2.291e-05 0.01%
MLCellLinOp::solutionResidual() 42 2.253e-05 2.253e-05 2.253e-05 0.01%
MLMG::mgVcycle_up::1 36 2.204e-05 2.204e-05 2.204e-05 0.01%
Castro::swap_state_time_levels() 5 2.145e-05 2.145e-05 2.145e-05 0.01%
Castro::initMFs() 1 1.953e-05 1.953e-05 1.953e-05 0.01%
Castro::finalize_do_advance() 5 1.877e-05 1.877e-05 1.877e-05 0.01%
MLMG::ResNormInf() 42 1.697e-05 1.697e-05 1.697e-05 0.00%
Castro::construct_new_gravity() 5 1.515e-05 1.515e-05 1.515e-05 0.00%
FillPatchSingleLevel 20 1.467e-05 1.467e-05 1.467e-05 0.00%
MLMG::mgVcycle_bottom 36 1.446e-05 1.446e-05 1.446e-05 0.00%
MLMG::computeResidual() 36 1.426e-05 1.426e-05 1.426e-05 0.00%
makeSFC 30 1.405e-05 1.405e-05 1.405e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.393e-05 1.393e-05 1.393e-05 0.00%
MLPoisson::define() 6 1.393e-05 1.393e-05 1.393e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 1.256e-05 1.256e-05 1.256e-05 0.00%
Castro::construct_new_source() 25 1.102e-05 1.102e-05 1.102e-05 0.00%
Castro::construct_old_source() 25 1.006e-05 1.006e-05 1.006e-05 0.00%
DistributionMapping::Distribute() 31 9.948e-06 9.948e-06 9.948e-06 0.00%
Amr::initSubcycle() 1 9.013e-06 9.013e-06 9.013e-06 0.00%
Castro::do_new_sources() 5 8.724e-06 8.724e-06 8.724e-06 0.00%
Castro::do_old_sources() 5 8.027e-06 8.027e-06 8.027e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 7.187e-06 7.187e-06 7.187e-06 0.00%
Gravity::actual_multilevel_solve() 1 6.732e-06 6.732e-06 6.732e-06 0.00%
Castro::check_for_nan() 10 6.272e-06 6.272e-06 6.272e-06 0.00%
Castro::apply_source_to_state() 10 5.937e-06 5.937e-06 5.937e-06 0.00%
Castro::post_timestep() 5 5.808e-06 5.808e-06 5.808e-06 0.00%
Castro::construct_old_gravity() 5 5.486e-06 5.486e-06 5.486e-06 0.00%
MLLinOp::define() 6 5.159e-06 5.159e-06 5.159e-06 0.00%
MLPoisson::prepareForSolve() 6 4.586e-06 4.586e-06 4.586e-06 0.00%
Gravity::swapTimeLevels() 5 4.34e-06 4.34e-06 4.34e-06 0.00%
Castro::computeNewDt() 5 3.996e-06 3.996e-06 3.996e-06 0.00%
MLMG::computeMLResidual() 6 3.543e-06 3.543e-06 3.543e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.2e-06 3.2e-06 3.2e-06 0.00%
MLMG::getGradSolution() 6 2.937e-06 2.937e-06 2.937e-06 0.00%
Gravity::set_mass_offset() 6 2.635e-06 2.635e-06 2.635e-06 0.00%
Castro::create_source_corrector() 5 2.155e-06 2.155e-06 2.155e-06 0.00%
MLMG::MLResNormInf() 6 2.078e-06 2.078e-06 2.078e-06 0.00%
MLMG::MLRhsNormInf() 6 2.024e-06 2.024e-06 2.024e-06 0.00%
Castro::retry_advance_ctu() 5 1.996e-06 1.996e-06 1.996e-06 0.00%
Castro::FluxRegCrseInit 5 1.435e-06 1.435e-06 1.435e-06 0.00%
AmrLevel::AmrLevel() 1 1.274e-06 1.274e-06 1.274e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.273e-06 1.273e-06 1.273e-06 0.00%
Castro::FluxRegFineAdd() 5 1.261e-06 1.261e-06 1.261e-06 0.00%
Amr::init() 1 9.93e-07 9.93e-07 9.93e-07 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.3651 0.3651 0.3651 100.00%
Amr::coarseTimeStep() 5 0.2723 0.2723 0.2723 74.57%
Amr::timeStep() 5 0.2707 0.2707 0.2707 74.13%
Castro::advance() 5 0.2668 0.2668 0.2668 73.07%
Castro::subcycle_advance_ctu() 5 0.2604 0.2604 0.2604 71.31%
Castro::do_advance_ctu() 5 0.2603 0.2603 0.2603 71.29%
Castro::construct_new_gravity() 5 0.1324 0.1324 0.1324 36.25%
Gravity::solve_phi_with_mlmg() 6 0.1277 0.1277 0.1277 34.98%
Gravity::solve_for_phi() 5 0.1247 0.1247 0.1247 34.16%
Gravity::actual_solve_with_mlmg() 6 0.1245 0.1245 0.1245 34.09%
MLMG::solve() 6 0.1134 0.1134 0.1134 31.06%
MLMG::oneIter() 36 0.1066 0.1066 0.1066 29.21%
MLMG::mgVcycle() 36 0.1059 0.1059 0.1059 29.01%
Castro::construct_ctu_hydro_source() 5 0.09331 0.09331 0.09331 25.56%
Amr::init() 1 0.05892 0.05892 0.05892 16.14%
Amr::restart() 1 0.05892 0.05892 0.05892 16.14%
MLCellLinOp::smooth() 720 0.05529 0.05529 0.05529 15.14%
AmrLevel::restart() 1 0.04069 0.04069 0.04069 11.14%
StateData::restartDoit() 4 0.0406 0.0406 0.0406 11.12%
VisMF::Read() 3 0.04049 0.04049 0.04049 11.09%
MLCellLinOp::applyBC() 1946 0.03916 0.03916 0.03916 10.73%
Amr::writePlotFile() 1 0.03316 0.03316 0.03316 9.08%
VisMF::Write(FabArray) 1 0.03152 0.03152 0.03152 8.63%
MLMG::mgVcycle_bottom 36 0.03142 0.03142 0.03142 8.61%
MLMG::actualBottomSolve() 36 0.03141 0.03141 0.03141 8.60%
MLCGSolver::bicgstab 36 0.0311 0.0311 0.0311 8.52%
MLPoisson::Fsmooth() 1440 0.02576 0.02576 0.02576 7.06%
FillPatchIterator::Initialize 20 0.01978 0.01978 0.01978 5.42%
FillPatchSingleLevel 20 0.01904 0.01904 0.01904 5.21%
Castro::clean_state() 30 0.01903 0.01903 0.01903 5.21%
StateDataPhysBCFunct::() 20 0.01703 0.01703 0.01703 4.66%
MLMG::mgVcycle_down::0 36 0.01505 0.01505 0.01505 4.12%
MLCellLinOp::apply() 500 0.01455 0.01455 0.01455 3.99%
MLMG::mgVcycle_up::0 36 0.01295 0.01295 0.01295 3.55%
StateData::FillBoundary(geom) 160 0.01117 0.01117 0.01117 3.06%
Castro::initialize_do_advance() 5 0.009015 0.009015 0.009015 2.47%
Castro::computeTemp() 30 0.008916 0.008916 0.008916 2.44%
MLPoisson::define() 6 0.008897 0.008897 0.008897 2.44%
amrex::Dot() 484 0.008691 0.008691 0.008691 2.38%
MLCellLinOp::correctionResidual() 216 0.008517 0.008517 0.008517 2.33%
Gravity::get_new_grav_vector() 5 0.007514 0.007514 0.007514 2.06%
MLMG:computeResOfCorrection() 180 0.007383 0.007383 0.007383 2.02%
Castro::construct_old_gravity() 5 0.007169 0.007169 0.007169 1.96%
Gravity::get_old_grav_vector() 5 0.007163 0.007163 0.007163 1.96%
MLMG::mgVcycle_down::1 36 0.006989 0.006989 0.006989 1.91%
amrex::Copy() 463 0.006907 0.006907 0.006907 1.89%
MLMG::mgVcycle_down::2 36 0.006778 0.006778 0.006778 1.86%
FabArray::FillBoundary() 1766 0.006763 0.006763 0.006763 1.85%
FabArray::ParallelCopy() 380 0.006433 0.006433 0.006433 1.76%
FillBoundary_nowait() 1766 0.006414 0.006414 0.006414 1.76%
MLMG::mgVcycle_down::3 36 0.006393 0.006393 0.006393 1.75%
FabArray::ParallelCopy_nowait() 380 0.006327 0.006327 0.006327 1.73%
FabArray::setVal() 537 0.006239 0.006239 0.006239 1.71%
FabArray::norminf() 326 0.006169 0.006169 0.006169 1.69%
Castro::normalize_species() 30 0.006113 0.006113 0.006113 1.67%
MLCellLinOp::defineAuxData() 6 0.006098 0.006098 0.006098 1.67%
Castro::initialize_advance() 5 0.006066 0.006066 0.006066 1.66%
MLMG::mgVcycle_down::4 36 0.006032 0.006032 0.006032 1.65%
Castro::expand_state() 5 0.005609 0.005609 0.005609 1.54%
Castro::do_new_sources() 5 0.005549 0.005549 0.005549 1.52%
MLCGSolver::ParallelAllReduce 659 0.005235 0.005235 0.005235 1.43%
MLMG::mgVcycle_up::2 36 0.005205 0.005205 0.005205 1.43%
MLMG::mgVcycle_up::1 36 0.005142 0.005142 0.005142 1.41%
MLMG::addInterpCorrection() 180 0.005006 0.005006 0.005006 1.37%
MLMG::mgVcycle_up::3 36 0.004898 0.004898 0.004898 1.34%
MLMG::mgVcycle_up::4 36 0.00488 0.00488 0.00488 1.34%
amrex::average_down 180 0.004754 0.004754 0.004754 1.30%
MLPoisson::Fapply() 500 0.004494 0.004494 0.004494 1.23%
Castro::do_old_sources() 5 0.0043 0.0043 0.0043 1.18%
Castro::post_timestep() 5 0.003806 0.003806 0.003806 1.04%
FabArray::Saxpy() 355 0.003651 0.003651 0.003651 1.00%
FabArray::Xpay() 361 0.00355 0.00355 0.00355 0.97%
Castro::post_restart() 1 0.003371 0.003371 0.003371 0.92%
Gravity::multilevel_solve_for_new_phi() 1 0.003255 0.003255 0.003255 0.89%
Castro::reset_internal_energy(MultiFab) 30 0.003247 0.003247 0.003247 0.89%
Gravity::actual_multilevel_solve() 1 0.003241 0.003241 0.003241 0.89%
Castro::enforce_min_density() 30 0.003218 0.003218 0.003218 0.88%
MLCellLinOp::solutionResidual() 42 0.003194 0.003194 0.003194 0.87%
Gravity::fill_multipole_BCs() 6 0.00311 0.00311 0.00311 0.85%
MLCellLinOp::defineBC() 6 0.002661 0.002661 0.002661 0.73%
MLMG::computeResidual() 36 0.002647 0.002647 0.002647 0.72%
BndryData::define() 6 0.002554 0.002554 0.002554 0.70%
MLMG::prepareForSolve() 6 0.002475 0.002475 0.002475 0.68%
Castro::estTimeStep() 10 0.002169 0.002169 0.002169 0.59%
FabArray::LinComb() 242 0.001913 0.001913 0.001913 0.52%
Castro::construct_new_source() 25 0.001624 0.001624 0.001624 0.44%
Castro::construct_new_gravity_source() 5 0.001613 0.001613 0.001613 0.44%
Castro::reset_internal_energy(Fab) 240 0.001433 0.001433 0.001433 0.39%
Castro::computeNewDt() 5 0.001386 0.001386 0.001386 0.38%
Castro::construct_old_source() 25 0.00128 0.00128 0.00128 0.35%
Castro::construct_old_gravity_source() 5 0.001269 0.001269 0.001269 0.35%
amrex::Add() 72 0.001036 0.001036 0.001036 0.28%
MLMG::ResNormInf() 42 0.0009383 0.0009383 0.0009383 0.26%
Castro::apply_source_to_state() 10 0.0009196 0.0009196 0.0009196 0.25%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0008869 0.0008869 0.0008869 0.24%
MLMG::getGradSolution() 6 0.000752 0.000752 0.000752 0.21%
FabArrayBase::getCPC() 632 0.0007495 0.0007495 0.0007495 0.21%
MLCellLinOp::compGrad() 6 0.0007491 0.0007491 0.0007491 0.21%
Castro::enforce_speed_limit() 30 0.0007427 0.0007427 0.0007427 0.20%
MLCellLinOp::setLevelBC() 6 0.0007316 0.0007316 0.0007316 0.20%
FabArray::mult() 22 0.0006529 0.0006529 0.0006529 0.18%
FabArray::setDomainBndry() 20 0.0006399 0.0006399 0.0006399 0.18%
MLPoisson::prepareForSolve() 6 0.0006049 0.0006049 0.0006049 0.17%
MLCellLinOp::prepareForSolve() 6 0.0006003 0.0006003 0.0006003 0.16%
Castro::check_for_nan() 10 0.0005919 0.0005919 0.0005919 0.16%
MultiFab::contains_nan() 10 0.0005857 0.0005857 0.0005857 0.16%
MLMG::computeMLResidual() 6 0.0005652 0.0005652 0.0005652 0.15%
Amr::InitAmr() 1 0.000438 0.000438 0.000438 0.12%
Gravity::update_max_rhs() 6 0.0004369 0.0004369 0.0004369 0.12%
FabArrayBase::CPC::define() 244 0.0003987 0.0003987 0.0003987 0.11%
Castro::finalize_advance() 5 0.0003216 0.0003216 0.0003216 0.09%
FabArrayBase::getFB() 1766 0.0003094 0.0003094 0.0003094 0.08%
Gravity::swapTimeLevels() 5 0.0002231 0.0002231 0.0002231 0.06%
MLMG::MLResNormInf() 6 0.0001484 0.0001484 0.0001484 0.04%
Castro::buildMetrics() 1 0.000146 0.000146 0.000146 0.04%
MultiFab::max() 6 0.0001355 0.0001355 0.0001355 0.04%
MLLinOp::define() 6 0.0001246 0.0001246 0.0001246 0.03%
MLLinOp::defineGrids() 6 0.0001195 0.0001195 0.0001195 0.03%
MLMG::MLRhsNormInf() 6 0.0001148 0.0001148 0.0001148 0.03%
FabArrayBase::FB::FB() 26 6.16e-05 6.16e-05 6.16e-05 0.02%
MLLinOp::makeAgglomeratedDMap 6 2.936e-05 2.936e-05 2.936e-05 0.01%
Amr::writeSmallPlotFile() 1 2.594e-05 2.594e-05 2.594e-05 0.01%
makeSFC 30 2.217e-05 2.217e-05 2.217e-05 0.01%
Castro::swap_state_time_levels() 5 2.145e-05 2.145e-05 2.145e-05 0.01%
Castro::initMFs() 1 1.953e-05 1.953e-05 1.953e-05 0.01%
Castro::finalize_do_advance() 5 1.877e-05 1.877e-05 1.877e-05 0.01%
DistributionMapping::Distribute() 31 9.948e-06 9.948e-06 9.948e-06 0.00%
Amr::initSubcycle() 1 9.013e-06 9.013e-06 9.013e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 5.024e-06 5.024e-06 5.024e-06 0.00%
Gravity::set_mass_offset() 6 2.635e-06 2.635e-06 2.635e-06 0.00%
Castro::create_source_corrector() 5 2.155e-06 2.155e-06 2.155e-06 0.00%
Castro::retry_advance_ctu() 5 1.996e-06 1.996e-06 1.996e-06 0.00%
Castro::FluxRegCrseInit 5 1.435e-06 1.435e-06 1.435e-06 0.00%
AmrLevel::AmrLevel() 1 1.274e-06 1.274e-06 1.274e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.273e-06 1.273e-06 1.273e-06 0.00%
Castro::FluxRegFineAdd() 5 1.261e-06 1.261e-06 1.261e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12056
Free GPU global memory (MB): 2459
[The Arena] space allocated (MB): 9042
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.12-17-gc2decc83820a) finalized