Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.05-3-gdf9b6944a6d5) initialized
Starting run at 08:24:19 UTC on 2022-05-03.
Successfully read inputs file ...
Castro git describe: 22.05-2-g1289ecab7
AMReX git describe: 22.05-3-gdf9b6944a
Microphysics git describe: 22.05
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
Successfully read inputs file ...
INITIAL GRIDS
Level 0 8 grids 262144 cells 100 % of domain
smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32
CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.040808456 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.024024475 seconds
[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells
[STEP 1] Coarse TimeStep time: 0.047552028
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05
[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells
[STEP 2] Coarse TimeStep time: 0.049967114
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05
[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells
[STEP 3] Coarse TimeStep time: 0.062873487
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05
[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells
[STEP 4] Coarse TimeStep time: 0.062009292
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05
[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells
[STEP 5] Coarse TimeStep time: 0.067431812
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05
CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.038322211 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.064287838
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.059694126
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.062445346
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.060721222
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.065753721
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.038594767 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.024359909 seconds
Ending run at 08:24:20 UTC on 2022-05-03.
Run time = 0.816949813
Run time without initialization = 0.704553659
Average number of zones advanced per microsecond: 3.721
Average number of zones advanced per microsecond per rank: 3.721
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.817 ... 0.817 ... 0.817
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source() 10 0.2192 0.2192 0.2192 26.83%
VisMF::Write(FabArray) 11 0.1597 0.1597 0.1597 19.54%
MLCellLinOp::applyBC() 4379 0.07389 0.07389 0.07389 9.04%
MLPoisson::Fsmooth() 3240 0.06034 0.06034 0.06034 7.39%
StateData::FillBoundary(geom) 328 0.02473 0.02473 0.02473 3.03%
MLCGSolver::bicgstab 81 0.02165 0.02165 0.02165 2.65%
MultiFab::Dot() 1100 0.02126 0.02126 0.02126 2.60%
Castro::normalize_species() 62 0.01871 0.01871 0.01871 2.29%
Castro::computeTemp() 63 0.01526 0.01526 0.01526 1.87%
FillBoundary_nowait() 3974 0.01436 0.01436 0.01436 1.76%
FabArray::setVal() 1135 0.01351 0.01351 0.01351 1.65%
FabArray::ParallelCopy_nowait() 851 0.01337 0.01337 0.01337 1.64%
Castro::enforce_min_density() 62 0.01288 0.01288 0.01288 1.58%
MultiFab::LinComb() 1566 0.01262 0.01262 0.01262 1.54%
StateDataPhysBCFunct::() 41 0.01124 0.01124 0.01124 1.38%
MLPoisson::Fapply() 1128 0.01084 0.01084 0.01084 1.33%
MLCellLinOp::defineAuxData() 11 0.01017 0.01017 0.01017 1.24%
Gravity::fill_multipole_BCs() 11 0.00988 0.00988 0.00988 1.21%
MLMG::addInterpCorrection() 405 0.00674 0.00674 0.00674 0.82%
Castro::estTimeStep() 21 0.006597 0.006597 0.006597 0.81%
amrex::average_down 405 0.006309 0.006309 0.006309 0.77%
MultiFab::Xpay() 578 0.006126 0.006126 0.006126 0.75%
Castro::do_advance_ctu() 10 0.005179 0.005179 0.005179 0.63%
Castro::reset_internal_energy(MultiFab) 63 0.004652 0.004652 0.004652 0.57%
Amr::checkPoint() 3 0.004052 0.004052 0.004052 0.50%
Castro::enforce_speed_limit() 62 0.003977 0.003977 0.003977 0.49%
BndryData::define() 11 0.003848 0.003848 0.003848 0.47%
Castro::construct_new_gravity_source() 10 0.003063 0.003063 0.003063 0.37%
Castro::construct_old_gravity_source() 10 0.002722 0.002722 0.002722 0.33%
Amr::writePlotFile() 2 0.002433 0.002433 0.002433 0.30%
Gravity::get_new_grav_vector() 11 0.001935 0.001935 0.001935 0.24%
MLMG::ResNormInf() 92 0.001884 0.001884 0.001884 0.23%
MultiFab::Saxpy() 20 0.001821 0.001821 0.001821 0.22%
Castro::expand_state() 10 0.001743 0.001743 0.001743 0.21%
Gravity::get_old_grav_vector() 10 0.001733 0.001733 0.001733 0.21%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001693 0.001693 0.001693 0.21%
MLMG::oneIter() 81 0.001669 0.001669 0.001669 0.20%
Castro::reset_internal_energy(Fab) 504 0.001631 0.001631 0.001631 0.20%
Gravity::actual_solve_with_mlmg() 11 0.001374 0.001374 0.001374 0.17%
MLCellLinOp::setLevelBC() 11 0.001344 0.001344 0.001344 0.16%
FabArray::mult() 43 0.001319 0.001319 0.001319 0.16%
FabArray::setDomainBndry() 41 0.001319 0.001319 0.001319 0.16%
MLCellLinOp::smooth() 1620 0.001194 0.001194 0.001194 0.15%
MultiFab::contains_nan() 20 0.001188 0.001188 0.001188 0.15%
MLCellLinOp::prepareForSolve() 11 0.001113 0.001113 0.001113 0.14%
Castro::initData() 1 0.001102 0.001102 0.001102 0.13%
MLCellLinOp::compGrad() 11 0.0009184 0.0009184 0.0009184 0.11%
FabArrayBase::getCPC() 1313 0.0008379 0.0008379 0.0008379 0.10%
MLMG::prepareForSolve() 11 0.0008009 0.0008009 0.0008009 0.10%
FabArray::FillBoundary() 3974 0.0007959 0.0007959 0.0007959 0.10%
FabArrayBase::CPC::define() 454 0.0007154 0.0007154 0.0007154 0.09%
FabArrayBase::getFB() 3974 0.0006833 0.0006833 0.0006833 0.08%
MLCellLinOp::apply() 1128 0.0005303 0.0005303 0.0005303 0.06%
Amr::InitAmr() 1 0.0004444 0.0004444 0.0004444 0.05%
Gravity::update_max_rhs() 11 0.0004138 0.0004138 0.0004138 0.05%
CGSolver::sxay() 1566 0.0004004 0.0004004 0.0004004 0.05%
Gravity::solve_for_phi() 10 0.000396 0.000396 0.000396 0.05%
MLLinOp::defineGrids() 11 0.0003914 0.0003914 0.0003914 0.05%
MLMG::mgVcycle() 81 0.0003802 0.0003802 0.0003802 0.05%
MLCGSolver::ParallelAllReduce 1495 0.0003388 0.0003388 0.0003388 0.04%
main() 1 0.0002792 0.0002792 0.0002792 0.03%
FabArray::ParallelCopy() 851 0.0002715 0.0002715 0.0002715 0.03%
MultiFab::Copy() 11 0.0002683 0.0002683 0.0002683 0.03%
MultiFab::max() 11 0.0002575 0.0002575 0.0002575 0.03%
Amr::coarseTimeStep() 10 0.0002511 0.0002511 0.0002511 0.03%
FillPatchIterator::Initialize 41 0.0002509 0.0002509 0.0002509 0.03%
MLCellLinOp::correctionResidual() 486 0.0002349 0.0002349 0.0002349 0.03%
Castro::construct_new_gravity() 10 0.0002142 0.0002142 0.0002142 0.03%
Castro::subcycle_advance_ctu() 10 0.0002116 0.0002116 0.0002116 0.03%
MLMG::MLRhsNormInf() 11 0.0002022 0.0002022 0.0002022 0.02%
MLCellLinOp::defineBC() 11 0.0001889 0.0001889 0.0001889 0.02%
Amr::timeStep() 10 0.0001844 0.0001844 0.0001844 0.02%
Castro::create_source_corrector() 10 0.0001564 0.0001564 0.0001564 0.02%
MLMG:computeResOfCorrection() 405 0.000144 0.000144 0.000144 0.02%
StateData::checkPoint() 12 0.0001272 0.0001272 0.0001272 0.02%
MLMG::actualBottomSolve() 81 0.0001089 0.0001089 0.0001089 0.01%
MLMG::mgVcycle_down::0 81 9.967e-05 9.967e-05 9.967e-05 0.01%
MLMG::mgVcycle_down::1 81 9.159e-05 9.159e-05 9.159e-05 0.01%
FabArrayBase::FB::FB() 56 9.095e-05 9.095e-05 9.095e-05 0.01%
MLMG::mgVcycle_down::2 81 8.699e-05 8.699e-05 8.699e-05 0.01%
Castro::Castro() 1 8.357e-05 8.357e-05 8.357e-05 0.01%
MLMG::mgVcycle_down::4 81 8.143e-05 8.143e-05 8.143e-05 0.01%
Castro::advance() 10 8.129e-05 8.129e-05 8.129e-05 0.01%
MLMG::mgVcycle_down::3 81 8.111e-05 8.111e-05 8.111e-05 0.01%
MLMG::solve() 11 7.962e-05 7.962e-05 7.962e-05 0.01%
Castro::initialize_advance() 10 7.781e-05 7.781e-05 7.781e-05 0.01%
Castro::clean_state() 62 7.596e-05 7.596e-05 7.596e-05 0.01%
AmrLevel::checkPoint() 3 6.821e-05 6.821e-05 6.821e-05 0.01%
Castro::finalize_advance() 10 6.402e-05 6.402e-05 6.402e-05 0.01%
MLMG::mgVcycle_up::4 81 6.165e-05 6.165e-05 6.165e-05 0.01%
Castro::initialize_do_advance() 10 5.606e-05 5.606e-05 5.606e-05 0.01%
MLMG::mgVcycle_up::0 81 5.3e-05 5.3e-05 5.3e-05 0.01%
MLMG::mgVcycle_up::1 81 5.201e-05 5.201e-05 5.201e-05 0.01%
MLMG::mgVcycle_up::2 81 5.163e-05 5.163e-05 5.163e-05 0.01%
MLMG::mgVcycle_up::3 81 4.947e-05 4.947e-05 4.947e-05 0.01%
MLCellLinOp::solutionResidual() 92 4.762e-05 4.762e-05 4.762e-05 0.01%
StateData::define() 4 4.264e-05 4.264e-05 4.264e-05 0.01%
Castro::post_timestep() 10 4.059e-05 4.059e-05 4.059e-05 0.00%
Castro::construct_new_source() 50 3.708e-05 3.708e-05 3.708e-05 0.00%
Castro::swap_state_time_levels() 10 3.641e-05 3.641e-05 3.641e-05 0.00%
Castro::finalize_do_advance() 10 3.604e-05 3.604e-05 3.604e-05 0.00%
Castro::enforce_consistent_e() 1 3.538e-05 3.538e-05 3.538e-05 0.00%
MLMG::mgVcycle_bottom 81 3.362e-05 3.362e-05 3.362e-05 0.00%
Gravity::actual_multilevel_solve() 1 3.156e-05 3.156e-05 3.156e-05 0.00%
MLMG::computeResidual() 81 3.15e-05 3.15e-05 3.15e-05 0.00%
FillPatchSingleLevel 41 2.886e-05 2.886e-05 2.886e-05 0.00%
Amr::writeSmallPlotFile() 1 2.641e-05 2.641e-05 2.641e-05 0.00%
MLLinOp::define() 11 2.519e-05 2.519e-05 2.519e-05 0.00%
makeSFC 55 2.442e-05 2.442e-05 2.442e-05 0.00%
Castro::initMFs() 1 2.43e-05 2.43e-05 2.43e-05 0.00%
Gravity::solve_phi_with_mlmg() 11 2.418e-05 2.418e-05 2.418e-05 0.00%
MLPoisson::define() 11 2.414e-05 2.414e-05 2.414e-05 0.00%
Amr::FinalizeInit() 1 2.039e-05 2.039e-05 2.039e-05 0.00%
Castro::buildMetrics() 1 2.024e-05 2.024e-05 2.024e-05 0.00%
Castro::construct_old_source() 50 1.87e-05 1.87e-05 1.87e-05 0.00%
Castro::do_new_sources() 10 1.731e-05 1.731e-05 1.731e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.664e-05 1.664e-05 1.664e-05 0.00%
Castro::do_old_sources() 10 1.612e-05 1.612e-05 1.612e-05 0.00%
DistributionMapping::Distribute() 56 1.56e-05 1.56e-05 1.56e-05 0.00%
AmrLevel::AmrLevel(dm) 1 1.538e-05 1.538e-05 1.538e-05 0.00%
Amr::defBaseLevel() 1 1.492e-05 1.492e-05 1.492e-05 0.00%
MLLinOp::makeAgglomeratedDMap 11 1.376e-05 1.376e-05 1.376e-05 0.00%
Castro::check_for_nan() 20 1.227e-05 1.227e-05 1.227e-05 0.00%
Castro::apply_source_to_state() 20 1.149e-05 1.149e-05 1.149e-05 0.00%
Castro::construct_old_gravity() 10 9.91e-06 9.91e-06 9.91e-06 0.00%
Amr::initSubcycle() 1 9.257e-06 9.257e-06 9.257e-06 0.00%
Gravity::swapTimeLevels() 10 8.96e-06 8.96e-06 8.96e-06 0.00%
MLPoisson::prepareForSolve() 11 8.903e-06 8.903e-06 8.903e-06 0.00%
Amr::InitializeInit() 1 6.758e-06 6.758e-06 6.758e-06 0.00%
MLMG::computeMLResidual() 11 6.733e-06 6.733e-06 6.733e-06 0.00%
Castro::computeNewDt() 9 6.437e-06 6.437e-06 6.437e-06 0.00%
AmrLevel::checkPointPost() 3 5.956e-06 5.956e-06 5.956e-06 0.00%
MLMG::getGradSolution() 11 5.339e-06 5.339e-06 5.339e-06 0.00%
MLMG::buildFineMask() 11 5.283e-06 5.283e-06 5.283e-06 0.00%
Gravity::set_mass_offset() 11 5.06e-06 5.06e-06 5.06e-06 0.00%
MLMG::MLResNormInf() 11 4.868e-06 4.868e-06 4.868e-06 0.00%
Amr::initialInit() 1 4.834e-06 4.834e-06 4.834e-06 0.00%
Castro::retry_advance_ctu() 10 3.808e-06 3.808e-06 3.808e-06 0.00%
Castro::post_init() 1 3.502e-06 3.502e-06 3.502e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 2.982e-06 2.982e-06 2.982e-06 0.00%
Castro::FluxRegCrseInit 10 2.843e-06 2.843e-06 2.843e-06 0.00%
Castro::computeInitialDt() 2 2.437e-06 2.437e-06 2.437e-06 0.00%
Amr::init() 1 2.39e-06 2.39e-06 2.39e-06 0.00%
Castro::FluxRegFineAdd() 10 2.296e-06 2.296e-06 2.296e-06 0.00%
AmrLevel::checkPointPre() 3 2.179e-06 2.179e-06 2.179e-06 0.00%
MLLinOp::makeSubCommunicator() 11 1.918e-06 1.918e-06 1.918e-06 0.00%
Castro::post_regrid() 1 1.407e-06 1.407e-06 1.407e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.817 0.817 0.817 100.00%
Amr::coarseTimeStep() 10 0.68 0.68 0.68 83.23%
Amr::timeStep() 10 0.5991 0.5991 0.5991 73.33%
Castro::advance() 10 0.5882 0.5882 0.5882 71.99%
Castro::subcycle_advance_ctu() 10 0.5745 0.5745 0.5745 70.32%
Castro::do_advance_ctu() 10 0.5742 0.5742 0.5742 70.29%
Gravity::solve_phi_with_mlmg() 11 0.296 0.296 0.296 36.23%
Gravity::actual_solve_with_mlmg() 11 0.2859 0.2859 0.2859 35.00%
Castro::construct_new_gravity() 10 0.2719 0.2719 0.2719 33.28%
MLMG::solve() 11 0.2647 0.2647 0.2647 32.40%
Gravity::solve_for_phi() 10 0.2564 0.2564 0.2564 31.38%
MLMG::oneIter() 81 0.2508 0.2508 0.2508 30.70%
MLMG::mgVcycle() 81 0.2491 0.2491 0.2491 30.49%
Castro::construct_ctu_hydro_source() 10 0.2192 0.2192 0.2192 26.83%
VisMF::Write(FabArray) 11 0.1597 0.1597 0.1597 19.54%
MLCellLinOp::smooth() 1620 0.1281 0.1281 0.1281 15.68%
Amr::checkPoint() 3 0.1179 0.1179 0.1179 14.43%
AmrLevel::checkPoint() 3 0.1138 0.1138 0.1138 13.93%
StateData::checkPoint() 12 0.1137 0.1137 0.1137 13.92%
Amr::init() 1 0.1118 0.1118 0.1118 13.69%
MLCellLinOp::applyBC() 4379 0.08982 0.08982 0.08982 10.99%
MLMG::mgVcycle_bottom 81 0.07521 0.07521 0.07521 9.21%
MLMG::actualBottomSolve() 81 0.07517 0.07517 0.07517 9.20%
MLCGSolver::bicgstab 81 0.07441 0.07441 0.07441 9.11%
MLPoisson::Fsmooth() 3240 0.06034 0.06034 0.06034 7.39%
Castro::clean_state() 62 0.05659 0.05659 0.05659 6.93%
Amr::writePlotFile() 2 0.0485 0.0485 0.0485 5.94%
Amr::initialInit() 1 0.04687 0.04687 0.04687 5.74%
Amr::FinalizeInit() 1 0.04318 0.04318 0.04318 5.28%
Castro::post_init() 1 0.0419 0.0419 0.0419 5.13%
FillPatchIterator::Initialize 41 0.04153 0.04153 0.04153 5.08%
Gravity::multilevel_solve_for_new_phi() 1 0.04005 0.04005 0.04005 4.90%
Gravity::actual_multilevel_solve() 1 0.04003 0.04003 0.04003 4.90%
FillPatchSingleLevel 41 0.03996 0.03996 0.03996 4.89%
StateDataPhysBCFunct::() 41 0.03597 0.03597 0.03597 4.40%
MLMG::mgVcycle_down::0 81 0.03455 0.03455 0.03455 4.23%
MLCellLinOp::apply() 1128 0.03414 0.03414 0.03414 4.18%
MLMG::mgVcycle_up::0 81 0.02958 0.02958 0.02958 3.62%
StateData::FillBoundary(geom) 328 0.02473 0.02473 0.02473 3.03%
Castro::computeTemp() 63 0.02155 0.02155 0.02155 2.64%
MultiFab::Dot() 1100 0.02126 0.02126 0.02126 2.60%
Castro::initialize_do_advance() 10 0.02079 0.02079 0.02079 2.54%
MLCellLinOp::correctionResidual() 486 0.01992 0.01992 0.01992 2.44%
Castro::normalize_species() 62 0.01871 0.01871 0.01871 2.29%
MLMG:computeResOfCorrection() 405 0.01726 0.01726 0.01726 2.11%
MLPoisson::define() 11 0.01706 0.01706 0.01706 2.09%
Gravity::get_new_grav_vector() 11 0.01706 0.01706 0.01706 2.09%
MLMG::mgVcycle_down::1 81 0.01658 0.01658 0.01658 2.03%
MLMG::mgVcycle_down::2 81 0.01595 0.01595 0.01595 1.95%
FabArray::FillBoundary() 3974 0.01593 0.01593 0.01593 1.95%
MLMG::mgVcycle_down::3 81 0.0152 0.0152 0.0152 1.86%
FillBoundary_nowait() 3974 0.01513 0.01513 0.01513 1.85%
Castro::construct_old_gravity() 10 0.01465 0.01465 0.01465 1.79%
Gravity::get_old_grav_vector() 10 0.01464 0.01464 0.01464 1.79%
FabArray::ParallelCopy() 851 0.01453 0.01453 0.01453 1.78%
MLMG::mgVcycle_down::4 81 0.01434 0.01434 0.01434 1.76%
FabArray::ParallelCopy_nowait() 851 0.01425 0.01425 0.01425 1.74%
Castro::initialize_advance() 10 0.01356 0.01356 0.01356 1.66%
FabArray::setVal() 1135 0.01351 0.01351 0.01351 1.65%
CGSolver::sxay() 1566 0.01302 0.01302 0.01302 1.59%
Castro::enforce_min_density() 62 0.01288 0.01288 0.01288 1.58%
MLCGSolver::ParallelAllReduce 1495 0.01276 0.01276 0.01276 1.56%
Castro::do_old_sources() 10 0.01269 0.01269 0.01269 1.55%
MultiFab::LinComb() 1566 0.01262 0.01262 0.01262 1.54%
MLMG::mgVcycle_up::2 81 0.01229 0.01229 0.01229 1.50%
MLMG::mgVcycle_up::1 81 0.01207 0.01207 0.01207 1.48%
MLMG::addInterpCorrection() 405 0.012 0.012 0.012 1.47%
Castro::do_new_sources() 10 0.01184 0.01184 0.01184 1.45%
amrex::average_down 405 0.01162 0.01162 0.01162 1.42%
MLCellLinOp::defineAuxData() 11 0.01157 0.01157 0.01157 1.42%
MLMG::mgVcycle_up::3 81 0.01155 0.01155 0.01155 1.41%
MLMG::mgVcycle_up::4 81 0.01143 0.01143 0.01143 1.40%
Castro::expand_state() 10 0.01126 0.01126 0.01126 1.38%
MLPoisson::Fapply() 1128 0.01084 0.01084 0.01084 1.33%
Castro::post_timestep() 10 0.01073 0.01073 0.01073 1.31%
Gravity::fill_multipole_BCs() 11 0.00988 0.00988 0.00988 1.21%
MLCellLinOp::solutionResidual() 92 0.007054 0.007054 0.007054 0.86%
Castro::estTimeStep() 21 0.006597 0.006597 0.006597 0.81%
Castro::reset_internal_energy(MultiFab) 63 0.006283 0.006283 0.006283 0.77%
MultiFab::Xpay() 578 0.006126 0.006126 0.006126 0.75%
MLMG::computeResidual() 81 0.006071 0.006071 0.006071 0.74%
MLCellLinOp::defineBC() 11 0.004996 0.004996 0.004996 0.61%
BndryData::define() 11 0.004807 0.004807 0.004807 0.59%
MLMG::prepareForSolve() 11 0.004662 0.004662 0.004662 0.57%
Castro::enforce_speed_limit() 62 0.003977 0.003977 0.003977 0.49%
Amr::InitializeInit() 1 0.003687 0.003687 0.003687 0.45%
Amr::defBaseLevel() 1 0.00368 0.00368 0.00368 0.45%
Castro::computeNewDt() 9 0.003171 0.003171 0.003171 0.39%
Castro::initData() 1 0.003171 0.003171 0.003171 0.39%
Castro::construct_new_source() 50 0.0031 0.0031 0.0031 0.38%
Castro::construct_new_gravity_source() 10 0.003063 0.003063 0.003063 0.37%
Castro::construct_old_source() 50 0.002741 0.002741 0.002741 0.34%
Castro::construct_old_gravity_source() 10 0.002722 0.002722 0.002722 0.33%
MLMG::ResNormInf() 92 0.001884 0.001884 0.001884 0.23%
Castro::apply_source_to_state() 20 0.001833 0.001833 0.001833 0.22%
MultiFab::Saxpy() 20 0.001821 0.001821 0.001821 0.22%
FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.001693 0.001693 0.001693 0.21%
Castro::reset_internal_energy(Fab) 504 0.001631 0.001631 0.001631 0.20%
FabArrayBase::getCPC() 1313 0.001553 0.001553 0.001553 0.19%
MLMG::getGradSolution() 11 0.001395 0.001395 0.001395 0.17%
MLCellLinOp::compGrad() 11 0.00139 0.00139 0.00139 0.17%
MLCellLinOp::setLevelBC() 11 0.001344 0.001344 0.001344 0.16%
FabArray::mult() 43 0.001319 0.001319 0.001319 0.16%
FabArray::setDomainBndry() 41 0.001319 0.001319 0.001319 0.16%
Castro::check_for_nan() 20 0.001201 0.001201 0.001201 0.15%
MultiFab::contains_nan() 20 0.001188 0.001188 0.001188 0.15%
MLPoisson::prepareForSolve() 11 0.001122 0.001122 0.001122 0.14%
Castro::post_regrid() 1 0.001121 0.001121 0.001121 0.14%
MLCellLinOp::prepareForSolve() 11 0.001113 0.001113 0.001113 0.14%
MLMG::computeMLResidual() 11 0.001021 0.001021 0.001021 0.13%
Gravity::update_max_rhs() 11 0.0008175 0.0008175 0.0008175 0.10%
FabArrayBase::getFB() 3974 0.0007742 0.0007742 0.0007742 0.09%
FabArrayBase::CPC::define() 454 0.0007154 0.0007154 0.0007154 0.09%
Castro::computeInitialDt() 2 0.0005949 0.0005949 0.0005949 0.07%
MLLinOp::define() 11 0.000471 0.000471 0.000471 0.06%
Amr::InitAmr() 1 0.0004537 0.0004537 0.0004537 0.06%
Gravity::swapTimeLevels() 10 0.0004476 0.0004476 0.0004476 0.05%
MLLinOp::defineGrids() 11 0.0004458 0.0004458 0.0004458 0.05%
Castro::Castro() 1 0.000432 0.000432 0.000432 0.05%
MultiFab::Copy() 11 0.0002683 0.0002683 0.0002683 0.03%
MultiFab::max() 11 0.0002575 0.0002575 0.0002575 0.03%
MLMG::MLResNormInf() 11 0.0002559 0.0002559 0.0002559 0.03%
MLMG::MLRhsNormInf() 11 0.0002022 0.0002022 0.0002022 0.02%
Castro::buildMetrics() 1 0.0001595 0.0001595 0.0001595 0.02%
Castro::create_source_corrector() 10 0.0001564 0.0001564 0.0001564 0.02%
FabArrayBase::FB::FB() 56 9.095e-05 9.095e-05 9.095e-05 0.01%
Castro::finalize_advance() 10 6.916e-05 6.916e-05 6.916e-05 0.01%
AmrLevel::AmrLevel(dm) 1 5.802e-05 5.802e-05 5.802e-05 0.01%
MLLinOp::makeAgglomeratedDMap 11 5.25e-05 5.25e-05 5.25e-05 0.01%
StateData::define() 4 4.264e-05 4.264e-05 4.264e-05 0.01%
makeSFC 55 3.874e-05 3.874e-05 3.874e-05 0.00%
Castro::swap_state_time_levels() 10 3.641e-05 3.641e-05 3.641e-05 0.00%
Castro::finalize_do_advance() 10 3.604e-05 3.604e-05 3.604e-05 0.00%
Castro::enforce_consistent_e() 1 3.538e-05 3.538e-05 3.538e-05 0.00%
Amr::writeSmallPlotFile() 1 2.641e-05 2.641e-05 2.641e-05 0.00%
Castro::initMFs() 1 2.43e-05 2.43e-05 2.43e-05 0.00%
DistributionMapping::Distribute() 56 1.56e-05 1.56e-05 1.56e-05 0.00%
Amr::initSubcycle() 1 9.257e-06 9.257e-06 9.257e-06 0.00%
AmrLevel::checkPointPost() 3 5.956e-06 5.956e-06 5.956e-06 0.00%
MLMG::buildFineMask() 11 5.283e-06 5.283e-06 5.283e-06 0.00%
Gravity::set_mass_offset() 11 5.06e-06 5.06e-06 5.06e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.265e-06 4.265e-06 4.265e-06 0.00%
Castro::retry_advance_ctu() 10 3.808e-06 3.808e-06 3.808e-06 0.00%
Castro::FluxRegCrseInit 10 2.843e-06 2.843e-06 2.843e-06 0.00%
Castro::FluxRegFineAdd() 10 2.296e-06 2.296e-06 2.296e-06 0.00%
AmrLevel::checkPointPre() 3 2.179e-06 2.179e-06 2.179e-06 0.00%
MLLinOp::makeSubCommunicator() 11 1.918e-06 1.918e-06 1.918e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 2465
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.05-3-gdf9b6944a6d5) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.05-3-gdf9b6944a6d5) initialized
Starting run at 08:24:21 UTC on 2022-05-03.
Successfully read inputs file ...
Castro git describe: 22.05-2-g1289ecab7
AMReX git describe: 22.05-3-gdf9b6944a
Microphysics git describe: 22.05
reading extern runtime parameters ...
3 Species:
C12 O16 Mg24
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ...
read CPU time: 0.440470701
Restart time = 0.044869746 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells
[STEP 6] Coarse TimeStep time: 0.051525282
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05
[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells
[STEP 7] Coarse TimeStep time: 0.062233288
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05
[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells
[STEP 8] Coarse TimeStep time: 0.062522459
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05
[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells
[STEP 9] Coarse TimeStep time: 0.065770508
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05
[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells
[STEP 10] Coarse TimeStep time: 0.064584905
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]
STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.341867034 seconds
Ending run at 08:24:21 UTC on 2022-05-03.
Run time = 0.694254213
Run time without initialization = 0.648838756
Average number of zones advanced per microsecond: 2.020
Average number of zones advanced per microsecond per rank: 2.020
CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728
TinyProfiler total time across processes [min...avg...max]: 0.6943 ... 0.6943 ... 0.6943
--------------------------------------------------------------------------------------------
Name NCalls Excl. Min Excl. Avg Excl. Max Max %
--------------------------------------------------------------------------------------------
VisMF::Write(FabArray) 1 0.3405 0.3405 0.3405 49.04%
Castro::construct_ctu_hydro_source() 5 0.1163 0.1163 0.1163 16.76%
VisMF::Read() 3 0.03786 0.03786 0.03786 5.45%
MLCellLinOp::applyBC() 1946 0.03209 0.03209 0.03209 4.62%
MLPoisson::Fsmooth() 1440 0.02592 0.02592 0.02592 3.73%
StateData::FillBoundary(geom) 160 0.01166 0.01166 0.01166 1.68%
MLCGSolver::bicgstab 36 0.009282 0.009282 0.009282 1.34%
MultiFab::Dot() 484 0.009124 0.009124 0.009124 1.31%
Castro::normalize_species() 30 0.008548 0.008548 0.008548 1.23%
Castro::computeTemp() 30 0.007861 0.007861 0.007861 1.13%
FabArray::setVal() 537 0.006453 0.006453 0.006453 0.93%
FillBoundary_nowait() 1766 0.006339 0.006339 0.006339 0.91%
FabArray::ParallelCopy_nowait() 380 0.006028 0.006028 0.006028 0.87%
Castro::enforce_min_density() 30 0.006005 0.006005 0.006005 0.86%
MLCellLinOp::defineAuxData() 6 0.005428 0.005428 0.005428 0.78%
MultiFab::LinComb() 690 0.005425 0.005425 0.005425 0.78%
MLPoisson::Fapply() 500 0.004675 0.004675 0.004675 0.67%
Gravity::fill_multipole_BCs() 6 0.004667 0.004667 0.004667 0.67%
StateDataPhysBCFunct::() 20 0.004394 0.004394 0.004394 0.63%
Castro::estTimeStep() 10 0.003266 0.003266 0.003266 0.47%
MLMG::addInterpCorrection() 180 0.002924 0.002924 0.002924 0.42%
Amr::restart() 1 0.002875 0.002875 0.002875 0.41%
amrex::average_down 180 0.00273 0.00273 0.00273 0.39%
MultiFab::Xpay() 258 0.002697 0.002697 0.002697 0.39%
Castro::do_advance_ctu() 5 0.002236 0.002236 0.002236 0.32%
BndryData::define() 6 0.002088 0.002088 0.002088 0.30%
Castro::enforce_speed_limit() 30 0.001742 0.001742 0.001742 0.25%
Castro::construct_new_gravity_source() 5 0.001696 0.001696 0.001696 0.24%
Castro::reset_internal_energy(MultiFab) 30 0.001603 0.001603 0.001603 0.23%
Amr::writePlotFile() 1 0.001489 0.001489 0.001489 0.21%
Castro::construct_old_gravity_source() 5 0.00137 0.00137 0.00137 0.20%
Gravity::get_new_grav_vector() 5 0.0009737 0.0009737 0.0009737 0.14%
MultiFab::Saxpy() 10 0.0009323 0.0009323 0.0009323 0.13%
Gravity::get_old_grav_vector() 5 0.000919 0.000919 0.000919 0.13%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0009122 0.0009122 0.0009122 0.13%
Castro::expand_state() 5 0.0008729 0.0008729 0.0008729 0.13%
Castro::reset_internal_energy(Fab) 240 0.0008686 0.0008686 0.0008686 0.13%
MLMG::ResNormInf() 42 0.0008343 0.0008343 0.0008343 0.12%
Gravity::actual_solve_with_mlmg() 6 0.0007271 0.0007271 0.0007271 0.10%
MLMG::oneIter() 36 0.000725 0.000725 0.000725 0.10%
MLCellLinOp::setLevelBC() 6 0.0007238 0.0007238 0.0007238 0.10%
FabArray::mult() 22 0.0006483 0.0006483 0.0006483 0.09%
FabArray::setDomainBndry() 20 0.0006433 0.0006433 0.0006433 0.09%
MLCellLinOp::prepareForSolve() 6 0.0006095 0.0006095 0.0006095 0.09%
MultiFab::contains_nan() 10 0.000598 0.000598 0.000598 0.09%
MLCellLinOp::smooth() 720 0.0005074 0.0005074 0.0005074 0.07%
MLCellLinOp::compGrad() 6 0.0004856 0.0004856 0.0004856 0.07%
MLMG::prepareForSolve() 6 0.0004312 0.0004312 0.0004312 0.06%
FabArrayBase::CPC::define() 244 0.0004194 0.0004194 0.0004194 0.06%
FabArray::FillBoundary() 1766 0.0003826 0.0003826 0.0003826 0.06%
FabArrayBase::getCPC() 632 0.0003826 0.0003826 0.0003826 0.06%
Amr::InitAmr() 1 0.0003789 0.0003789 0.0003789 0.05%
FabArrayBase::getFB() 1766 0.00029 0.00029 0.00029 0.04%
main() 1 0.0002708 0.0002708 0.0002708 0.04%
Gravity::update_max_rhs() 6 0.0002288 0.0002288 0.0002288 0.03%
MLCellLinOp::apply() 500 0.0002198 0.0002198 0.0002198 0.03%
Gravity::solve_for_phi() 5 0.0001977 0.0001977 0.0001977 0.03%
CGSolver::sxay() 690 0.0001937 0.0001937 0.0001937 0.03%
MLLinOp::defineGrids() 6 0.0001757 0.0001757 0.0001757 0.03%
MLMG::mgVcycle() 36 0.0001596 0.0001596 0.0001596 0.02%
Castro::advance() 5 0.0001487 0.0001487 0.0001487 0.02%
Castro::construct_new_gravity() 5 0.0001458 0.0001458 0.0001458 0.02%
MultiFab::Copy() 6 0.000142 0.000142 0.000142 0.02%
MLCGSolver::ParallelAllReduce 659 0.0001394 0.0001394 0.0001394 0.02%
FabArray::ParallelCopy() 380 0.000135 0.000135 0.000135 0.02%
MultiFab::max() 6 0.0001337 0.0001337 0.0001337 0.02%
Castro::create_source_corrector() 5 0.0001326 0.0001326 0.0001326 0.02%
Amr::coarseTimeStep() 5 0.0001324 0.0001324 0.0001324 0.02%
Castro::construct_new_source() 25 0.0001254 0.0001254 0.0001254 0.02%
FillPatchIterator::Initialize 20 0.0001253 0.0001253 0.0001253 0.02%
MLMG::MLRhsNormInf() 6 0.0001074 0.0001074 0.0001074 0.02%
MLCellLinOp::defineBC() 6 0.0001036 0.0001036 0.0001036 0.01%
MLCellLinOp::correctionResidual() 216 0.0001031 0.0001031 0.0001031 0.01%
Amr::timeStep() 5 9.126e-05 9.126e-05 9.126e-05 0.01%
Castro::construct_old_source() 25 8.051e-05 8.051e-05 8.051e-05 0.01%
Castro::subcycle_advance_ctu() 5 7.752e-05 7.752e-05 7.752e-05 0.01%
AmrLevel::restart() 1 7.477e-05 7.477e-05 7.477e-05 0.01%
StateData::restartDoit() 4 7.147e-05 7.147e-05 7.147e-05 0.01%
Castro::post_timestep() 5 6.932e-05 6.932e-05 6.932e-05 0.01%
FabArrayBase::FB::FB() 26 6.07e-05 6.07e-05 6.07e-05 0.01%
Castro::initialize_do_advance() 5 5.927e-05 5.927e-05 5.927e-05 0.01%
MLMG:computeResOfCorrection() 180 5.415e-05 5.415e-05 5.415e-05 0.01%
MLMG::actualBottomSolve() 36 4.687e-05 4.687e-05 4.687e-05 0.01%
MLMG::mgVcycle_down::0 36 4.23e-05 4.23e-05 4.23e-05 0.01%
MLMG::mgVcycle_down::1 36 3.772e-05 3.772e-05 3.772e-05 0.01%
Castro::clean_state() 30 3.683e-05 3.683e-05 3.683e-05 0.01%
MLMG::solve() 6 3.585e-05 3.585e-05 3.585e-05 0.01%
Castro::construct_old_gravity() 5 3.582e-05 3.582e-05 3.582e-05 0.01%
Castro::initialize_advance() 5 3.496e-05 3.496e-05 3.496e-05 0.01%
MLMG::mgVcycle_down::2 36 3.486e-05 3.486e-05 3.486e-05 0.01%
MLMG::mgVcycle_down::4 36 3.322e-05 3.322e-05 3.322e-05 0.00%
MLMG::mgVcycle_down::3 36 3.196e-05 3.196e-05 3.196e-05 0.00%
Castro::post_restart() 1 3.079e-05 3.079e-05 3.079e-05 0.00%
Gravity::actual_multilevel_solve() 1 2.955e-05 2.955e-05 2.955e-05 0.00%
Castro::buildMetrics() 1 2.888e-05 2.888e-05 2.888e-05 0.00%
Castro::do_new_sources() 5 2.792e-05 2.792e-05 2.792e-05 0.00%
MLMG::mgVcycle_up::4 36 2.74e-05 2.74e-05 2.74e-05 0.00%
Amr::writeSmallPlotFile() 1 2.731e-05 2.731e-05 2.731e-05 0.00%
Gravity::solve_phi_with_mlmg() 6 2.709e-05 2.709e-05 2.709e-05 0.00%
Castro::initMFs() 1 2.592e-05 2.592e-05 2.592e-05 0.00%
Castro::swap_state_time_levels() 5 2.539e-05 2.539e-05 2.539e-05 0.00%
Castro::finalize_advance() 5 2.482e-05 2.482e-05 2.482e-05 0.00%
MLCellLinOp::solutionResidual() 42 2.27e-05 2.27e-05 2.27e-05 0.00%
MLMG::mgVcycle_up::3 36 2.18e-05 2.18e-05 2.18e-05 0.00%
MLMG::mgVcycle_up::0 36 2.166e-05 2.166e-05 2.166e-05 0.00%
MLMG::mgVcycle_up::2 36 2.154e-05 2.154e-05 2.154e-05 0.00%
MLLinOp::define() 6 2.1e-05 2.1e-05 2.1e-05 0.00%
Castro::computeNewDt() 5 2.098e-05 2.098e-05 2.098e-05 0.00%
MLMG::mgVcycle_up::1 36 2.034e-05 2.034e-05 2.034e-05 0.00%
Castro::finalize_do_advance() 5 1.917e-05 1.917e-05 1.917e-05 0.00%
MLMG::mgVcycle_bottom 36 1.66e-05 1.66e-05 1.66e-05 0.00%
Gravity::multilevel_solve_for_new_phi() 1 1.645e-05 1.645e-05 1.645e-05 0.00%
MLMG::computeResidual() 36 1.578e-05 1.578e-05 1.578e-05 0.00%
MLPoisson::define() 6 1.403e-05 1.403e-05 1.403e-05 0.00%
makeSFC 30 1.37e-05 1.37e-05 1.37e-05 0.00%
FillPatchSingleLevel 20 1.322e-05 1.322e-05 1.322e-05 0.00%
Amr::initSubcycle() 1 9e-06 9e-06 9e-06 0.00%
DistributionMapping::Distribute() 31 8.808e-06 8.808e-06 8.808e-06 0.00%
Castro::do_old_sources() 5 8.397e-06 8.397e-06 8.397e-06 0.00%
MLLinOp::makeAgglomeratedDMap 6 7.196e-06 7.196e-06 7.196e-06 0.00%
Castro::check_for_nan() 10 6.829e-06 6.829e-06 6.829e-06 0.00%
Castro::apply_source_to_state() 10 5.903e-06 5.903e-06 5.903e-06 0.00%
Gravity::swapTimeLevels() 5 4.323e-06 4.323e-06 4.323e-06 0.00%
MLPoisson::prepareForSolve() 6 4.321e-06 4.321e-06 4.321e-06 0.00%
MLMG::buildFineMask() 6 3.311e-06 3.311e-06 3.311e-06 0.00%
MLMG::computeMLResidual() 6 3.284e-06 3.284e-06 3.284e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 3.146e-06 3.146e-06 3.146e-06 0.00%
MLMG::getGradSolution() 6 3.016e-06 3.016e-06 3.016e-06 0.00%
MLMG::MLResNormInf() 6 2.695e-06 2.695e-06 2.695e-06 0.00%
Gravity::set_mass_offset() 6 2.256e-06 2.256e-06 2.256e-06 0.00%
Castro::retry_advance_ctu() 5 1.957e-06 1.957e-06 1.957e-06 0.00%
Castro::FluxRegCrseInit 5 1.711e-06 1.711e-06 1.711e-06 0.00%
Castro::FluxRegFineAdd() 5 1.336e-06 1.336e-06 1.336e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.222e-06 1.222e-06 1.222e-06 0.00%
AmrLevel::AmrLevel() 1 1.201e-06 1.201e-06 1.201e-06 0.00%
Amr::init() 1 1.052e-06 1.052e-06 1.052e-06 0.00%
--------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------
Name NCalls Incl. Min Incl. Avg Incl. Max Max %
--------------------------------------------------------------------------------------------
main() 1 0.6943 0.6943 0.6943 100.00%
Amr::writePlotFile() 1 0.3419 0.3419 0.3419 49.25%
VisMF::Write(FabArray) 1 0.3405 0.3405 0.3405 49.04%
Amr::coarseTimeStep() 5 0.3067 0.3067 0.3067 44.18%
Amr::timeStep() 5 0.3048 0.3048 0.3048 43.91%
Castro::advance() 5 0.2999 0.2999 0.2999 43.19%
Castro::subcycle_advance_ctu() 5 0.2931 0.2931 0.2931 42.21%
Castro::do_advance_ctu() 5 0.293 0.293 0.293 42.20%
Castro::construct_new_gravity() 5 0.1354 0.1354 0.1354 19.50%
Gravity::solve_phi_with_mlmg() 6 0.1311 0.1311 0.1311 18.89%
Gravity::solve_for_phi() 5 0.1277 0.1277 0.1277 18.40%
Gravity::actual_solve_with_mlmg() 6 0.1263 0.1263 0.1263 18.20%
Castro::construct_ctu_hydro_source() 5 0.1163 0.1163 0.1163 16.76%
MLMG::solve() 6 0.115 0.115 0.115 16.56%
MLMG::oneIter() 36 0.1082 0.1082 0.1082 15.59%
MLMG::mgVcycle() 36 0.1075 0.1075 0.1075 15.48%
MLCellLinOp::smooth() 720 0.05537 0.05537 0.05537 7.97%
Amr::init() 1 0.0449 0.0449 0.0449 6.47%
Amr::restart() 1 0.0449 0.0449 0.0449 6.47%
MLCellLinOp::applyBC() 1946 0.03916 0.03916 0.03916 5.64%
AmrLevel::restart() 1 0.03807 0.03807 0.03807 5.48%
StateData::restartDoit() 4 0.03799 0.03799 0.03799 5.47%
VisMF::Read() 3 0.03786 0.03786 0.03786 5.45%
MLMG::mgVcycle_bottom 36 0.03229 0.03229 0.03229 4.65%
MLMG::actualBottomSolve() 36 0.03227 0.03227 0.03227 4.65%
MLCGSolver::bicgstab 36 0.03194 0.03194 0.03194 4.60%
Castro::clean_state() 30 0.02666 0.02666 0.02666 3.84%
MLPoisson::Fsmooth() 1440 0.02592 0.02592 0.02592 3.73%
FillPatchIterator::Initialize 20 0.01883 0.01883 0.01883 2.71%
FillPatchSingleLevel 20 0.01806 0.01806 0.01806 2.60%
StateDataPhysBCFunct::() 20 0.01606 0.01606 0.01606 2.31%
MLMG::mgVcycle_down::0 36 0.01508 0.01508 0.01508 2.17%
MLCellLinOp::apply() 500 0.01486 0.01486 0.01486 2.14%
MLMG::mgVcycle_up::0 36 0.01281 0.01281 0.01281 1.85%
StateData::FillBoundary(geom) 160 0.01166 0.01166 0.01166 1.68%
Castro::computeTemp() 30 0.01033 0.01033 0.01033 1.49%
Castro::initialize_do_advance() 5 0.01009 0.01009 0.01009 1.45%
MLPoisson::define() 6 0.009177 0.009177 0.009177 1.32%
MultiFab::Dot() 484 0.009124 0.009124 0.009124 1.31%
MLCellLinOp::correctionResidual() 216 0.008685 0.008685 0.008685 1.25%
Castro::normalize_species() 30 0.008548 0.008548 0.008548 1.23%
Gravity::get_new_grav_vector() 5 0.007535 0.007535 0.007535 1.09%
MLMG:computeResOfCorrection() 180 0.007509 0.007509 0.007509 1.08%
Castro::construct_old_gravity() 5 0.007248 0.007248 0.007248 1.04%
Gravity::get_old_grav_vector() 5 0.007212 0.007212 0.007212 1.04%
MLMG::mgVcycle_down::1 36 0.007155 0.007155 0.007155 1.03%
FabArray::FillBoundary() 1766 0.007072 0.007072 0.007072 1.02%
Castro::do_new_sources() 5 0.006977 0.006977 0.006977 1.00%
MLMG::mgVcycle_down::2 36 0.006868 0.006868 0.006868 0.99%
FillBoundary_nowait() 1766 0.00669 0.00669 0.00669 0.96%
Castro::initialize_advance() 5 0.006635 0.006635 0.006635 0.96%
FabArray::ParallelCopy() 380 0.006559 0.006559 0.006559 0.94%
MLMG::mgVcycle_down::3 36 0.006505 0.006505 0.006505 0.94%
FabArray::setVal() 537 0.006453 0.006453 0.006453 0.93%
FabArray::ParallelCopy_nowait() 380 0.006424 0.006424 0.006424 0.93%
MLMG::mgVcycle_down::4 36 0.006235 0.006235 0.006235 0.90%
MLCellLinOp::defineAuxData() 6 0.006195 0.006195 0.006195 0.89%
Castro::enforce_min_density() 30 0.006005 0.006005 0.006005 0.86%
Castro::do_old_sources() 5 0.005757 0.005757 0.005757 0.83%
CGSolver::sxay() 690 0.005618 0.005618 0.005618 0.81%
MLCGSolver::ParallelAllReduce 659 0.005502 0.005502 0.005502 0.79%
MultiFab::LinComb() 690 0.005425 0.005425 0.005425 0.78%
MLMG::mgVcycle_up::2 36 0.005286 0.005286 0.005286 0.76%
MLMG::mgVcycle_up::1 36 0.005205 0.005205 0.005205 0.75%
MLMG::addInterpCorrection() 180 0.005179 0.005179 0.005179 0.75%
amrex::average_down 180 0.005047 0.005047 0.005047 0.73%
MLMG::mgVcycle_up::3 36 0.004965 0.004965 0.004965 0.72%
MLMG::mgVcycle_up::4 36 0.004949 0.004949 0.004949 0.71%
Castro::post_timestep() 5 0.004865 0.004865 0.004865 0.70%
Castro::expand_state() 5 0.004811 0.004811 0.004811 0.69%
MLPoisson::Fapply() 500 0.004675 0.004675 0.004675 0.67%
Gravity::fill_multipole_BCs() 6 0.004667 0.004667 0.004667 0.67%
Castro::post_restart() 1 0.003773 0.003773 0.003773 0.54%
Gravity::multilevel_solve_for_new_phi() 1 0.003651 0.003651 0.003651 0.53%
Gravity::actual_multilevel_solve() 1 0.003635 0.003635 0.003635 0.52%
Castro::estTimeStep() 10 0.003266 0.003266 0.003266 0.47%
MLCellLinOp::solutionResidual() 42 0.003207 0.003207 0.003207 0.46%
MLCellLinOp::defineBC() 6 0.002742 0.002742 0.002742 0.39%
MultiFab::Xpay() 258 0.002697 0.002697 0.002697 0.39%
MLMG::computeResidual() 36 0.002659 0.002659 0.002659 0.38%
BndryData::define() 6 0.002638 0.002638 0.002638 0.38%
MLMG::prepareForSolve() 6 0.002526 0.002526 0.002526 0.36%
Castro::reset_internal_energy(MultiFab) 30 0.002471 0.002471 0.002471 0.36%
Castro::construct_new_source() 25 0.001822 0.001822 0.001822 0.26%
Castro::computeNewDt() 5 0.001754 0.001754 0.001754 0.25%
Castro::enforce_speed_limit() 30 0.001742 0.001742 0.001742 0.25%
Castro::construct_new_gravity_source() 5 0.001696 0.001696 0.001696 0.24%
Castro::construct_old_source() 25 0.001451 0.001451 0.001451 0.21%
Castro::construct_old_gravity_source() 5 0.00137 0.00137 0.00137 0.20%
Castro::apply_source_to_state() 10 0.0009382 0.0009382 0.0009382 0.14%
MultiFab::Saxpy() 10 0.0009323 0.0009323 0.0009323 0.13%
FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.0009122 0.0009122 0.0009122 0.13%
Castro::reset_internal_energy(Fab) 240 0.0008686 0.0008686 0.0008686 0.13%
MLMG::ResNormInf() 42 0.0008343 0.0008343 0.0008343 0.12%
FabArrayBase::getCPC() 632 0.000802 0.000802 0.000802 0.12%
MLMG::getGradSolution() 6 0.0007492 0.0007492 0.0007492 0.11%
MLCellLinOp::compGrad() 6 0.0007462 0.0007462 0.0007462 0.11%
MLCellLinOp::setLevelBC() 6 0.0007238 0.0007238 0.0007238 0.10%
FabArray::mult() 22 0.0006483 0.0006483 0.0006483 0.09%
FabArray::setDomainBndry() 20 0.0006433 0.0006433 0.0006433 0.09%
MLPoisson::prepareForSolve() 6 0.0006138 0.0006138 0.0006138 0.09%
MLCellLinOp::prepareForSolve() 6 0.0006095 0.0006095 0.0006095 0.09%
Castro::check_for_nan() 10 0.0006048 0.0006048 0.0006048 0.09%
MultiFab::contains_nan() 10 0.000598 0.000598 0.000598 0.09%
MLMG::computeMLResidual() 6 0.0005679 0.0005679 0.0005679 0.08%
Gravity::update_max_rhs() 6 0.0004411 0.0004411 0.0004411 0.06%
FabArrayBase::CPC::define() 244 0.0004194 0.0004194 0.0004194 0.06%
Amr::InitAmr() 1 0.0003879 0.0003879 0.0003879 0.06%
FabArrayBase::getFB() 1766 0.0003508 0.0003508 0.0003508 0.05%
Gravity::swapTimeLevels() 5 0.0002297 0.0002297 0.0002297 0.03%
MLLinOp::define() 6 0.0002261 0.0002261 0.0002261 0.03%
MLLinOp::defineGrids() 6 0.0002051 0.0002051 0.0002051 0.03%
Castro::buildMetrics() 1 0.0001587 0.0001587 0.0001587 0.02%
MultiFab::Copy() 6 0.000142 0.000142 0.000142 0.02%
MLMG::MLResNormInf() 6 0.0001368 0.0001368 0.0001368 0.02%
MultiFab::max() 6 0.0001337 0.0001337 0.0001337 0.02%
Castro::create_source_corrector() 5 0.0001326 0.0001326 0.0001326 0.02%
MLMG::MLRhsNormInf() 6 0.0001074 0.0001074 0.0001074 0.02%
FabArrayBase::FB::FB() 26 6.07e-05 6.07e-05 6.07e-05 0.01%
MLLinOp::makeAgglomeratedDMap 6 2.824e-05 2.824e-05 2.824e-05 0.00%
Castro::finalize_advance() 5 2.787e-05 2.787e-05 2.787e-05 0.00%
Amr::writeSmallPlotFile() 1 2.731e-05 2.731e-05 2.731e-05 0.00%
Castro::initMFs() 1 2.592e-05 2.592e-05 2.592e-05 0.00%
Castro::swap_state_time_levels() 5 2.539e-05 2.539e-05 2.539e-05 0.00%
makeSFC 30 2.105e-05 2.105e-05 2.105e-05 0.00%
Castro::finalize_do_advance() 5 1.917e-05 1.917e-05 1.917e-05 0.00%
Amr::initSubcycle() 1 9e-06 9e-06 9e-06 0.00%
DistributionMapping::Distribute() 31 8.808e-06 8.808e-06 8.808e-06 0.00%
DistributionMapping::SFCProcessorMapDoIt() 1 4.604e-06 4.604e-06 4.604e-06 0.00%
MLMG::buildFineMask() 6 3.311e-06 3.311e-06 3.311e-06 0.00%
Gravity::set_mass_offset() 6 2.256e-06 2.256e-06 2.256e-06 0.00%
Castro::retry_advance_ctu() 5 1.957e-06 1.957e-06 1.957e-06 0.00%
Castro::FluxRegCrseInit 5 1.711e-06 1.711e-06 1.711e-06 0.00%
Castro::FluxRegFineAdd() 5 1.336e-06 1.336e-06 1.336e-06 0.00%
MLLinOp::makeSubCommunicator() 6 1.222e-06 1.222e-06 1.222e-06 0.00%
AmrLevel::AmrLevel() 1 1.201e-06 1.201e-06 1.201e-06 0.00%
--------------------------------------------------------------------------------------------
Unused ParmParse Variables:
[TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2]
[TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2]
Total GPU global memory (MB): 12066
Free GPU global memory (MB): 2465
[The Arena] space allocated (MB): 9049
[The Arena] space used (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used (MB): 0
[The Pinned Arena] space allocated (MB): 8
[The Pinned Arena] space used (MB): 0
AMReX (22.05-3-gdf9b6944a6d5) finalized