Initializing CUDA... CUDA initialized with 1 GPU AMReX (22.01-1-ga73ce6759c91) initialized Starting run at 09:05:21 UTC on 2022-01-03. Successfully read inputs file ... Castro git describe: 21.12-18-g669e45228 AMReX git describe: 22.01-1-ga73ce6759 Microphysics git describe: 22.01-4-gaba1dfe8 reading extern runtime parameters ... 3 Species: C12 O16 Mg24 Successfully read inputs file ... INITIAL GRIDS Level 0 8 grids 262144 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 CHECKPOINT: file = dustcollapse-restart_chk00000 checkPoint() time = 0.041680813 secs. PLOTFILE: file = dustcollapse-restart_plt00000 Write plotfile time = 0.02479997 seconds [Level 0 step 1] ADVANCE with dt = 4.541742215e-05 [Level 0 step 1] Advanced 262144 cells [STEP 1] Coarse TimeStep time: 0.149611824 [STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05 [Level 0 step 2] ADVANCE with dt = 4.768829326e-05 [Level 0 step 2] Advanced 262144 cells [STEP 2] Coarse TimeStep time: 0.072561524 [STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05 [Level 0 step 3] ADVANCE with dt = 5.007270792e-05 [Level 0 step 3] Advanced 262144 cells [STEP 3] Coarse TimeStep time: 0.070792848 [STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05 [Level 0 step 4] ADVANCE with dt = 5.257634331e-05 [Level 0 step 4] Advanced 262144 cells [STEP 4] Coarse TimeStep time: 0.074886787 [STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05 [Level 0 step 5] ADVANCE with dt = 5.520516048e-05 [Level 0 step 5] Advanced 262144 cells [STEP 5] Coarse TimeStep time: 0.080970737 [STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05 CHECKPOINT: file = dustcollapse-restart_chk00005 checkPoint() time = 0.03980122 secs. [Level 0 step 6] ADVANCE with dt = 5.79654185e-05 [Level 0 step 6] Advanced 262144 cells [STEP 6] Coarse TimeStep time: 0.085397679 [STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05 [Level 0 step 7] ADVANCE with dt = 6.086368943e-05 [Level 0 step 7] Advanced 262144 cells [STEP 7] Coarse TimeStep time: 0.086943354 [STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05 [Level 0 step 8] ADVANCE with dt = 6.39068739e-05 [Level 0 step 8] Advanced 262144 cells [STEP 8] Coarse TimeStep time: 0.079626397 [STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05 [Level 0 step 9] ADVANCE with dt = 6.71022176e-05 [Level 0 step 9] Advanced 262144 cells [STEP 9] Coarse TimeStep time: 0.082959082 [STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05 [Level 0 step 10] ADVANCE with dt = 7.045732848e-05 [Level 0 step 10] Advanced 262144 cells [STEP 10] Coarse TimeStep time: 0.077200758 [STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05 CHECKPOINT: file = dustcollapse-restart_chk00010 checkPoint() time = 0.041977731 secs. PLOTFILE: file = dustcollapse-restart_plt00010 Write plotfile time = 0.024768136 seconds Ending run at 09:05:23 UTC on 2022-01-03. Run time = 1.134857918 Run time without initialization = 0.968136167 Average number of zones advanced per microsecond: 2.708 Average number of zones advanced per microsecond per rank: 2.708 CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489334272 TinyProfiler total time across processes [min...avg...max]: 1.135 ... 1.135 ... 1.135 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 10 0.2323 0.2323 0.2323 20.47% VisMF::Write(FabArray) 11 0.1647 0.1647 0.1647 14.51% MLCellLinOp::applyBC() 4433 0.119 0.119 0.119 10.48% MLPoisson::Fsmooth() 3280 0.0751 0.0751 0.0751 6.62% FillBoundary_nowait() 4023 0.04022 0.04022 0.04022 3.54% MLCGSolver::bicgstab 82 0.0382 0.0382 0.0382 3.37% StateDataPhysBCFunct::() 41 0.0346 0.0346 0.0346 3.05% FabArray::setVal() 1140 0.03099 0.03099 0.03099 2.73% MultiFab::Dot() 1114 0.02813 0.02813 0.02813 2.48% StateData::FillBoundary(geom) 328 0.02566 0.02566 0.02566 2.26% MLCellLinOp::smooth() 1640 0.02369 0.02369 0.02369 2.09% FabArray::FillBoundary() 4023 0.02289 0.02289 0.02289 2.02% FabArray::ParallelCopy_nowait() 861 0.022 0.022 0.022 1.94% MultiFab::LinComb() 1586 0.02019 0.02019 0.02019 1.78% MLPoisson::Fapply() 1142 0.01597 0.01597 0.01597 1.41% MLCellLinOp::defineAuxData() 11 0.01412 0.01412 0.01412 1.24% Gravity::fill_multipole_BCs() 11 0.01218 0.01218 0.01218 1.07% Castro::computeTemp() 63 0.01203 0.01203 0.01203 1.06% FabArrayBase::getFB() 4023 0.01189 0.01189 0.01189 1.05% MLMG::addInterpCorrection() 410 0.01003 0.01003 0.01003 0.88% MLCellLinOp::apply() 1142 0.009903 0.009903 0.009903 0.87% amrex::average_down 410 0.009366 0.009366 0.009366 0.83% CGSolver::sxay() 1586 0.00916 0.00916 0.00916 0.81% MultiFab::Xpay() 585 0.008976 0.008976 0.008976 0.79% Castro::enforce_min_density() 62 0.007054 0.007054 0.007054 0.62% MLCGSolver::ParallelAllReduce 1514 0.006173 0.006173 0.006173 0.54% FabArray::setDomainBndry() 41 0.005903 0.005903 0.005903 0.52% Castro::reset_internal_energy() 63 0.005863 0.005863 0.005863 0.52% Castro::estTimeStep() 21 0.005833 0.005833 0.005833 0.51% FabArrayBase::getCPC() 1323 0.005758 0.005758 0.005758 0.51% FabArray::ParallelCopy() 861 0.005114 0.005114 0.005114 0.45% Amr::checkPoint() 3 0.004824 0.004824 0.004824 0.43% Castro::expand_state() 10 0.004582 0.004582 0.004582 0.40% BndryData::define() 11 0.004566 0.004566 0.004566 0.40% MLCellLinOp::correctionResidual() 492 0.004295 0.004295 0.004295 0.38% Castro::do_advance_ctu() 10 0.003785 0.003785 0.003785 0.33% FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.003646 0.003646 0.003646 0.32% Castro::enforce_speed_limit() 62 0.003443 0.003443 0.003443 0.30% Amr::writePlotFile() 2 0.003397 0.003397 0.003397 0.30% MLMG::mgVcycle() 82 0.003138 0.003138 0.003138 0.28% Gravity::get_new_grav_vector() 11 0.003026 0.003026 0.003026 0.27% Castro::normalize_species() 62 0.003012 0.003012 0.003012 0.27% Castro::construct_new_gravity_source() 10 0.002895 0.002895 0.002895 0.26% MLMG:computeResOfCorrection() 410 0.002377 0.002377 0.002377 0.21% MLMG::ResNormInf() 93 0.002307 0.002307 0.002307 0.20% MLMG::oneIter() 82 0.002307 0.002307 0.002307 0.20% Castro::construct_old_gravity_source() 10 0.002173 0.002173 0.002173 0.19% Amr::defBaseLevel() 1 0.002157 0.002157 0.002157 0.19% MLCellLinOp::compGrad() 11 0.002088 0.002088 0.002088 0.18% FabArrayBase::CPC::define() 454 0.001973 0.001973 0.001973 0.17% Gravity::get_old_grav_vector() 10 0.001969 0.001969 0.001969 0.17% MultiFab::Saxpy() 20 0.001918 0.001918 0.001918 0.17% MLCellLinOp::setLevelBC() 11 0.001687 0.001687 0.001687 0.15% MLMG::prepareForSolve() 11 0.001659 0.001659 0.001659 0.15% FabArray::mult() 43 0.001531 0.001531 0.001531 0.13% Gravity::actual_solve_with_mlmg() 11 0.001507 0.001507 0.001507 0.13% MLMG::mgVcycle_down::4 82 0.001462 0.001462 0.001462 0.13% MLMG::mgVcycle_down::3 82 0.001433 0.001433 0.001433 0.13% MLMG::mgVcycle_down::1 82 0.001422 0.001422 0.001422 0.13% MLMG::mgVcycle_down::0 82 0.001422 0.001422 0.001422 0.13% MLMG::mgVcycle_down::2 82 0.00142 0.00142 0.00142 0.13% MLCellLinOp::prepareForSolve() 11 0.001401 0.001401 0.001401 0.12% Castro::initData() 1 0.001353 0.001353 0.001353 0.12% MLMG::actualBottomSolve() 82 0.001252 0.001252 0.001252 0.11% MLMG::solve() 11 0.001025 0.001025 0.001025 0.09% MLMG::mgVcycle_up::0 82 0.0009685 0.0009685 0.0009685 0.09% MLMG::mgVcycle_up::1 82 0.0009497 0.0009497 0.0009497 0.08% MLMG::mgVcycle_up::4 82 0.0009456 0.0009456 0.0009456 0.08% MLMG::mgVcycle_up::3 82 0.0009379 0.0009379 0.0009379 0.08% MLMG::mgVcycle_up::2 82 0.0009363 0.0009363 0.0009363 0.08% Castro::clean_state() 62 0.000926 0.000926 0.000926 0.08% MLCellLinOp::solutionResidual() 93 0.0008054 0.0008054 0.0008054 0.07% Gravity::update_max_rhs() 11 0.0007331 0.0007331 0.0007331 0.06% Gravity::solve_for_phi() 10 0.000601 0.000601 0.000601 0.05% FillPatchIterator::Initialize 41 0.0005877 0.0005877 0.0005877 0.05% MultiFab::Copy() 11 0.0005762 0.0005762 0.0005762 0.05% Amr::InitAmr() 1 0.0005086 0.0005086 0.0005086 0.04% MLMG::computeResidual() 82 0.0004906 0.0004906 0.0004906 0.04% MLMG::mgVcycle_bottom 82 0.0004764 0.0004764 0.0004764 0.04% Castro::initialize_advance() 10 0.0004353 0.0004353 0.0004353 0.04% Amr::coarseTimeStep() 10 0.0004137 0.0004137 0.0004137 0.04% MultiFab::min() 10 0.0003848 0.0003848 0.0003848 0.03% main() 1 0.0003835 0.0003835 0.0003835 0.03% FillPatchSingleLevel 41 0.0003644 0.0003644 0.0003644 0.03% Castro::construct_new_gravity() 10 0.0003458 0.0003458 0.0003458 0.03% Castro::subcycle_advance_ctu() 10 0.0003429 0.0003429 0.0003429 0.03% makeSFC 55 0.0003287 0.0003287 0.0003287 0.03% Castro::do_old_sources() 10 0.000311 0.000311 0.000311 0.03% MultiFab::max() 11 0.0003054 0.0003054 0.0003054 0.03% Castro::do_new_sources() 10 0.0002895 0.0002895 0.0002895 0.03% Castro::advance() 10 0.0002745 0.0002745 0.0002745 0.02% MLLinOp::defineGrids() 11 0.0002701 0.0002701 0.0002701 0.02% FabArrayBase::FB::FB() 56 0.0002672 0.0002672 0.0002672 0.02% Amr::timeStep() 10 0.0002654 0.0002654 0.0002654 0.02% MLCellLinOp::defineBC() 11 0.0002572 0.0002572 0.0002572 0.02% MLMG::MLRhsNormInf() 11 0.0002396 0.0002396 0.0002396 0.02% Castro::construct_new_source() 50 0.0002355 0.0002355 0.0002355 0.02% Gravity::actual_multilevel_solve() 1 0.0002315 0.0002315 0.0002315 0.02% Castro::construct_old_source() 50 0.0002251 0.0002251 0.0002251 0.02% DistributionMapping::Distribute() 56 0.0002003 0.0002003 0.0002003 0.02% MLLinOp::makeAgglomeratedDMap 11 0.0001986 0.0001986 0.0001986 0.02% StateData::checkPoint() 12 0.0001919 0.0001919 0.0001919 0.02% Gravity::solve_phi_with_mlmg() 11 0.0001782 0.0001782 0.0001782 0.02% MLPoisson::define() 11 0.0001462 0.0001462 0.0001462 0.01% Castro::initialize_do_advance() 10 0.0001436 0.0001436 0.0001436 0.01% Castro::apply_source_to_state() 20 0.0001397 0.0001397 0.0001397 0.01% Castro::finalize_advance() 10 0.0001374 0.0001374 0.0001374 0.01% AmrLevel::checkPoint() 3 0.0001297 0.0001297 0.0001297 0.01% Gravity::swapTimeLevels() 10 0.0001177 0.0001177 0.0001177 0.01% MLLinOp::define() 11 9.192e-05 9.192e-05 9.192e-05 0.01% Castro::swap_state_time_levels() 10 6.871e-05 6.871e-05 6.871e-05 0.01% MLPoisson::prepareForSolve() 11 6.812e-05 6.812e-05 6.812e-05 0.01% MLMG::computeMLResidual() 11 6.679e-05 6.679e-05 6.679e-05 0.01% Castro::construct_old_gravity() 10 6.625e-05 6.625e-05 6.625e-05 0.01% MLMG::getGradSolution() 11 6.596e-05 6.596e-05 6.596e-05 0.01% Castro::finalize_do_advance() 10 6.471e-05 6.471e-05 6.471e-05 0.01% MLMG::MLResNormInf() 11 6.433e-05 6.433e-05 6.433e-05 0.01% Castro::post_timestep() 10 6.158e-05 6.158e-05 6.158e-05 0.01% Castro::computeNewDt() 9 5.658e-05 5.658e-05 5.658e-05 0.00% StateData::define() 4 5.638e-05 5.638e-05 5.638e-05 0.00% MLMG::buildFineMask() 11 5.171e-05 5.171e-05 5.171e-05 0.00% Amr::FinalizeInit() 1 4.242e-05 4.242e-05 4.242e-05 0.00% Castro::enforce_consistent_e() 1 3.783e-05 3.783e-05 3.783e-05 0.00% Gravity::set_mass_offset() 11 3.575e-05 3.575e-05 3.575e-05 0.00% MLLinOp::makeSubCommunicator() 11 3.254e-05 3.254e-05 3.254e-05 0.00% Castro::create_source_corrector() 10 3.242e-05 3.242e-05 3.242e-05 0.00% Castro::retry_advance_ctu() 10 3.146e-05 3.146e-05 3.146e-05 0.00% Castro::FluxRegFineAdd() 10 3.083e-05 3.083e-05 3.083e-05 0.00% AmrLevel::AmrLevel(dm) 1 2.725e-05 2.725e-05 2.725e-05 0.00% Gravity::multilevel_solve_for_new_phi() 1 2.283e-05 2.283e-05 2.283e-05 0.00% Castro::post_init() 1 1.838e-05 1.838e-05 1.838e-05 0.00% Castro::computeInitialDt() 2 1.421e-05 1.421e-05 1.421e-05 0.00% Amr::init() 1 1.412e-05 1.412e-05 1.412e-05 0.00% Amr::writeSmallPlotFile() 1 1.358e-05 1.358e-05 1.358e-05 0.00% AmrLevel::checkPointPost() 3 1.312e-05 1.312e-05 1.312e-05 0.00% Amr::InitializeInit() 1 1.171e-05 1.171e-05 1.171e-05 0.00% AmrLevel::checkPointPre() 3 1e-05 1e-05 1e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 9.583e-06 9.583e-06 9.583e-06 0.00% Amr::initialInit() 1 9.25e-06 9.25e-06 9.25e-06 0.00% Amr::initSubcycle() 1 8.834e-06 8.834e-06 8.834e-06 0.00% Castro::post_regrid() 1 6.834e-06 6.834e-06 6.834e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 1.135 1.135 1.135 100.00% Amr::coarseTimeStep() 10 0.9431 0.9431 0.9431 83.10% Amr::timeStep() 10 0.8576 0.8576 0.8576 75.56% Castro::advance() 10 0.8523 0.8523 0.8523 75.10% Castro::subcycle_advance_ctu() 10 0.8376 0.8376 0.8376 73.80% Castro::do_advance_ctu() 10 0.8372 0.8372 0.8372 73.77% Gravity::solve_phi_with_mlmg() 11 0.5697 0.5697 0.5697 50.20% Gravity::actual_solve_with_mlmg() 11 0.5571 0.5571 0.5571 49.08% MLMG::solve() 11 0.5232 0.5232 0.5232 46.10% Castro::construct_new_gravity() 10 0.5208 0.5208 0.5208 45.89% MLMG::oneIter() 82 0.4986 0.4986 0.4986 43.93% MLMG::mgVcycle() 82 0.4963 0.4963 0.4963 43.73% Gravity::solve_for_phi() 10 0.4853 0.4853 0.4853 42.77% MLCellLinOp::smooth() 1640 0.2413 0.2413 0.2413 21.26% Castro::construct_ctu_hydro_source() 10 0.2323 0.2323 0.2323 20.47% MLCellLinOp::applyBC() 4433 0.1943 0.1943 0.1943 17.12% Amr::init() 1 0.166 0.166 0.166 14.63% VisMF::Write(FabArray) 11 0.1647 0.1647 0.1647 14.51% MLMG::mgVcycle_bottom 82 0.1486 0.1486 0.1486 13.10% MLMG::actualBottomSolve() 82 0.1482 0.1482 0.1482 13.06% MLCGSolver::bicgstab 82 0.1459 0.1459 0.1459 12.85% Amr::checkPoint() 3 0.1236 0.1236 0.1236 10.89% AmrLevel::checkPoint() 3 0.1188 0.1188 0.1188 10.46% StateData::checkPoint() 12 0.1186 0.1186 0.1186 10.45% Amr::initialInit() 1 0.09944 0.09944 0.09944 8.76% Amr::FinalizeInit() 1 0.08894 0.08894 0.08894 7.84% Castro::post_init() 1 0.08815 0.08815 0.08815 7.77% Gravity::multilevel_solve_for_new_phi() 1 0.08518 0.08518 0.08518 7.51% Gravity::actual_multilevel_solve() 1 0.08516 0.08516 0.08516 7.50% MLCellLinOp::apply() 1142 0.07685 0.07685 0.07685 6.77% FabArray::FillBoundary() 4023 0.07528 0.07528 0.07528 6.63% MLPoisson::Fsmooth() 3280 0.0751 0.0751 0.0751 6.62% FillPatchIterator::Initialize 41 0.072 0.072 0.072 6.34% FillPatchSingleLevel 41 0.06551 0.06551 0.06551 5.77% StateDataPhysBCFunct::() 41 0.06026 0.06026 0.06026 5.31% MLMG::mgVcycle_down::0 82 0.05468 0.05468 0.05468 4.82% FillBoundary_nowait() 4023 0.05238 0.05238 0.05238 4.62% Amr::writePlotFile() 2 0.04968 0.04968 0.04968 4.38% MLMG::mgVcycle_up::0 82 0.04567 0.04567 0.04567 4.02% MLCellLinOp::correctionResidual() 492 0.04473 0.04473 0.04473 3.94% MLMG:computeResOfCorrection() 410 0.04016 0.04016 0.04016 3.54% Gravity::get_new_grav_vector() 11 0.0378 0.0378 0.0378 3.33% MLMG::mgVcycle_down::1 82 0.03523 0.03523 0.03523 3.10% MLMG::mgVcycle_down::2 82 0.03498 0.03498 0.03498 3.08% MLMG::mgVcycle_down::3 82 0.03407 0.03407 0.03407 3.00% MLMG::mgVcycle_down::4 82 0.03314 0.03314 0.03314 2.92% Castro::clean_state() 62 0.03146 0.03146 0.03146 2.77% FabArray::ParallelCopy() 861 0.031 0.031 0.031 2.73% FabArray::setVal() 1140 0.03099 0.03099 0.03099 2.73% CGSolver::sxay() 1586 0.02935 0.02935 0.02935 2.59% MultiFab::Dot() 1114 0.02813 0.02813 0.02813 2.48% MLPoisson::define() 11 0.0277 0.0277 0.0277 2.44% MLMG::mgVcycle_up::2 82 0.0271 0.0271 0.0271 2.39% MLMG::mgVcycle_up::1 82 0.02704 0.02704 0.02704 2.38% MLMG::mgVcycle_up::3 82 0.02638 0.02638 0.02638 2.32% MLMG::mgVcycle_up::4 82 0.02623 0.02623 0.02623 2.31% FabArray::ParallelCopy_nowait() 861 0.02589 0.02589 0.02589 2.28% StateData::FillBoundary(geom) 328 0.02566 0.02566 0.02566 2.26% Castro::initialize_do_advance() 10 0.02515 0.02515 0.02515 2.22% MLMG::addInterpCorrection() 410 0.02303 0.02303 0.02303 2.03% MLCGSolver::ParallelAllReduce 1514 0.02258 0.02258 0.02258 1.99% amrex::average_down 410 0.02247 0.02247 0.02247 1.98% MultiFab::LinComb() 1586 0.02019 0.02019 0.02019 1.78% Castro::expand_state() 10 0.01991 0.01991 0.01991 1.75% MLCellLinOp::defineAuxData() 11 0.01989 0.01989 0.01989 1.75% Castro::construct_old_gravity() 10 0.01896 0.01896 0.01896 1.67% Gravity::get_old_grav_vector() 10 0.01889 0.01889 0.01889 1.66% Castro::computeTemp() 63 0.01789 0.01789 0.01789 1.58% MLPoisson::Fapply() 1142 0.01597 0.01597 0.01597 1.41% Castro::initialize_advance() 10 0.01427 0.01427 0.01427 1.26% Castro::do_new_sources() 10 0.01256 0.01256 0.01256 1.11% Gravity::fill_multipole_BCs() 11 0.01218 0.01218 0.01218 1.07% FabArrayBase::getFB() 4023 0.01216 0.01216 0.01216 1.07% MLCellLinOp::solutionResidual() 93 0.01212 0.01212 0.01212 1.07% MLMG::computeResidual() 82 0.01101 0.01101 0.01101 0.97% Amr::InitializeInit() 1 0.01049 0.01049 0.01049 0.92% Amr::defBaseLevel() 1 0.01048 0.01048 0.01048 0.92% MultiFab::Xpay() 585 0.008976 0.008976 0.008976 0.79% MLMG::prepareForSolve() 11 0.008308 0.008308 0.008308 0.73% Castro::do_old_sources() 10 0.008205 0.008205 0.008205 0.72% FabArrayBase::getCPC() 1323 0.007731 0.007731 0.007731 0.68% Castro::enforce_min_density() 62 0.007054 0.007054 0.007054 0.62% MLCellLinOp::defineBC() 11 0.006548 0.006548 0.006548 0.58% BndryData::define() 11 0.006291 0.006291 0.006291 0.55% FabArray::setDomainBndry() 41 0.005903 0.005903 0.005903 0.52% Castro::reset_internal_energy() 63 0.005863 0.005863 0.005863 0.52% Castro::estTimeStep() 21 0.005833 0.005833 0.005833 0.51% Castro::post_timestep() 10 0.005016 0.005016 0.005016 0.44% Castro::initData() 1 0.003735 0.003735 0.003735 0.33% FabArray::setVal(val, thecmd, scomp, ncomp) 462 0.003646 0.003646 0.003646 0.32% Castro::enforce_speed_limit() 62 0.003443 0.003443 0.003443 0.30% Castro::construct_new_source() 50 0.003131 0.003131 0.003131 0.28% Castro::normalize_species() 62 0.003012 0.003012 0.003012 0.27% MLMG::getGradSolution() 11 0.002937 0.002937 0.002937 0.26% Castro::construct_new_gravity_source() 10 0.002895 0.002895 0.002895 0.26% MLCellLinOp::compGrad() 11 0.002871 0.002871 0.002871 0.25% Castro::computeNewDt() 9 0.002724 0.002724 0.002724 0.24% Castro::construct_old_source() 50 0.002398 0.002398 0.002398 0.21% MLMG::ResNormInf() 93 0.002307 0.002307 0.002307 0.20% Castro::construct_old_gravity_source() 10 0.002173 0.002173 0.002173 0.19% Castro::apply_source_to_state() 20 0.002057 0.002057 0.002057 0.18% FabArrayBase::CPC::define() 454 0.001973 0.001973 0.001973 0.17% MultiFab::Saxpy() 20 0.001918 0.001918 0.001918 0.17% MLCellLinOp::setLevelBC() 11 0.001687 0.001687 0.001687 0.15% MLMG::computeMLResidual() 11 0.001667 0.001667 0.001667 0.15% FabArray::mult() 43 0.001531 0.001531 0.001531 0.13% MLPoisson::prepareForSolve() 11 0.001469 0.001469 0.001469 0.13% MLCellLinOp::prepareForSolve() 11 0.001401 0.001401 0.001401 0.12% Gravity::swapTimeLevels() 10 0.00135 0.00135 0.00135 0.12% Gravity::update_max_rhs() 11 0.00124 0.00124 0.00124 0.11% MLLinOp::define() 11 0.001118 0.001118 0.001118 0.10% MLLinOp::defineGrids() 11 0.001027 0.001027 0.001027 0.09% MLLinOp::makeAgglomeratedDMap 11 0.0007239 0.0007239 0.0007239 0.06% Castro::computeInitialDt() 2 0.0007232 0.0007232 0.0007232 0.06% MultiFab::Copy() 11 0.0005762 0.0005762 0.0005762 0.05% Castro::post_regrid() 1 0.000529 0.000529 0.000529 0.05% makeSFC 55 0.0005253 0.0005253 0.0005253 0.05% Amr::InitAmr() 1 0.0005175 0.0005175 0.0005175 0.05% MultiFab::min() 10 0.0003848 0.0003848 0.0003848 0.03% MLMG::MLResNormInf() 11 0.0003558 0.0003558 0.0003558 0.03% MultiFab::max() 11 0.0003054 0.0003054 0.0003054 0.03% FabArrayBase::FB::FB() 56 0.0002672 0.0002672 0.0002672 0.02% MLMG::MLRhsNormInf() 11 0.0002396 0.0002396 0.0002396 0.02% DistributionMapping::Distribute() 56 0.0002003 0.0002003 0.0002003 0.02% Castro::finalize_advance() 10 0.0001683 0.0001683 0.0001683 0.01% AmrLevel::AmrLevel(dm) 1 8.363e-05 8.363e-05 8.363e-05 0.01% Castro::swap_state_time_levels() 10 6.871e-05 6.871e-05 6.871e-05 0.01% Castro::finalize_do_advance() 10 6.471e-05 6.471e-05 6.471e-05 0.01% StateData::define() 4 5.638e-05 5.638e-05 5.638e-05 0.00% MLMG::buildFineMask() 11 5.171e-05 5.171e-05 5.171e-05 0.00% Castro::enforce_consistent_e() 1 3.783e-05 3.783e-05 3.783e-05 0.00% Gravity::set_mass_offset() 11 3.575e-05 3.575e-05 3.575e-05 0.00% MLLinOp::makeSubCommunicator() 11 3.254e-05 3.254e-05 3.254e-05 0.00% Castro::create_source_corrector() 10 3.242e-05 3.242e-05 3.242e-05 0.00% Castro::retry_advance_ctu() 10 3.146e-05 3.146e-05 3.146e-05 0.00% Castro::FluxRegFineAdd() 10 3.083e-05 3.083e-05 3.083e-05 0.00% Amr::writeSmallPlotFile() 1 1.358e-05 1.358e-05 1.358e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 1.333e-05 1.333e-05 1.333e-05 0.00% AmrLevel::checkPointPost() 3 1.312e-05 1.312e-05 1.312e-05 0.00% AmrLevel::checkPointPre() 3 1e-05 1e-05 1e-05 0.00% Amr::initSubcycle() 1 8.834e-06 8.834e-06 8.834e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2] [TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2] Total GPU global memory (MB): 12066 Free GPU global memory (MB): 10622 [The Arena] space allocated (MB): 9049 [The Arena] space used (MB): 0 [The Pinned Arena] space allocated (MB): 8 [The Pinned Arena] space used (MB): 0 AMReX (22.01-1-ga73ce6759c91) finalized Initializing CUDA... CUDA initialized with 1 GPU AMReX (22.01-1-ga73ce6759c91) initialized Starting run at 09:05:23 UTC on 2022-01-03. Successfully read inputs file ... Castro git describe: 21.12-18-g669e45228 AMReX git describe: 22.01-1-ga73ce6759 Microphysics git describe: 22.01-4-gaba1dfe8 reading extern runtime parameters ... 3 Species: C12 O16 Mg24 restarting calculation from file: dustcollapse-restart_chk00005 Successfully read inputs file ... read CPU time: 0.65528036 Restart time = 0.059386119 seconds. [Level 0 step 6] ADVANCE with dt = 5.79654185e-05 [Level 0 step 6] Advanced 262144 cells [STEP 6] Coarse TimeStep time: 0.145801636 [STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05 [Level 0 step 7] ADVANCE with dt = 6.086368943e-05 [Level 0 step 7] Advanced 262144 cells [STEP 7] Coarse TimeStep time: 0.07665113 [STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05 [Level 0 step 8] ADVANCE with dt = 6.39068739e-05 [Level 0 step 8] Advanced 262144 cells [STEP 8] Coarse TimeStep time: 0.083311876 [STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05 [Level 0 step 9] ADVANCE with dt = 6.71022176e-05 [Level 0 step 9] Advanced 262144 cells [STEP 9] Coarse TimeStep time: 0.086100933 [STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05 [Level 0 step 10] ADVANCE with dt = 7.045732848e-05 [Level 0 step 10] Advanced 262144 cells [STEP 10] Coarse TimeStep time: 0.095342983 [STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638] STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05 PLOTFILE: file = dustcollapse-restart_plt00010 Write plotfile time = 0.098009165 seconds Ending run at 09:05:24 UTC on 2022-01-03. Run time = 0.645681974 Run time without initialization = 0.585649977 Average number of zones advanced per microsecond: 2.238 Average number of zones advanced per microsecond per rank: 2.238 CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489334272 TinyProfiler total time across processes [min...avg...max]: 0.6457 ... 0.6457 ... 0.6457 -------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % -------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 5 0.1499 0.1499 0.1499 23.22% Amr::writePlotFile() 1 0.07247 0.07247 0.07247 11.22% MLCellLinOp::applyBC() 1946 0.05502 0.05502 0.05502 8.52% VisMF::Read() 3 0.0448 0.0448 0.0448 6.94% MLPoisson::Fsmooth() 1440 0.03278 0.03278 0.03278 5.08% VisMF::Write(FabArray) 1 0.02561 0.02561 0.02561 3.97% FabArray::setVal() 533 0.01909 0.01909 0.01909 2.96% FillBoundary_nowait() 1766 0.01848 0.01848 0.01848 2.86% MLCGSolver::bicgstab 36 0.01658 0.01658 0.01658 2.57% StateData::FillBoundary(geom) 160 0.01249 0.01249 0.01249 1.93% MultiFab::Dot() 484 0.01223 0.01223 0.01223 1.89% MLCellLinOp::smooth() 720 0.01042 0.01042 0.01042 1.61% StateDataPhysBCFunct::() 20 0.0104 0.0104 0.0104 1.61% FabArray::ParallelCopy_nowait() 380 0.01036 0.01036 0.01036 1.60% FabArray::FillBoundary() 1766 0.01014 0.01014 0.01014 1.57% MultiFab::LinComb() 690 0.008797 0.008797 0.008797 1.36% MLCellLinOp::defineAuxData() 6 0.007608 0.007608 0.007608 1.18% Gravity::fill_multipole_BCs() 6 0.007264 0.007264 0.007264 1.12% Castro::computeTemp() 30 0.007206 0.007206 0.007206 1.12% MLPoisson::Fapply() 500 0.00693 0.00693 0.00693 1.07% FabArrayBase::getFB() 1766 0.005246 0.005246 0.005246 0.81% Castro::enforce_min_density() 30 0.00504 0.00504 0.00504 0.78% Amr::restart() 1 0.004831 0.004831 0.004831 0.75% Castro::expand_state() 5 0.004442 0.004442 0.004442 0.69% MLMG::addInterpCorrection() 180 0.004416 0.004416 0.004416 0.68% MLCellLinOp::apply() 500 0.0044 0.0044 0.0044 0.68% amrex::average_down 180 0.004159 0.004159 0.004159 0.64% FabArray::setDomainBndry() 20 0.004112 0.004112 0.004112 0.64% Castro::estTimeStep() 10 0.004024 0.004024 0.004024 0.62% CGSolver::sxay() 690 0.003988 0.003988 0.003988 0.62% MultiFab::Xpay() 258 0.003957 0.003957 0.003957 0.61% Castro::do_advance_ctu() 5 0.002969 0.002969 0.002969 0.46% FabArrayBase::getCPC() 632 0.002833 0.002833 0.002833 0.44% MLCGSolver::ParallelAllReduce 659 0.002748 0.002748 0.002748 0.43% Castro::reset_internal_energy() 30 0.002581 0.002581 0.002581 0.40% BndryData::define() 6 0.002478 0.002478 0.002478 0.38% Castro::enforce_speed_limit() 30 0.002255 0.002255 0.002255 0.35% FabArray::ParallelCopy() 380 0.002233 0.002233 0.002233 0.35% FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001991 0.001991 0.001991 0.31% MLCellLinOp::correctionResidual() 216 0.001867 0.001867 0.001867 0.29% Gravity::get_new_grav_vector() 5 0.001854 0.001854 0.001854 0.29% Castro::construct_new_gravity_source() 5 0.00169 0.00169 0.00169 0.26% MLCellLinOp::compGrad() 6 0.001559 0.001559 0.001559 0.24% Castro::normalize_species() 30 0.00147 0.00147 0.00147 0.23% Castro::construct_old_gravity_source() 5 0.001415 0.001415 0.001415 0.22% MLMG::mgVcycle() 36 0.001363 0.001363 0.001363 0.21% FabArrayBase::CPC::define() 244 0.001089 0.001089 0.001089 0.17% MLMG::ResNormInf() 42 0.00105 0.00105 0.00105 0.16% MLMG:computeResOfCorrection() 180 0.001031 0.001031 0.001031 0.16% MLMG::oneIter() 36 0.001027 0.001027 0.001027 0.16% Gravity::get_old_grav_vector() 5 0.001023 0.001023 0.001023 0.16% MultiFab::Saxpy() 10 0.0009758 0.0009758 0.0009758 0.15% MLCellLinOp::setLevelBC() 6 0.0009304 0.0009304 0.0009304 0.14% Castro::post_timestep() 5 0.0009111 0.0009111 0.0009111 0.14% MLMG::prepareForSolve() 6 0.0008882 0.0008882 0.0008882 0.14% Castro::create_source_corrector() 5 0.0008755 0.0008755 0.0008755 0.14% Gravity::actual_solve_with_mlmg() 6 0.0008628 0.0008628 0.0008628 0.13% MLCellLinOp::prepareForSolve() 6 0.0007933 0.0007933 0.0007933 0.12% FabArray::mult() 22 0.0007621 0.0007621 0.0007621 0.12% MLMG::mgVcycle_down::0 36 0.0006523 0.0006523 0.0006523 0.10% Gravity::update_max_rhs() 6 0.0006485 0.0006485 0.0006485 0.10% MLMG::mgVcycle_down::4 36 0.0006432 0.0006432 0.0006432 0.10% MLMG::mgVcycle_down::2 36 0.0006313 0.0006313 0.0006313 0.10% MLMG::mgVcycle_down::3 36 0.0006277 0.0006277 0.0006277 0.10% MLMG::mgVcycle_down::1 36 0.0006194 0.0006194 0.0006194 0.10% MLMG::actualBottomSolve() 36 0.0005498 0.0005498 0.0005498 0.09% MLMG::solve() 6 0.0004684 0.0004684 0.0004684 0.07% Castro::clean_state() 30 0.0004617 0.0004617 0.0004617 0.07% Amr::InitAmr() 1 0.0004355 0.0004355 0.0004355 0.07% MLMG::mgVcycle_up::3 36 0.0004264 0.0004264 0.0004264 0.07% MLMG::mgVcycle_up::2 36 0.0004254 0.0004254 0.0004254 0.07% MultiFab::Copy() 6 0.0004173 0.0004173 0.0004173 0.06% MLMG::mgVcycle_up::4 36 0.0004163 0.0004163 0.0004163 0.06% MLMG::mgVcycle_up::0 36 0.0004126 0.0004126 0.0004126 0.06% MLMG::mgVcycle_up::1 36 0.0004104 0.0004104 0.0004104 0.06% MLCellLinOp::solutionResidual() 42 0.0003831 0.0003831 0.0003831 0.06% main() 1 0.0003343 0.0003343 0.0003343 0.05% Castro::construct_new_gravity() 5 0.0002959 0.0002959 0.0002959 0.05% FillPatchIterator::Initialize 20 0.0002948 0.0002948 0.0002948 0.05% Gravity::solve_for_phi() 5 0.0002842 0.0002842 0.0002842 0.04% Castro::initialize_advance() 5 0.0002808 0.0002808 0.0002808 0.04% Amr::coarseTimeStep() 5 0.0002491 0.0002491 0.0002491 0.04% Gravity::actual_multilevel_solve() 1 0.0002456 0.0002456 0.0002456 0.04% Castro::do_new_sources() 5 0.0002319 0.0002319 0.0002319 0.04% MLMG::computeResidual() 36 0.0002107 0.0002107 0.0002107 0.03% MLMG::mgVcycle_bottom 36 0.0002096 0.0002096 0.0002096 0.03% Castro::construct_old_source() 25 0.0002035 0.0002035 0.0002035 0.03% MultiFab::min() 5 0.0001848 0.0001848 0.0001848 0.03% Castro::subcycle_advance_ctu() 5 0.0001811 0.0001811 0.0001811 0.03% makeSFC 30 0.0001804 0.0001804 0.0001804 0.03% FillPatchSingleLevel 20 0.0001788 0.0001788 0.0001788 0.03% Castro::construct_new_source() 25 0.0001704 0.0001704 0.0001704 0.03% MultiFab::max() 6 0.0001644 0.0001644 0.0001644 0.03% MLLinOp::defineGrids() 6 0.0001515 0.0001515 0.0001515 0.02% Castro::do_old_sources() 5 0.0001451 0.0001451 0.0001451 0.02% MLCellLinOp::defineBC() 6 0.0001399 0.0001399 0.0001399 0.02% Amr::timeStep() 5 0.000138 0.000138 0.000138 0.02% FabArrayBase::FB::FB() 26 0.0001356 0.0001356 0.0001356 0.02% MLMG::MLRhsNormInf() 6 0.0001276 0.0001276 0.0001276 0.02% Castro::advance() 5 0.0001158 0.0001158 0.0001158 0.02% MLLinOp::makeAgglomeratedDMap 6 0.0001068 0.0001068 0.0001068 0.02% StateData::restartDoit() 4 9.929e-05 9.929e-05 9.929e-05 0.02% Castro::initialize_do_advance() 5 9.879e-05 9.879e-05 9.879e-05 0.02% AmrLevel::restart() 1 9.604e-05 9.604e-05 9.604e-05 0.01% DistributionMapping::Distribute() 31 9.375e-05 9.375e-05 9.375e-05 0.01% MLPoisson::define() 6 8.312e-05 8.312e-05 8.312e-05 0.01% Castro::construct_old_gravity() 5 8.229e-05 8.229e-05 8.229e-05 0.01% Gravity::solve_phi_with_mlmg() 6 8.175e-05 8.175e-05 8.175e-05 0.01% Castro::apply_source_to_state() 10 7.992e-05 7.992e-05 7.992e-05 0.01% Castro::finalize_advance() 5 7.592e-05 7.592e-05 7.592e-05 0.01% Gravity::swapTimeLevels() 5 5.929e-05 5.929e-05 5.929e-05 0.01% MLLinOp::define() 6 4.388e-05 4.388e-05 4.388e-05 0.01% Castro::swap_state_time_levels() 5 4.154e-05 4.154e-05 4.154e-05 0.01% Castro::post_restart() 1 3.8e-05 3.8e-05 3.8e-05 0.01% MLPoisson::prepareForSolve() 6 3.758e-05 3.758e-05 3.758e-05 0.01% MLMG::computeMLResidual() 6 3.725e-05 3.725e-05 3.725e-05 0.01% MLMG::getGradSolution() 6 3.575e-05 3.575e-05 3.575e-05 0.01% MLMG::MLResNormInf() 6 3.537e-05 3.537e-05 3.537e-05 0.01% Castro::finalize_do_advance() 5 3.262e-05 3.262e-05 3.262e-05 0.01% Castro::computeNewDt() 5 3.158e-05 3.158e-05 3.158e-05 0.00% Gravity::multilevel_solve_for_new_phi() 1 2.242e-05 2.242e-05 2.242e-05 0.00% Gravity::set_mass_offset() 6 2.046e-05 2.046e-05 2.046e-05 0.00% MLMG::buildFineMask() 6 2.004e-05 2.004e-05 2.004e-05 0.00% MLLinOp::makeSubCommunicator() 6 1.775e-05 1.775e-05 1.775e-05 0.00% Castro::retry_advance_ctu() 5 1.55e-05 1.55e-05 1.55e-05 0.00% Castro::FluxRegFineAdd() 5 1.525e-05 1.525e-05 1.525e-05 0.00% Amr::writeSmallPlotFile() 1 1.408e-05 1.408e-05 1.408e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 9.501e-06 9.501e-06 9.501e-06 0.00% Amr::initSubcycle() 1 8.583e-06 8.583e-06 8.583e-06 0.00% Amr::init() 1 6.75e-06 6.75e-06 6.75e-06 0.00% AmrLevel::AmrLevel() 1 3.584e-06 3.584e-06 3.584e-06 0.00% -------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % -------------------------------------------------------------------------------------------- main() 1 0.6457 0.6457 0.6457 100.00% Amr::coarseTimeStep() 5 0.4874 0.4874 0.4874 75.48% Amr::timeStep() 5 0.4855 0.4855 0.4855 75.18% Castro::advance() 5 0.4805 0.4805 0.4805 74.41% Castro::subcycle_advance_ctu() 5 0.4681 0.4681 0.4681 72.49% Castro::do_advance_ctu() 5 0.4679 0.4679 0.4679 72.46% Castro::construct_new_gravity() 5 0.265 0.265 0.265 41.04% Gravity::solve_phi_with_mlmg() 6 0.2611 0.2611 0.2611 40.43% Gravity::actual_solve_with_mlmg() 6 0.2536 0.2536 0.2536 39.27% Gravity::solve_for_phi() 5 0.2534 0.2534 0.2534 39.25% MLMG::solve() 6 0.2346 0.2346 0.2346 36.34% MLMG::oneIter() 36 0.2221 0.2221 0.2221 34.39% MLMG::mgVcycle() 36 0.221 0.221 0.221 34.23% Castro::construct_ctu_hydro_source() 5 0.1499 0.1499 0.1499 23.22% MLCellLinOp::smooth() 720 0.1083 0.1083 0.1083 16.77% Amr::writePlotFile() 1 0.09808 0.09808 0.09808 15.19% MLCellLinOp::applyBC() 1946 0.08903 0.08903 0.08903 13.79% MLMG::mgVcycle_bottom 36 0.06531 0.06531 0.06531 10.11% MLMG::actualBottomSolve() 36 0.0651 0.0651 0.0651 10.08% MLCGSolver::bicgstab 36 0.06409 0.06409 0.06409 9.93% Amr::init() 1 0.05943 0.05943 0.05943 9.20% Amr::restart() 1 0.05943 0.05943 0.05943 9.20% AmrLevel::restart() 1 0.04593 0.04593 0.04593 7.11% StateData::restartDoit() 4 0.04582 0.04582 0.04582 7.10% VisMF::Read() 3 0.0448 0.0448 0.0448 6.94% MLCellLinOp::apply() 500 0.03479 0.03479 0.03479 5.39% FabArray::FillBoundary() 1766 0.03401 0.03401 0.03401 5.27% MLPoisson::Fsmooth() 1440 0.03278 0.03278 0.03278 5.08% FillPatchIterator::Initialize 20 0.02995 0.02995 0.02995 4.64% VisMF::Write(FabArray) 1 0.02561 0.02561 0.02561 3.97% FillPatchSingleLevel 20 0.02554 0.02554 0.02554 3.96% MLMG::mgVcycle_down::0 36 0.02498 0.02498 0.02498 3.87% FillBoundary_nowait() 1766 0.02387 0.02387 0.02387 3.70% StateDataPhysBCFunct::() 20 0.02289 0.02289 0.02289 3.54% MLMG::mgVcycle_up::0 36 0.02097 0.02097 0.02097 3.25% MLCellLinOp::correctionResidual() 216 0.02 0.02 0.02 3.10% FabArray::setVal() 533 0.01909 0.01909 0.01909 2.96% Castro::clean_state() 30 0.01901 0.01901 0.01901 2.94% MLMG:computeResOfCorrection() 180 0.0179 0.0179 0.0179 2.77% Castro::initialize_do_advance() 5 0.01653 0.01653 0.01653 2.56% MLMG::mgVcycle_down::1 36 0.01561 0.01561 0.01561 2.42% MLMG::mgVcycle_down::2 36 0.01553 0.01553 0.01553 2.40% MLPoisson::define() 6 0.01508 0.01508 0.01508 2.33% MLMG::mgVcycle_down::3 36 0.015 0.015 0.015 2.32% MLMG::mgVcycle_down::4 36 0.01492 0.01492 0.01492 2.31% FabArray::ParallelCopy() 380 0.01433 0.01433 0.01433 2.22% Castro::expand_state() 5 0.01342 0.01342 0.01342 2.08% CGSolver::sxay() 690 0.01278 0.01278 0.01278 1.98% StateData::FillBoundary(geom) 160 0.01249 0.01249 0.01249 1.93% MultiFab::Dot() 484 0.01223 0.01223 0.01223 1.89% Castro::initialize_advance() 5 0.01217 0.01217 0.01217 1.88% FabArray::ParallelCopy_nowait() 380 0.0121 0.0121 0.0121 1.87% MLMG::mgVcycle_up::2 36 0.01209 0.01209 0.01209 1.87% MLMG::mgVcycle_up::1 36 0.01187 0.01187 0.01187 1.84% MLMG::mgVcycle_up::4 36 0.01176 0.01176 0.01176 1.82% MLMG::mgVcycle_up::3 36 0.01163 0.01163 0.01163 1.80% Gravity::get_new_grav_vector() 5 0.01127 0.01127 0.01127 1.75% MLCellLinOp::defineAuxData() 6 0.01082 0.01082 0.01082 1.68% MLMG::addInterpCorrection() 180 0.01035 0.01035 0.01035 1.60% amrex::average_down 180 0.01008 0.01008 0.01008 1.56% MLCGSolver::ParallelAllReduce 659 0.009936 0.009936 0.009936 1.54% Castro::construct_old_gravity() 5 0.009795 0.009795 0.009795 1.52% Castro::computeTemp() 30 0.009787 0.009787 0.009787 1.52% Gravity::get_old_grav_vector() 5 0.009713 0.009713 0.009713 1.50% MultiFab::LinComb() 690 0.008797 0.008797 0.008797 1.36% Castro::post_restart() 1 0.008662 0.008662 0.008662 1.34% Gravity::multilevel_solve_for_new_phi() 1 0.008188 0.008188 0.008188 1.27% Gravity::actual_multilevel_solve() 1 0.008166 0.008166 0.008166 1.26% Castro::do_new_sources() 5 0.007451 0.007451 0.007451 1.15% Gravity::fill_multipole_BCs() 6 0.007264 0.007264 0.007264 1.12% MLPoisson::Fapply() 500 0.00693 0.00693 0.00693 1.07% MLCellLinOp::solutionResidual() 42 0.005651 0.005651 0.005651 0.88% FabArrayBase::getFB() 1766 0.005382 0.005382 0.005382 0.83% Castro::enforce_min_density() 30 0.00504 0.00504 0.00504 0.78% MLMG::prepareForSolve() 6 0.005003 0.005003 0.005003 0.77% Castro::do_old_sources() 5 0.004996 0.004996 0.004996 0.77% MLMG::computeResidual() 36 0.004937 0.004937 0.004937 0.76% Castro::post_timestep() 5 0.004841 0.004841 0.004841 0.75% FabArray::setDomainBndry() 20 0.004112 0.004112 0.004112 0.64% Castro::estTimeStep() 10 0.004024 0.004024 0.004024 0.62% MultiFab::Xpay() 258 0.003957 0.003957 0.003957 0.61% FabArrayBase::getCPC() 632 0.003922 0.003922 0.003922 0.61% MLCellLinOp::defineBC() 6 0.003586 0.003586 0.003586 0.56% BndryData::define() 6 0.003446 0.003446 0.003446 0.53% Castro::reset_internal_energy() 30 0.002581 0.002581 0.002581 0.40% Castro::enforce_speed_limit() 30 0.002255 0.002255 0.002255 0.35% MLMG::getGradSolution() 6 0.002057 0.002057 0.002057 0.32% MLCellLinOp::compGrad() 6 0.002022 0.002022 0.002022 0.31% FabArray::setVal(val, thecmd, scomp, ncomp) 252 0.001991 0.001991 0.001991 0.31% Castro::construct_new_source() 25 0.00186 0.00186 0.00186 0.29% Castro::construct_new_gravity_source() 5 0.00169 0.00169 0.00169 0.26% Castro::computeNewDt() 5 0.001663 0.001663 0.001663 0.26% Castro::construct_old_source() 25 0.001619 0.001619 0.001619 0.25% Castro::normalize_species() 30 0.00147 0.00147 0.00147 0.23% Castro::construct_old_gravity_source() 5 0.001415 0.001415 0.001415 0.22% Gravity::swapTimeLevels() 5 0.001105 0.001105 0.001105 0.17% FabArrayBase::CPC::define() 244 0.001089 0.001089 0.001089 0.17% Castro::apply_source_to_state() 10 0.001056 0.001056 0.001056 0.16% MLMG::ResNormInf() 42 0.00105 0.00105 0.00105 0.16% MultiFab::Saxpy() 10 0.0009758 0.0009758 0.0009758 0.15% MLMG::computeMLResidual() 6 0.0009612 0.0009612 0.0009612 0.15% MLCellLinOp::setLevelBC() 6 0.0009304 0.0009304 0.0009304 0.14% Gravity::update_max_rhs() 6 0.0009178 0.0009178 0.0009178 0.14% Castro::create_source_corrector() 5 0.0008755 0.0008755 0.0008755 0.14% MLPoisson::prepareForSolve() 6 0.0008309 0.0008309 0.0008309 0.13% MLCellLinOp::prepareForSolve() 6 0.0007933 0.0007933 0.0007933 0.12% FabArray::mult() 22 0.0007621 0.0007621 0.0007621 0.12% MLLinOp::define() 6 0.0005901 0.0005901 0.0005901 0.09% MLLinOp::defineGrids() 6 0.0005462 0.0005462 0.0005462 0.08% Amr::InitAmr() 1 0.000444 0.000444 0.000444 0.07% MultiFab::Copy() 6 0.0004173 0.0004173 0.0004173 0.06% MLLinOp::makeAgglomeratedDMap 6 0.000377 0.000377 0.000377 0.06% makeSFC 30 0.0002702 0.0002702 0.0002702 0.04% MLMG::MLResNormInf() 6 0.0001945 0.0001945 0.0001945 0.03% MultiFab::min() 5 0.0001848 0.0001848 0.0001848 0.03% MultiFab::max() 6 0.0001644 0.0001644 0.0001644 0.03% FabArrayBase::FB::FB() 26 0.0001356 0.0001356 0.0001356 0.02% MLMG::MLRhsNormInf() 6 0.0001276 0.0001276 0.0001276 0.02% DistributionMapping::Distribute() 31 9.375e-05 9.375e-05 9.375e-05 0.01% Castro::finalize_advance() 5 9.117e-05 9.117e-05 9.117e-05 0.01% Castro::swap_state_time_levels() 5 4.154e-05 4.154e-05 4.154e-05 0.01% Castro::finalize_do_advance() 5 3.262e-05 3.262e-05 3.262e-05 0.01% Gravity::set_mass_offset() 6 2.046e-05 2.046e-05 2.046e-05 0.00% MLMG::buildFineMask() 6 2.004e-05 2.004e-05 2.004e-05 0.00% MLLinOp::makeSubCommunicator() 6 1.775e-05 1.775e-05 1.775e-05 0.00% Castro::retry_advance_ctu() 5 1.55e-05 1.55e-05 1.55e-05 0.00% Castro::FluxRegFineAdd() 5 1.525e-05 1.525e-05 1.525e-05 0.00% Amr::writeSmallPlotFile() 1 1.408e-05 1.408e-05 1.408e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 1.346e-05 1.346e-05 1.346e-05 0.00% Amr::initSubcycle() 1 8.583e-06 8.583e-06 8.583e-06 0.00% AmrLevel::AmrLevel() 1 3.584e-06 3.584e-06 3.584e-06 0.00% -------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2] [TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2] Total GPU global memory (MB): 12066 Free GPU global memory (MB): 10622 [The Arena] space allocated (MB): 9049 [The Arena] space used (MB): 0 [The Pinned Arena] space allocated (MB): 8 [The Pinned Arena] space used (MB): 0 AMReX (22.01-1-ga73ce6759c91) finalized