Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.03-21-g336ca7002b45) initialized

Starting run at 08:25:47 UTC on 2022-03-22.
Successfully read inputs file ... 

Castro git describe: 22.02-13-ga12efaab3
AMReX git describe: 22.03-21-g336ca7002
Microphysics git describe: 22.03-8-g75721f58

reading extern runtime parameters ...
3 Species: 
C12  O16  Mg24  
Successfully read inputs file ... 
INITIAL GRIDS 
  Level 0   8 grids  262144 cells  100 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

CHECKPOINT: file = dustcollapse-restart_chk00000
checkPoint() time = 0.04080485 secs.
PLOTFILE: file = dustcollapse-restart_plt00000
Write plotfile time = 0.024708779  seconds

[Level 0 step 1] ADVANCE with dt = 4.541742215e-05
[Level 0 step 1] Advanced 262144 cells

[STEP 1] Coarse TimeStep time: 0.108220824
[STEP 1] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05

[Level 0 step 2] ADVANCE with dt = 4.768829326e-05
[Level 0 step 2] Advanced 262144 cells

[STEP 2] Coarse TimeStep time: 0.049067814
[STEP 2] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05

[Level 0 step 3] ADVANCE with dt = 5.007270792e-05
[Level 0 step 3] Advanced 262144 cells

[STEP 3] Coarse TimeStep time: 0.050872298
[STEP 3] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05

[Level 0 step 4] ADVANCE with dt = 5.257634331e-05
[Level 0 step 4] Advanced 262144 cells

[STEP 4] Coarse TimeStep time: 0.06200422
[STEP 4] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05

[Level 0 step 5] ADVANCE with dt = 5.520516048e-05
[Level 0 step 5] Advanced 262144 cells

[STEP 5] Coarse TimeStep time: 0.061025845
[STEP 5] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05

CHECKPOINT: file = dustcollapse-restart_chk00005
checkPoint() time = 0.038613388 secs.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells

[STEP 6] Coarse TimeStep time: 0.050198498
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05

[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells

[STEP 7] Coarse TimeStep time: 0.049809487
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05

[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells

[STEP 8] Coarse TimeStep time: 0.055818042
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05

[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells

[STEP 9] Coarse TimeStep time: 0.059721926
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05

[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells

[STEP 10] Coarse TimeStep time: 0.06539589
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05

CHECKPOINT: file = dustcollapse-restart_chk00010
checkPoint() time = 0.038666011 secs.
PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.024314093  seconds


Ending run at 08:25:48 UTC on 2022-03-22.
Run time = 0.841963359
Run time without initialization = 0.714194518

  Average number of zones advanced per microsecond: 3.670
  Average number of zones advanced per microsecond per rank: 3.670

CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728


TinyProfiler total time across processes [min...avg...max]: 0.842 ... 0.842 ... 0.842

--------------------------------------------------------------------------------------------
Name                                         NCalls  Excl. Min  Excl. Avg  Excl. Max   Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source()             10     0.2110     0.2110     0.2110  25.06%
VisMF::Write(FabArray)                           11     0.1602     0.1602     0.1602  19.02%
MLCellLinOp::applyBC()                         4433     0.0947     0.0947     0.0947  11.25%
MLPoisson::Fsmooth()                           3280    0.06017    0.06017    0.06017   7.15%
FabArray::setVal()                             1144    0.02598    0.02598    0.02598   3.08%
StateData::FillBoundary(geom)                   328    0.02464    0.02464    0.02464   2.93%
MLCGSolver::bicgstab                             82    0.02202    0.02202    0.02202   2.62%
MultiFab::Dot()                                1114      0.022      0.022      0.022   2.61%
FillBoundary_nowait()                          4023    0.01773    0.01773    0.01773   2.11%
FabArray::ParallelCopy_nowait()                 861    0.01621    0.01621    0.01621   1.93%
StateDataPhysBCFunct::()                         41    0.01459    0.01459    0.01459   1.73%
Castro::computeTemp()                            63    0.01402    0.01402    0.01402   1.66%
MultiFab::LinComb()                            1586    0.01254    0.01254    0.01254   1.49%
Gravity::fill_multipole_BCs()                    11    0.01179    0.01179    0.01179   1.40%
Castro::enforce_min_density()                    62    0.01161    0.01161    0.01161   1.38%
MLPoisson::Fapply()                            1142    0.01076    0.01076    0.01076   1.28%
MLCellLinOp::defineAuxData()                     11    0.01031    0.01031    0.01031   1.22%
MLMG::addInterpCorrection()                     410    0.00714    0.00714    0.00714   0.85%
amrex::average_down                             410   0.006675   0.006675   0.006675   0.79%
MultiFab::Xpay()                                585   0.006171   0.006171   0.006171   0.73%
FabArray::setDomainBndry()                       41   0.005543   0.005543   0.005543   0.66%
Castro::reset_internal_energy()                  63   0.005508   0.005508   0.005508   0.65%
Castro::estTimeStep()                            21   0.005435   0.005435   0.005435   0.65%
Castro::expand_state()                           10   0.004671   0.004671   0.004671   0.55%
Castro::enforce_speed_limit()                    62    0.00407    0.00407    0.00407   0.48%
Amr::checkPoint()                                 3   0.003821   0.003821   0.003821   0.45%
BndryData::define()                              11   0.003795   0.003795   0.003795   0.45%
Castro::do_advance_ctu()                         10   0.003225   0.003225   0.003225   0.38%
Amr::writePlotFile()                              2   0.003142   0.003142   0.003142   0.37%
Castro::normalize_species()                      62   0.002681   0.002681   0.002681   0.32%
Castro::construct_new_gravity_source()           10   0.002557   0.002557   0.002557   0.30%
Gravity::get_new_grav_vector()                   11   0.002384   0.002384   0.002384   0.28%
FabArray::setVal(val, thecmd, scomp, ncomp)     462    0.00234    0.00234    0.00234   0.28%
MLCellLinOp::compGrad()                          11   0.002112   0.002112   0.002112   0.25%
Castro::construct_old_gravity_source()           10   0.002061   0.002061   0.002061   0.24%
MLMG::ResNormInf()                               93   0.001901   0.001901   0.001901   0.23%
MultiFab::Saxpy()                                20   0.001816   0.001816   0.001816   0.22%
Gravity::get_old_grav_vector()                   10   0.001769   0.001769   0.001769   0.21%
MLMG::oneIter()                                  82    0.00167    0.00167    0.00167   0.20%
MLCellLinOp::setLevelBC()                        11   0.001546   0.001546   0.001546   0.18%
Gravity::actual_solve_with_mlmg()                11   0.001392   0.001392   0.001392   0.17%
MLCellLinOp::prepareForSolve()                   11   0.001375   0.001375   0.001375   0.16%
FabArray::mult()                                 43   0.001329   0.001329   0.001329   0.16%
Castro::initData()                                1   0.001238   0.001238   0.001238   0.15%
MLCellLinOp::smooth()                          1640   0.001156   0.001156   0.001156   0.14%
FabArray::FillBoundary()                       4023   0.000829   0.000829   0.000829   0.10%
FabArrayBase::getCPC()                         1323  0.0008282  0.0008282  0.0008282   0.10%
MLMG::prepareForSolve()                          11   0.000759   0.000759   0.000759   0.09%
FabArrayBase::CPC::define()                     454  0.0007019  0.0007019  0.0007019   0.08%
FabArrayBase::getFB()                          4023  0.0006705  0.0006705  0.0006705   0.08%
Gravity::update_max_rhs()                        11   0.000569   0.000569   0.000569   0.07%
MultiFab::Copy()                                 11  0.0005203  0.0005203  0.0005203   0.06%
MLCellLinOp::apply()                           1142  0.0005009  0.0005009  0.0005009   0.06%
Amr::InitAmr()                                    1  0.0004607  0.0004607  0.0004607   0.05%
Gravity::solve_for_phi()                         10  0.0004429  0.0004429  0.0004429   0.05%
CGSolver::sxay()                               1586  0.0004405  0.0004405  0.0004405   0.05%
MLMG::mgVcycle()                                 82  0.0004101  0.0004101  0.0004101   0.05%
MLCGSolver::ParallelAllReduce                  1514  0.0003274  0.0003274  0.0003274   0.04%
MultiFab::min()                                  10  0.0003199  0.0003199  0.0003199   0.04%
main()                                            1  0.0002989  0.0002989  0.0002989   0.04%
FabArray::ParallelCopy()                        861  0.0002942  0.0002942  0.0002942   0.03%
Castro::construct_new_gravity()                  10  0.0002633  0.0002633  0.0002633   0.03%
MultiFab::max()                                  11  0.0002524  0.0002524  0.0002524   0.03%
FillPatchIterator::Initialize                    41    0.00025    0.00025    0.00025   0.03%
Gravity::actual_multilevel_solve()                1  0.0002251  0.0002251  0.0002251   0.03%
MLCellLinOp::correctionResidual()               492  0.0002111  0.0002111  0.0002111   0.03%
Amr::coarseTimeStep()                            10  0.0002013  0.0002013  0.0002013   0.02%
MLMG::MLRhsNormInf()                             11  0.0002011  0.0002011  0.0002011   0.02%
MLCellLinOp::defineBC()                          11  0.0001777  0.0001777  0.0001777   0.02%
Amr::timeStep()                                  10  0.0001751  0.0001751  0.0001751   0.02%
MLLinOp::defineGrids()                           11  0.0001708  0.0001708  0.0001708   0.02%
Castro::subcycle_advance_ctu()                   10   0.000144   0.000144   0.000144   0.02%
Amr::defBaseLevel()                               1  0.0001429  0.0001429  0.0001429   0.02%
MLMG:computeResOfCorrection()                   410  0.0001332  0.0001332  0.0001332   0.02%
StateData::checkPoint()                          12  0.0001256  0.0001256  0.0001256   0.01%
MLMG::actualBottomSolve()                        82  0.0001068  0.0001068  0.0001068   0.01%
MLMG::mgVcycle_down::0                           82  9.474e-05  9.474e-05  9.474e-05   0.01%
FabArrayBase::FB::FB()                           56  9.138e-05  9.138e-05  9.138e-05   0.01%
MLMG::mgVcycle_down::1                           82  8.521e-05  8.521e-05  8.521e-05   0.01%
Castro::construct_new_source()                   50  8.279e-05  8.279e-05  8.279e-05   0.01%
MLMG::mgVcycle_down::2                           82  8.055e-05  8.055e-05  8.055e-05   0.01%
MLMG::mgVcycle_down::4                           82   7.83e-05   7.83e-05   7.83e-05   0.01%
Castro::initialize_advance()                     10  7.808e-05  7.808e-05  7.808e-05   0.01%
MLMG::solve()                                    11  7.685e-05  7.685e-05  7.685e-05   0.01%
Castro::advance()                                10  7.682e-05  7.682e-05  7.682e-05   0.01%
MLMG::mgVcycle_down::3                           82  7.652e-05  7.652e-05  7.652e-05   0.01%
Castro::post_timestep()                          10   7.52e-05   7.52e-05   7.52e-05   0.01%
Castro::clean_state()                            62  7.339e-05  7.339e-05  7.339e-05   0.01%
AmrLevel::checkPoint()                            3  6.963e-05  6.963e-05  6.963e-05   0.01%
MLMG::mgVcycle_up::4                             82  6.086e-05  6.086e-05  6.086e-05   0.01%
Castro::initialize_do_advance()                  10  5.964e-05  5.964e-05  5.964e-05   0.01%
Castro::finalize_advance()                       10  5.781e-05  5.781e-05  5.781e-05   0.01%
MLMG::mgVcycle_up::0                             82  5.326e-05  5.326e-05  5.326e-05   0.01%
StateData::define()                               4  5.291e-05  5.291e-05  5.291e-05   0.01%
MLMG::mgVcycle_up::3                             82  4.916e-05  4.916e-05  4.916e-05   0.01%
MLCellLinOp::solutionResidual()                  93  4.904e-05  4.904e-05  4.904e-05   0.01%
MLMG::mgVcycle_up::1                             82  4.817e-05  4.817e-05  4.817e-05   0.01%
MLMG::mgVcycle_up::2                             82  4.604e-05  4.604e-05  4.604e-05   0.01%
Castro::finalize_do_advance()                    10  3.698e-05  3.698e-05  3.698e-05   0.00%
Castro::swap_state_time_levels()                 10   3.57e-05   3.57e-05   3.57e-05   0.00%
Castro::enforce_consistent_e()                    1  3.564e-05  3.564e-05  3.564e-05   0.00%
MLMG::mgVcycle_bottom                            82  3.517e-05  3.517e-05  3.517e-05   0.00%
MLMG::computeResidual()                          82  3.273e-05  3.273e-05  3.273e-05   0.00%
FillPatchSingleLevel                             41  2.814e-05  2.814e-05  2.814e-05   0.00%
Gravity::solve_phi_with_mlmg()                   11  2.813e-05  2.813e-05  2.813e-05   0.00%
MLLinOp::define()                                11  2.768e-05  2.768e-05  2.768e-05   0.00%
makeSFC                                          55  2.666e-05  2.666e-05  2.666e-05   0.00%
Amr::writeSmallPlotFile()                         1  2.476e-05  2.476e-05  2.476e-05   0.00%
MLPoisson::define()                              11   2.23e-05   2.23e-05   2.23e-05   0.00%
Amr::FinalizeInit()                               1   1.89e-05   1.89e-05   1.89e-05   0.00%
Castro::construct_old_source()                   50  1.781e-05  1.781e-05  1.781e-05   0.00%
Castro::do_new_sources()                         10   1.71e-05   1.71e-05   1.71e-05   0.00%
Gravity::multilevel_solve_for_new_phi()           1  1.642e-05  1.642e-05  1.642e-05   0.00%
MLLinOp::makeAgglomeratedDMap                    11  1.627e-05  1.627e-05  1.627e-05   0.00%
Castro::do_old_sources()                         10   1.52e-05   1.52e-05   1.52e-05   0.00%
DistributionMapping::Distribute()                56  1.503e-05  1.503e-05  1.503e-05   0.00%
Castro::apply_source_to_state()                  20  1.153e-05  1.153e-05  1.153e-05   0.00%
Castro::construct_old_gravity()                  10   1.09e-05   1.09e-05   1.09e-05   0.00%
AmrLevel::AmrLevel(dm)                            1   9.44e-06   9.44e-06   9.44e-06   0.00%
Amr::initSubcycle()                               1  9.363e-06  9.363e-06  9.363e-06   0.00%
Gravity::swapTimeLevels()                        10  8.893e-06  8.893e-06  8.893e-06   0.00%
MLPoisson::prepareForSolve()                     11  8.654e-06  8.654e-06  8.654e-06   0.00%
MLMG::computeMLResidual()                        11  7.082e-06  7.082e-06  7.082e-06   0.00%
MLMG::buildFineMask()                            11  6.066e-06  6.066e-06  6.066e-06   0.00%
MLMG::getGradSolution()                          11  5.957e-06  5.957e-06  5.957e-06   0.00%
Castro::create_source_corrector()                10   5.92e-06   5.92e-06   5.92e-06   0.00%
Amr::InitializeInit()                             1  5.837e-06  5.837e-06  5.837e-06   0.00%
Castro::computeNewDt()                            9  5.827e-06  5.827e-06  5.827e-06   0.00%
MLMG::MLResNormInf()                             11  4.873e-06  4.873e-06  4.873e-06   0.00%
AmrLevel::checkPointPost()                        3  4.773e-06  4.773e-06  4.773e-06   0.00%
Castro::post_init()                               1  4.439e-06  4.439e-06  4.439e-06   0.00%
Gravity::set_mass_offset()                       11  4.244e-06  4.244e-06  4.244e-06   0.00%
Castro::retry_advance_ctu()                      10  3.793e-06  3.793e-06  3.793e-06   0.00%
Castro::FluxRegFineAdd()                         10  3.499e-06  3.499e-06  3.499e-06   0.00%
DistributionMapping::SFCProcessorMapDoIt()        1   3.36e-06   3.36e-06   3.36e-06   0.00%
Amr::init()                                       1  2.511e-06  2.511e-06  2.511e-06   0.00%
Castro::computeInitialDt()                        2  2.473e-06  2.473e-06  2.473e-06   0.00%
MLLinOp::makeSubCommunicator()                   11  2.032e-06  2.032e-06  2.032e-06   0.00%
AmrLevel::checkPointPre()                         3  1.873e-06  1.873e-06  1.873e-06   0.00%
Castro::post_regrid()                             1  1.469e-06  1.469e-06  1.469e-06   0.00%
Amr::initialInit()                                1  1.031e-06  1.031e-06  1.031e-06   0.00%
--------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------
Name                                         NCalls  Incl. Min  Incl. Avg  Incl. Max   Max %
--------------------------------------------------------------------------------------------
main()                                            1      0.842      0.842      0.842 100.00%
Amr::coarseTimeStep()                            10     0.6897     0.6897     0.6897  81.91%
Amr::timeStep()                                  10     0.6093     0.6093     0.6093  72.36%
Castro::advance()                                10     0.6019     0.6019     0.6019  71.49%
Castro::subcycle_advance_ctu()                   10     0.5881     0.5881     0.5881  69.85%
Castro::do_advance_ctu()                         10      0.588      0.588      0.588  69.83%
Gravity::solve_phi_with_mlmg()                   11     0.3294     0.3294     0.3294  39.12%
Gravity::actual_solve_with_mlmg()                11     0.3173     0.3173     0.3173  37.69%
Castro::construct_new_gravity()                  10     0.2981     0.2981     0.2981  35.40%
MLMG::solve()                                    11     0.2942     0.2942     0.2942  34.94%
Gravity::solve_for_phi()                         10     0.2807     0.2807     0.2807  33.33%
MLMG::oneIter()                                  82     0.2781     0.2781     0.2781  33.03%
MLMG::mgVcycle()                                 82     0.2765     0.2765     0.2765  32.83%
Castro::construct_ctu_hydro_source()             10      0.211      0.211      0.211  25.06%
VisMF::Write(FabArray)                           11     0.1602     0.1602     0.1602  19.02%
MLCellLinOp::smooth()                          1640      0.146      0.146      0.146  17.34%
Amr::init()                                       1     0.1272     0.1272     0.1272  15.10%
Amr::checkPoint()                                 3     0.1182     0.1182     0.1182  14.04%
AmrLevel::checkPoint()                            3     0.1144     0.1144     0.1144  13.58%
StateData::checkPoint()                          12     0.1143     0.1143     0.1143  13.57%
MLCellLinOp::applyBC()                         4433      0.114      0.114      0.114  13.54%
MLMG::mgVcycle_bottom                            82    0.07911    0.07911    0.07911   9.40%
MLMG::actualBottomSolve()                        82    0.07908    0.07908    0.07908   9.39%
MLCGSolver::bicgstab                             82    0.07833    0.07833    0.07833   9.30%
Amr::initialInit()                                1    0.06154    0.06154    0.06154   7.31%
MLPoisson::Fsmooth()                           3280    0.06017    0.06017    0.06017   7.15%
Amr::FinalizeInit()                               1    0.05246    0.05246    0.05246   6.23%
Castro::post_init()                               1    0.05175    0.05175    0.05175   6.15%
Gravity::multilevel_solve_for_new_phi()           1    0.04937    0.04937    0.04937   5.86%
Gravity::actual_multilevel_solve()                1    0.04935    0.04935    0.04935   5.86%
FillPatchIterator::Initialize                    41    0.04926    0.04926    0.04926   5.85%
Amr::writePlotFile()                              2    0.04913    0.04913    0.04913   5.83%
FillPatchSingleLevel                             41    0.04347    0.04347    0.04347   5.16%
MLCellLinOp::apply()                           1142    0.04008    0.04008    0.04008   4.76%
StateDataPhysBCFunct::()                         41    0.03923    0.03923    0.03923   4.66%
MLMG::mgVcycle_down::0                           82    0.03834    0.03834    0.03834   4.55%
Castro::clean_state()                            62    0.03727    0.03727    0.03727   4.43%
MLMG::mgVcycle_up::0                             82    0.03287    0.03287    0.03287   3.90%
FabArray::setVal()                             1144    0.02598    0.02598    0.02598   3.08%
StateData::FillBoundary(geom)                   328    0.02464    0.02464    0.02464   2.93%
Castro::initialize_do_advance()                  10     0.0242     0.0242     0.0242   2.87%
MLCellLinOp::correctionResidual()               492    0.02263    0.02263    0.02263   2.69%
MultiFab::Dot()                                1114      0.022      0.022      0.022   2.61%
MLMG:computeResOfCorrection()                   410    0.01958    0.01958    0.01958   2.33%
Castro::computeTemp()                            63    0.01953    0.01953    0.01953   2.32%
Gravity::get_new_grav_vector()                   11    0.01938    0.01938    0.01938   2.30%
FabArray::FillBoundary()                       4023    0.01932    0.01932    0.01932   2.29%
MLMG::mgVcycle_down::1                           82    0.01881    0.01881    0.01881   2.23%
FillBoundary_nowait()                          4023    0.01849    0.01849    0.01849   2.20%
Castro::expand_state()                           10    0.01833    0.01833    0.01833   2.18%
MLMG::mgVcycle_down::2                           82    0.01818    0.01818    0.01818   2.16%
MLPoisson::define()                              11    0.01757    0.01757    0.01757   2.09%
MLMG::mgVcycle_down::3                           82     0.0174     0.0174     0.0174   2.07%
FabArray::ParallelCopy()                        861    0.01737    0.01737    0.01737   2.06%
FabArray::ParallelCopy_nowait()                 861    0.01707    0.01707    0.01707   2.03%
MLMG::mgVcycle_down::4                           82    0.01664    0.01664    0.01664   1.98%
Castro::construct_old_gravity()                  10    0.01617    0.01617    0.01617   1.92%
Gravity::get_old_grav_vector()                   10    0.01616    0.01616    0.01616   1.92%
MLMG::mgVcycle_up::2                             82    0.01413    0.01413    0.01413   1.68%
MLMG::mgVcycle_up::1                             82    0.01391    0.01391    0.01391   1.65%
MLMG::addInterpCorrection()                     410    0.01372    0.01372    0.01372   1.63%
Castro::initialize_advance()                     10    0.01367    0.01367    0.01367   1.62%
MLMG::mgVcycle_up::3                             82    0.01342    0.01342    0.01342   1.59%
amrex::average_down                             410    0.01326    0.01326    0.01326   1.57%
MLMG::mgVcycle_up::4                             82    0.01324    0.01324    0.01324   1.57%
MLCGSolver::ParallelAllReduce                  1514    0.01321    0.01321    0.01321   1.57%
CGSolver::sxay()                               1586    0.01298    0.01298    0.01298   1.54%
MultiFab::LinComb()                            1586    0.01254    0.01254    0.01254   1.49%
MLCellLinOp::defineAuxData()                     11    0.01201    0.01201    0.01201   1.43%
Gravity::fill_multipole_BCs()                    11    0.01179    0.01179    0.01179   1.40%
Castro::enforce_min_density()                    62    0.01161    0.01161    0.01161   1.38%
Castro::do_new_sources()                         10    0.01128    0.01128    0.01128   1.34%
MLPoisson::Fapply()                            1142    0.01076    0.01076    0.01076   1.28%
Amr::InitializeInit()                             1   0.009081   0.009081   0.009081   1.08%
Amr::defBaseLevel()                               1   0.009075   0.009075   0.009075   1.08%
MLCellLinOp::solutionResidual()                  93    0.00783    0.00783    0.00783   0.93%
Castro::do_old_sources()                         10    0.00781    0.00781    0.00781   0.93%
Castro::post_timestep()                          10   0.007185   0.007185   0.007185   0.85%
MLMG::computeResidual()                          82   0.006772   0.006772   0.006772   0.80%
MultiFab::Xpay()                                585   0.006171   0.006171   0.006171   0.73%
MLMG::prepareForSolve()                          11   0.005972   0.005972   0.005972   0.71%
FabArray::setDomainBndry()                       41   0.005543   0.005543   0.005543   0.66%
Castro::reset_internal_energy()                  63   0.005508   0.005508   0.005508   0.65%
Castro::estTimeStep()                            21   0.005435   0.005435   0.005435   0.65%
MLCellLinOp::defineBC()                          11   0.005275   0.005275   0.005275   0.63%
BndryData::define()                              11   0.005098   0.005098   0.005098   0.61%
Castro::enforce_speed_limit()                    62    0.00407    0.00407    0.00407   0.48%
Castro::initData()                                1   0.003397   0.003397   0.003397   0.40%
Castro::normalize_species()                      62   0.002681   0.002681   0.002681   0.32%
MLMG::getGradSolution()                          11   0.002663   0.002663   0.002663   0.32%
MLCellLinOp::compGrad()                          11   0.002657   0.002657   0.002657   0.32%
Castro::construct_new_source()                   50    0.00264    0.00264    0.00264   0.31%
Castro::construct_new_gravity_source()           10   0.002557   0.002557   0.002557   0.30%
Castro::computeNewDt()                            9   0.002367   0.002367   0.002367   0.28%
FabArray::setVal(val, thecmd, scomp, ncomp)     462    0.00234    0.00234    0.00234   0.28%
Castro::construct_old_source()                   50   0.002079   0.002079   0.002079   0.25%
Castro::construct_old_gravity_source()           10   0.002061   0.002061   0.002061   0.24%
MLMG::ResNormInf()                               93   0.001901   0.001901   0.001901   0.23%
Castro::apply_source_to_state()                  20   0.001828   0.001828   0.001828   0.22%
MultiFab::Saxpy()                                20   0.001816   0.001816   0.001816   0.22%
MLCellLinOp::setLevelBC()                        11   0.001546   0.001546   0.001546   0.18%
FabArrayBase::getCPC()                         1323    0.00153    0.00153    0.00153   0.18%
MLPoisson::prepareForSolve()                     11   0.001384   0.001384   0.001384   0.16%
MLCellLinOp::prepareForSolve()                   11   0.001375   0.001375   0.001375   0.16%
FabArray::mult()                                 43   0.001329   0.001329   0.001329   0.16%
Gravity::swapTimeLevels()                        10   0.001146   0.001146   0.001146   0.14%
MLMG::computeMLResidual()                        11   0.001098   0.001098   0.001098   0.13%
Gravity::update_max_rhs()                        11  0.0009697  0.0009697  0.0009697   0.12%
FabArrayBase::getFB()                          4023  0.0007619  0.0007619  0.0007619   0.09%
FabArrayBase::CPC::define()                     454  0.0007019  0.0007019  0.0007019   0.08%
Castro::computeInitialDt()                        2  0.0006653  0.0006653  0.0006653   0.08%
MultiFab::Copy()                                 11  0.0005203  0.0005203  0.0005203   0.06%
Castro::post_regrid()                             1  0.0004772  0.0004772  0.0004772   0.06%
Amr::InitAmr()                                    1    0.00047    0.00047    0.00047   0.06%
MultiFab::min()                                  10  0.0003199  0.0003199  0.0003199   0.04%
MLLinOp::define()                                11  0.0002571  0.0002571  0.0002571   0.03%
MLMG::MLResNormInf()                             11  0.0002549  0.0002549  0.0002549   0.03%
MultiFab::max()                                  11  0.0002524  0.0002524  0.0002524   0.03%
MLLinOp::defineGrids()                           11  0.0002294  0.0002294  0.0002294   0.03%
MLMG::MLRhsNormInf()                             11  0.0002011  0.0002011  0.0002011   0.02%
FabArrayBase::FB::FB()                           56  9.138e-05  9.138e-05  9.138e-05   0.01%
AmrLevel::AmrLevel(dm)                            1  6.235e-05  6.235e-05  6.235e-05   0.01%
Castro::finalize_advance()                       10  6.131e-05  6.131e-05  6.131e-05   0.01%
MLLinOp::makeAgglomeratedDMap                    11  5.657e-05  5.657e-05  5.657e-05   0.01%
StateData::define()                               4  5.291e-05  5.291e-05  5.291e-05   0.01%
makeSFC                                          55   4.03e-05   4.03e-05   4.03e-05   0.00%
Castro::finalize_do_advance()                    10  3.698e-05  3.698e-05  3.698e-05   0.00%
Castro::swap_state_time_levels()                 10   3.57e-05   3.57e-05   3.57e-05   0.00%
Castro::enforce_consistent_e()                    1  3.564e-05  3.564e-05  3.564e-05   0.00%
Amr::writeSmallPlotFile()                         1  2.476e-05  2.476e-05  2.476e-05   0.00%
DistributionMapping::Distribute()                56  1.503e-05  1.503e-05  1.503e-05   0.00%
Amr::initSubcycle()                               1  9.363e-06  9.363e-06  9.363e-06   0.00%
MLMG::buildFineMask()                            11  6.066e-06  6.066e-06  6.066e-06   0.00%
Castro::create_source_corrector()                10   5.92e-06   5.92e-06   5.92e-06   0.00%
AmrLevel::checkPointPost()                        3  4.773e-06  4.773e-06  4.773e-06   0.00%
DistributionMapping::SFCProcessorMapDoIt()        1  4.751e-06  4.751e-06  4.751e-06   0.00%
Gravity::set_mass_offset()                       11  4.244e-06  4.244e-06  4.244e-06   0.00%
Castro::retry_advance_ctu()                      10  3.793e-06  3.793e-06  3.793e-06   0.00%
Castro::FluxRegFineAdd()                         10  3.499e-06  3.499e-06  3.499e-06   0.00%
MLLinOp::makeSubCommunicator()                   11  2.032e-06  2.032e-06  2.032e-06   0.00%
AmrLevel::checkPointPre()                         3  1.873e-06  1.873e-06  1.873e-06   0.00%
--------------------------------------------------------------------------------------------

Unused ParmParse Variables:
  [TOP]::amr.ref_ratio(nvals = 4)  :: [2, 2, 2, 2]
  [TOP]::amr.regrid_int(nvals = 4)  :: [2, 2, 2, 2]

Total GPU global memory (MB): 12066
Free  GPU global memory (MB): 10619
[The         Arena] space allocated (MB): 9049
[The         Arena] space used      (MB): 0
[The  Pinned Arena] space allocated (MB): 8
[The  Pinned Arena] space used      (MB): 0
AMReX (22.03-21-g336ca7002b45) finalized
Initializing CUDA...
CUDA initialized with 1 GPU
AMReX (22.03-21-g336ca7002b45) initialized

Starting run at 08:25:48 UTC on 2022-03-22.
Successfully read inputs file ... 

Castro git describe: 22.02-13-ga12efaab3
AMReX git describe: 22.03-21-g336ca7002
Microphysics git describe: 22.03-8-g75721f58

reading extern runtime parameters ...
3 Species: 
C12  O16  Mg24  
restarting calculation from file: dustcollapse-restart_chk00005
Successfully read inputs file ... 
read CPU time: 0.497504011
Restart time = 0.054595874 seconds.
[Level 0 step 6] ADVANCE with dt = 5.79654185e-05
[Level 0 step 6] Advanced 262144 cells

[STEP 6] Coarse TimeStep time: 0.111683906
[STEP 6] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05

[Level 0 step 7] ADVANCE with dt = 6.086368943e-05
[Level 0 step 7] Advanced 262144 cells

[STEP 7] Coarse TimeStep time: 0.048285463
[STEP 7] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05

[Level 0 step 8] ADVANCE with dt = 6.39068739e-05
[Level 0 step 8] Advanced 262144 cells

[STEP 8] Coarse TimeStep time: 0.050010495
[STEP 8] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05

[Level 0 step 9] ADVANCE with dt = 6.71022176e-05
[Level 0 step 9] Advanced 262144 cells

[STEP 9] Coarse TimeStep time: 0.058742854
[STEP 9] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05

[Level 0 step 10] ADVANCE with dt = 7.045732848e-05
[Level 0 step 10] Advanced 262144 cells

[STEP 10] Coarse TimeStep time: 0.065844044
[STEP 10] FAB kilobyte spread across MPI nodes: [319638 ... 319638]

STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05

PLOTFILE: file = dustcollapse-restart_plt00010
Write plotfile time = 0.024552345  seconds


Ending run at 08:25:49 UTC on 2022-03-22.
Run time = 0.414654913
Run time without initialization = 0.35947088

  Average number of zones advanced per microsecond: 3.646
  Average number of zones advanced per microsecond per rank: 3.646

CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9489481728


TinyProfiler total time across processes [min...avg...max]: 0.4147 ... 0.4147 ... 0.4147

--------------------------------------------------------------------------------------------
Name                                         NCalls  Excl. Min  Excl. Avg  Excl. Max   Max %
--------------------------------------------------------------------------------------------
Castro::construct_ctu_hydro_source()              5     0.1276     0.1276     0.1276  30.78%
VisMF::Read()                                     3    0.04272    0.04272    0.04272  10.30%
MLCellLinOp::applyBC()                         1946    0.03811    0.03811    0.03811   9.19%
MLPoisson::Fsmooth()                           1440    0.02585    0.02585    0.02585   6.23%
VisMF::Write(FabArray)                            1    0.02329    0.02329    0.02329   5.62%
FabArray::setVal()                              537    0.01576    0.01576    0.01576   3.80%
StateData::FillBoundary(geom)                   160    0.01147    0.01147    0.01147   2.77%
MLCGSolver::bicgstab                             36   0.009416   0.009416   0.009416   2.27%
MultiFab::Dot()                                 484   0.009338   0.009338   0.009338   2.25%
StateDataPhysBCFunct::()                         20   0.007885   0.007885   0.007885   1.90%
FillBoundary_nowait()                          1766   0.007166   0.007166   0.007166   1.73%
FabArray::ParallelCopy_nowait()                 380   0.007165   0.007165   0.007165   1.73%
Castro::computeTemp()                            30   0.007034   0.007034   0.007034   1.70%
Gravity::fill_multipole_BCs()                     6   0.006292   0.006292   0.006292   1.52%
MLCellLinOp::defineAuxData()                      6   0.005526   0.005526   0.005526   1.33%
MultiFab::LinComb()                             690   0.005346   0.005346   0.005346   1.29%
Castro::enforce_min_density()                    30   0.005085   0.005085   0.005085   1.23%
MLPoisson::Fapply()                             500   0.004685   0.004685   0.004685   1.13%
Castro::expand_state()                            5   0.004171   0.004171   0.004171   1.01%
FabArray::setDomainBndry()                       20   0.003783   0.003783   0.003783   0.91%
MLMG::addInterpCorrection()                     180   0.003045   0.003045   0.003045   0.73%
Amr::restart()                                    1   0.002983   0.002983   0.002983   0.72%
amrex::average_down                             180   0.002851   0.002851   0.002851   0.69%
MultiFab::Xpay()                                258   0.002687   0.002687   0.002687   0.65%
Castro::do_advance_ctu()                          5   0.002655   0.002655   0.002655   0.64%
Castro::reset_internal_energy()                  30   0.002407   0.002407   0.002407   0.58%
Castro::estTimeStep()                            10   0.002335   0.002335   0.002335   0.56%
BndryData::define()                               6   0.002042   0.002042   0.002042   0.49%
Gravity::get_new_grav_vector()                    5   0.001915   0.001915   0.001915   0.46%
MLCellLinOp::compGrad()                           6   0.001465   0.001465   0.001465   0.35%
Amr::writePlotFile()                              1   0.001358   0.001358   0.001358   0.33%
Castro::normalize_species()                      30   0.001305   0.001305   0.001305   0.31%
Castro::construct_new_gravity_source()            5   0.001296   0.001296   0.001296   0.31%
FabArray::setVal(val, thecmd, scomp, ncomp)     252   0.001237   0.001237   0.001237   0.30%
Castro::construct_old_gravity_source()            5    0.00112    0.00112    0.00112   0.27%
Gravity::get_old_grav_vector()                    5  0.0009215  0.0009215  0.0009215   0.22%
MultiFab::Saxpy()                                10  0.0009209  0.0009209  0.0009209   0.22%
MLMG::ResNormInf()                               42  0.0008405  0.0008405  0.0008405   0.20%
MLCellLinOp::setLevelBC()                         6  0.0008324  0.0008324  0.0008324   0.20%
MLMG::oneIter()                                  36  0.0007435  0.0007435  0.0007435   0.18%
Gravity::actual_solve_with_mlmg()                 6  0.0007302  0.0007302  0.0007302   0.18%
MLCellLinOp::prepareForSolve()                    6  0.0007027  0.0007027  0.0007027   0.17%
Castro::initialize_advance()                      5   0.000684   0.000684   0.000684   0.16%
Castro::subcycle_advance_ctu()                    5   0.000683   0.000683   0.000683   0.16%
FabArray::mult()                                 22  0.0006617  0.0006617  0.0006617   0.16%
Gravity::update_max_rhs()                         6   0.000514   0.000514   0.000514   0.12%
MLCellLinOp::smooth()                           720  0.0005096  0.0005096  0.0005096   0.12%
MLMG::prepareForSolve()                           6  0.0004381  0.0004381  0.0004381   0.11%
FabArrayBase::CPC::define()                     244  0.0004374  0.0004374  0.0004374   0.11%
FabArrayBase::getCPC()                          632   0.000427   0.000427   0.000427   0.10%
Castro::enforce_speed_limit()                    30   0.000417   0.000417   0.000417   0.10%
FabArray::FillBoundary()                       1766  0.0004056  0.0004056  0.0004056   0.10%
Amr::InitAmr()                                    1  0.0003963  0.0003963  0.0003963   0.10%
MultiFab::Copy()                                  6  0.0003707  0.0003707  0.0003707   0.09%
FabArrayBase::getFB()                          1766  0.0002993  0.0002993  0.0002993   0.07%
main()                                            1  0.0002962  0.0002962  0.0002962   0.07%
Gravity::solve_for_phi()                          5  0.0002265  0.0002265  0.0002265   0.05%
MLCellLinOp::apply()                            500   0.000219   0.000219   0.000219   0.05%
Gravity::actual_multilevel_solve()                1  0.0002178  0.0002178  0.0002178   0.05%
Castro::construct_new_gravity()                   5  0.0002005  0.0002005  0.0002005   0.05%
CGSolver::sxay()                                690  0.0001896  0.0001896  0.0001896   0.05%
Castro::construct_new_source()                   25  0.0001674  0.0001674  0.0001674   0.04%
MultiFab::min()                                   5  0.0001615  0.0001615  0.0001615   0.04%
MLMG::mgVcycle()                                 36   0.000158   0.000158   0.000158   0.04%
MLCGSolver::ParallelAllReduce                   659  0.0001416  0.0001416  0.0001416   0.03%
FabArray::ParallelCopy()                        380  0.0001336  0.0001336  0.0001336   0.03%
MultiFab::max()                                   6  0.0001315  0.0001315  0.0001315   0.03%
FillPatchIterator::Initialize                    20  0.0001245  0.0001245  0.0001245   0.03%
MLCellLinOp::defineBC()                           6   0.000109   0.000109   0.000109   0.03%
MLMG::MLRhsNormInf()                              6  0.0001067  0.0001067  0.0001067   0.03%
Amr::coarseTimeStep()                             5  0.0001017  0.0001017  0.0001017   0.02%
MLLinOp::defineGrids()                            6   9.28e-05   9.28e-05   9.28e-05   0.02%
MLCellLinOp::correctionResidual()               216  9.036e-05  9.036e-05  9.036e-05   0.02%
Amr::timeStep()                                   5  8.932e-05  8.932e-05  8.932e-05   0.02%
AmrLevel::restart()                               1  8.782e-05  8.782e-05  8.782e-05   0.02%
StateData::restartDoit()                          4   7.28e-05   7.28e-05   7.28e-05   0.02%
MLMG:computeResOfCorrection()                   180  6.832e-05  6.832e-05  6.832e-05   0.02%
FabArrayBase::FB::FB()                           26   6.07e-05   6.07e-05   6.07e-05   0.01%
Castro::construct_old_source()                   25  5.477e-05  5.477e-05  5.477e-05   0.01%
Castro::advance()                                 5  5.273e-05  5.273e-05  5.273e-05   0.01%
MLMG::actualBottomSolve()                        36  4.844e-05  4.844e-05  4.844e-05   0.01%
Castro::create_source_corrector()                 5  4.414e-05  4.414e-05  4.414e-05   0.01%
Castro::initialize_do_advance()                   5  4.378e-05  4.378e-05  4.378e-05   0.01%
Castro::construct_old_gravity()                   5  4.171e-05  4.171e-05  4.171e-05   0.01%
Castro::post_timestep()                           5  3.986e-05  3.986e-05  3.986e-05   0.01%
MLMG::mgVcycle_down::0                           36  3.793e-05  3.793e-05  3.793e-05   0.01%
Castro::do_new_sources()                          5  3.788e-05  3.788e-05  3.788e-05   0.01%
MLMG::mgVcycle_down::1                           36   3.71e-05   3.71e-05   3.71e-05   0.01%
MLMG::solve()                                     6  3.488e-05  3.488e-05  3.488e-05   0.01%
MLMG::mgVcycle_down::2                           36  3.427e-05  3.427e-05  3.427e-05   0.01%
Castro::clean_state()                            30  3.368e-05  3.368e-05  3.368e-05   0.01%
MLMG::mgVcycle_down::4                           36  3.264e-05  3.264e-05  3.264e-05   0.01%
MLMG::mgVcycle_down::3                           36  3.226e-05  3.226e-05  3.226e-05   0.01%
Castro::finalize_advance()                        5  2.664e-05  2.664e-05  2.664e-05   0.01%
MLMG::mgVcycle_up::4                             36  2.661e-05  2.661e-05  2.661e-05   0.01%
MLLinOp::define()                                 6  2.633e-05  2.633e-05  2.633e-05   0.01%
Amr::writeSmallPlotFile()                         1  2.573e-05  2.573e-05  2.573e-05   0.01%
Castro::swap_state_time_levels()                  5  2.551e-05  2.551e-05  2.551e-05   0.01%
MLMG::mgVcycle_up::0                             36  2.405e-05  2.405e-05  2.405e-05   0.01%
Castro::post_restart()                            1  2.343e-05  2.343e-05  2.343e-05   0.01%
MLCellLinOp::solutionResidual()                  42  2.214e-05  2.214e-05  2.214e-05   0.01%
MLMG::mgVcycle_up::3                             36  2.195e-05  2.195e-05  2.195e-05   0.01%
MLMG::mgVcycle_up::2                             36  2.153e-05  2.153e-05  2.153e-05   0.01%
MLMG::mgVcycle_up::1                             36  2.053e-05  2.053e-05  2.053e-05   0.00%
Castro::finalize_do_advance()                     5  2.035e-05  2.035e-05  2.035e-05   0.00%
makeSFC                                          30    1.8e-05    1.8e-05    1.8e-05   0.00%
Gravity::multilevel_solve_for_new_phi()           1  1.619e-05  1.619e-05  1.619e-05   0.00%
MLMG::mgVcycle_bottom                            36  1.596e-05  1.596e-05  1.596e-05   0.00%
MLMG::computeResidual()                          36  1.525e-05  1.525e-05  1.525e-05   0.00%
Gravity::solve_phi_with_mlmg()                    6  1.507e-05  1.507e-05  1.507e-05   0.00%
MLPoisson::define()                               6  1.431e-05  1.431e-05  1.431e-05   0.00%
FillPatchSingleLevel                             20  1.429e-05  1.429e-05  1.429e-05   0.00%
DistributionMapping::Distribute()                31   1.06e-05   1.06e-05   1.06e-05   0.00%
Amr::initSubcycle()                               1  9.283e-06  9.283e-06  9.283e-06   0.00%
MLLinOp::makeAgglomeratedDMap                     6  8.363e-06  8.363e-06  8.363e-06   0.00%
Castro::do_old_sources()                          5  7.766e-06  7.766e-06  7.766e-06   0.00%
Castro::apply_source_to_state()                  10  5.681e-06  5.681e-06  5.681e-06   0.00%
MLPoisson::prepareForSolve()                      6  4.656e-06  4.656e-06  4.656e-06   0.00%
Gravity::swapTimeLevels()                         5  4.312e-06  4.312e-06  4.312e-06   0.00%
Castro::retry_advance_ctu()                       5   3.56e-06   3.56e-06   3.56e-06   0.00%
Castro::computeNewDt()                            5  3.432e-06  3.432e-06  3.432e-06   0.00%
MLMG::computeMLResidual()                         6  3.181e-06  3.181e-06  3.181e-06   0.00%
MLMG::buildFineMask()                             6  3.151e-06  3.151e-06  3.151e-06   0.00%
MLMG::getGradSolution()                           6  3.021e-06  3.021e-06  3.021e-06   0.00%
DistributionMapping::SFCProcessorMapDoIt()        1  2.891e-06  2.891e-06  2.891e-06   0.00%
MLMG::MLResNormInf()                              6  2.551e-06  2.551e-06  2.551e-06   0.00%
Gravity::set_mass_offset()                        6  2.443e-06  2.443e-06  2.443e-06   0.00%
Castro::FluxRegFineAdd()                          5  1.462e-06  1.462e-06  1.462e-06   0.00%
Amr::init()                                       1   1.13e-06   1.13e-06   1.13e-06   0.00%
MLLinOp::makeSubCommunicator()                    6  1.064e-06  1.064e-06  1.064e-06   0.00%
AmrLevel::AmrLevel()                              1   8.97e-07   8.97e-07   8.97e-07   0.00%
--------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------
Name                                         NCalls  Incl. Min  Incl. Avg  Incl. Max   Max %
--------------------------------------------------------------------------------------------
main()                                            1     0.4147     0.4147     0.4147 100.00%
Amr::coarseTimeStep()                             5     0.3347     0.3347     0.3347  80.70%
Amr::timeStep()                                   5     0.3334     0.3334     0.3334  80.39%
Castro::advance()                                 5     0.3306     0.3306     0.3306  79.73%
Castro::subcycle_advance_ctu()                    5     0.3188     0.3188     0.3188  76.87%
Castro::do_advance_ctu()                          5     0.3181     0.3181     0.3181  76.71%
Castro::construct_new_gravity()                   5     0.1477     0.1477     0.1477  35.63%
Gravity::solve_phi_with_mlmg()                    6     0.1436     0.1436     0.1436  34.63%
Gravity::solve_for_phi()                          5     0.1377     0.1377     0.1377  33.20%
Gravity::actual_solve_with_mlmg()                 6     0.1372     0.1372     0.1372  33.08%
Castro::construct_ctu_hydro_source()              5     0.1276     0.1276     0.1276  30.78%
MLMG::solve()                                     6     0.1243     0.1243     0.1243  29.98%
MLMG::oneIter()                                  36     0.1163     0.1163     0.1163  28.05%
MLMG::mgVcycle()                                 36     0.1156     0.1156     0.1156  27.87%
MLCellLinOp::smooth()                           720    0.06035    0.06035    0.06035  14.55%
Amr::init()                                       1    0.05463    0.05463    0.05463  13.17%
Amr::restart()                                    1    0.05463    0.05463    0.05463  13.17%
MLCellLinOp::applyBC()                         1946    0.04604    0.04604    0.04604  11.10%
AmrLevel::restart()                               1    0.04363    0.04363    0.04363  10.52%
StateData::restartDoit()                          4    0.04353    0.04353    0.04353  10.50%
VisMF::Read()                                     3    0.04272    0.04272    0.04272  10.30%
MLMG::mgVcycle_bottom                            36    0.03346    0.03346    0.03346   8.07%
MLMG::actualBottomSolve()                        36    0.03344    0.03344    0.03344   8.06%
MLCGSolver::bicgstab                             36    0.03312    0.03312    0.03312   7.99%
MLPoisson::Fsmooth()                           1440    0.02585    0.02585    0.02585   6.23%
FillPatchIterator::Initialize                    20    0.02537    0.02537    0.02537   6.12%
Amr::writePlotFile()                              1    0.02465    0.02465    0.02465   5.94%
VisMF::Write(FabArray)                            1    0.02329    0.02329    0.02329   5.62%
FillPatchSingleLevel                             20    0.02146    0.02146    0.02146   5.17%
StateDataPhysBCFunct::()                         20    0.01936    0.01936    0.01936   4.67%
MLCellLinOp::apply()                            500    0.01665    0.01665    0.01665   4.02%
Castro::clean_state()                            30    0.01628    0.01628    0.01628   3.93%
MLMG::mgVcycle_down::0                           36    0.01604    0.01604    0.01604   3.87%
FabArray::setVal()                              537    0.01576    0.01576    0.01576   3.80%
Castro::initialize_do_advance()                   5    0.01463    0.01463    0.01463   3.53%
MLMG::mgVcycle_up::0                             36     0.0137     0.0137     0.0137   3.30%
Castro::expand_state()                            5    0.01184    0.01184    0.01184   2.85%
Castro::initialize_advance()                      5    0.01179    0.01179    0.01179   2.84%
StateData::FillBoundary(geom)                   160    0.01147    0.01147    0.01147   2.77%
Gravity::get_new_grav_vector()                    5   0.009881   0.009881   0.009881   2.38%
MLPoisson::define()                               6   0.009518   0.009518   0.009518   2.30%
Castro::computeTemp()                            30   0.009441   0.009441   0.009441   2.28%
MLCellLinOp::correctionResidual()               216   0.009405   0.009405   0.009405   2.27%
MultiFab::Dot()                                 484   0.009338   0.009338   0.009338   2.25%
Castro::construct_old_gravity()                   5   0.008314   0.008314   0.008314   2.00%
Gravity::get_old_grav_vector()                    5   0.008272   0.008272   0.008272   1.99%
MLMG:computeResOfCorrection()                   180   0.008155   0.008155   0.008155   1.97%
FabArray::FillBoundary()                       1766   0.007931   0.007931   0.007931   1.91%
MLMG::mgVcycle_down::1                           36   0.007845   0.007845   0.007845   1.89%
FabArray::ParallelCopy()                        380   0.007728   0.007728   0.007728   1.86%
FabArray::ParallelCopy_nowait()                 380   0.007594   0.007594   0.007594   1.83%
MLMG::mgVcycle_down::2                           36   0.007547   0.007547   0.007547   1.82%
FillBoundary_nowait()                          1766   0.007526   0.007526   0.007526   1.81%
MLMG::mgVcycle_down::3                           36   0.007197   0.007197   0.007197   1.74%
MLMG::mgVcycle_down::4                           36   0.006888   0.006888   0.006888   1.66%
Castro::post_restart()                            1    0.00676    0.00676    0.00676   1.63%
MLCellLinOp::defineAuxData()                      6   0.006476   0.006476   0.006476   1.56%
Gravity::multilevel_solve_for_new_phi()           1   0.006384   0.006384   0.006384   1.54%
Gravity::actual_multilevel_solve()                1   0.006368   0.006368   0.006368   1.54%
Gravity::fill_multipole_BCs()                     6   0.006292   0.006292   0.006292   1.52%
MLMG::mgVcycle_up::2                             36   0.005862   0.005862   0.005862   1.41%
MLMG::addInterpCorrection()                     180   0.005858   0.005858   0.005858   1.41%
MLMG::mgVcycle_up::1                             36   0.005795   0.005795   0.005795   1.40%
amrex::average_down                             180   0.005676   0.005676   0.005676   1.37%
Castro::do_new_sources()                          5   0.005669   0.005669   0.005669   1.37%
MLCGSolver::ParallelAllReduce                   659   0.005641   0.005641   0.005641   1.36%
MLMG::mgVcycle_up::3                             36   0.005575   0.005575   0.005575   1.34%
CGSolver::sxay()                                690   0.005536   0.005536   0.005536   1.34%
MLMG::mgVcycle_up::4                             36   0.005507   0.005507   0.005507   1.33%
MultiFab::LinComb()                             690   0.005346   0.005346   0.005346   1.29%
Castro::enforce_min_density()                    30   0.005085   0.005085   0.005085   1.23%
MLPoisson::Fapply()                             500   0.004685   0.004685   0.004685   1.13%
Castro::do_old_sources()                          5   0.003843   0.003843   0.003843   0.93%
FabArray::setDomainBndry()                       20   0.003783   0.003783   0.003783   0.91%
MLMG::prepareForSolve()                           6   0.003572   0.003572   0.003572   0.86%
MLCellLinOp::solutionResidual()                  42   0.003435   0.003435   0.003435   0.83%
MLCellLinOp::defineBC()                           6   0.002872   0.002872   0.002872   0.69%
MLMG::computeResidual()                          36   0.002852   0.002852   0.002852   0.69%
BndryData::define()                               6   0.002763   0.002763   0.002763   0.67%
MultiFab::Xpay()                                258   0.002687   0.002687   0.002687   0.65%
Castro::post_timestep()                           5   0.002624   0.002624   0.002624   0.63%
Castro::reset_internal_energy()                  30   0.002407   0.002407   0.002407   0.58%
Castro::estTimeStep()                            10   0.002335   0.002335   0.002335   0.56%
MLMG::getGradSolution()                           6    0.00176    0.00176    0.00176   0.42%
MLCellLinOp::compGrad()                           6   0.001757   0.001757   0.001757   0.42%
Castro::construct_new_source()                   25   0.001463   0.001463   0.001463   0.35%
Castro::normalize_species()                      30   0.001305   0.001305   0.001305   0.31%
Castro::construct_new_gravity_source()            5   0.001296   0.001296   0.001296   0.31%
FabArray::setVal(val, thecmd, scomp, ncomp)     252   0.001237   0.001237   0.001237   0.30%
Castro::computeNewDt()                            5     0.0012     0.0012     0.0012   0.29%
Castro::construct_old_source()                   25   0.001175   0.001175   0.001175   0.28%
Castro::construct_old_gravity_source()            5    0.00112    0.00112    0.00112   0.27%
Castro::apply_source_to_state()                  10  0.0009265  0.0009265  0.0009265   0.22%
MultiFab::Saxpy()                                10  0.0009209  0.0009209  0.0009209   0.22%
Gravity::swapTimeLevels()                         5  0.0008788  0.0008788  0.0008788   0.21%
FabArrayBase::getCPC()                          632  0.0008643  0.0008643  0.0008643   0.21%
MLMG::ResNormInf()                               42  0.0008405  0.0008405  0.0008405   0.20%
MLCellLinOp::setLevelBC()                         6  0.0008324  0.0008324  0.0008324   0.20%
Gravity::update_max_rhs()                         6  0.0007277  0.0007277  0.0007277   0.18%
MLPoisson::prepareForSolve()                      6  0.0007074  0.0007074  0.0007074   0.17%
MLCellLinOp::prepareForSolve()                    6  0.0007027  0.0007027  0.0007027   0.17%
FabArray::mult()                                 22  0.0006617  0.0006617  0.0006617   0.16%
MLMG::computeMLResidual()                         6  0.0006015  0.0006015  0.0006015   0.15%
FabArrayBase::CPC::define()                     244  0.0004374  0.0004374  0.0004374   0.11%
Castro::enforce_speed_limit()                    30   0.000417   0.000417   0.000417   0.10%
Amr::InitAmr()                                    1  0.0004056  0.0004056  0.0004056   0.10%
MultiFab::Copy()                                  6  0.0003707  0.0003707  0.0003707   0.09%
FabArrayBase::getFB()                          1766    0.00036    0.00036    0.00036   0.09%
MultiFab::min()                                   5  0.0001615  0.0001615  0.0001615   0.04%
MLLinOp::define()                                 6  0.0001557  0.0001557  0.0001557   0.04%
MLMG::MLResNormInf()                              6  0.0001354  0.0001354  0.0001354   0.03%
MultiFab::max()                                   6  0.0001315  0.0001315  0.0001315   0.03%
MLLinOp::defineGrids()                            6  0.0001294  0.0001294  0.0001294   0.03%
MLMG::MLRhsNormInf()                              6  0.0001067  0.0001067  0.0001067   0.03%
FabArrayBase::FB::FB()                           26   6.07e-05   6.07e-05   6.07e-05   0.01%
Castro::create_source_corrector()                 5  4.414e-05  4.414e-05  4.414e-05   0.01%
MLLinOp::makeAgglomeratedDMap                     6  3.553e-05  3.553e-05  3.553e-05   0.01%
Castro::finalize_advance()                        5   2.81e-05   2.81e-05   2.81e-05   0.01%
makeSFC                                          30  2.717e-05  2.717e-05  2.717e-05   0.01%
Amr::writeSmallPlotFile()                         1  2.573e-05  2.573e-05  2.573e-05   0.01%
Castro::swap_state_time_levels()                  5  2.551e-05  2.551e-05  2.551e-05   0.01%
Castro::finalize_do_advance()                     5  2.035e-05  2.035e-05  2.035e-05   0.00%
DistributionMapping::Distribute()                31   1.06e-05   1.06e-05   1.06e-05   0.00%
Amr::initSubcycle()                               1  9.283e-06  9.283e-06  9.283e-06   0.00%
DistributionMapping::SFCProcessorMapDoIt()        1  4.319e-06  4.319e-06  4.319e-06   0.00%
Castro::retry_advance_ctu()                       5   3.56e-06   3.56e-06   3.56e-06   0.00%
MLMG::buildFineMask()                             6  3.151e-06  3.151e-06  3.151e-06   0.00%
Gravity::set_mass_offset()                        6  2.443e-06  2.443e-06  2.443e-06   0.00%
Castro::FluxRegFineAdd()                          5  1.462e-06  1.462e-06  1.462e-06   0.00%
MLLinOp::makeSubCommunicator()                    6  1.064e-06  1.064e-06  1.064e-06   0.00%
AmrLevel::AmrLevel()                              1   8.97e-07   8.97e-07   8.97e-07   0.00%
--------------------------------------------------------------------------------------------

Unused ParmParse Variables:
  [TOP]::amr.ref_ratio(nvals = 4)  :: [2, 2, 2, 2]
  [TOP]::amr.regrid_int(nvals = 4)  :: [2, 2, 2, 2]

Total GPU global memory (MB): 12066
Free  GPU global memory (MB): 10619
[The         Arena] space allocated (MB): 9049
[The         Arena] space used      (MB): 0
[The  Pinned Arena] space allocated (MB): 8
[The  Pinned Arena] space used      (MB): 0
AMReX (22.03-21-g336ca7002b45) finalized