MPI initialized with 8 MPI processes MPI initialized with thread support level 3 AMReX (23.09-5-gb98bdae9fb67) initialized Starting run at 05:21:23 UTC on 2023-09-12. Successfully read inputs file ... Castro git describe: 23.09-4-ge16cd4ff2 AMReX git describe: 23.09-5-gb98bdae9f Microphysics git describe: 23.09-4-g2c408a9c reading extern runtime parameters ... 3 Species: C12 O16 Mg24 Successfully read inputs file ... INITIAL GRIDS Level 0 8 grids 262144 cells 100 % of domain smallest grid: 32 x 32 x 32 biggest grid: 32 x 32 x 32 PLOTFILE: file = dustcollapse-3d-monopole_plt00000 Write plotfile time = 0.014338571 seconds [Level 0 step 1] ADVANCE at time 0 with dt = 4.541742215e-05 [Level 0 step 1] Advanced 262144 cells [STEP 1] Coarse TimeStep time: 0.391732973 [STEP 1] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 1 TIME = 4.541742215e-05 DT = 4.541742215e-05 [Level 0 step 2] ADVANCE at time 4.541742215e-05 with dt = 4.768829326e-05 [Level 0 step 2] Advanced 262144 cells [STEP 2] Coarse TimeStep time: 0.350549624 [STEP 2] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 2 TIME = 9.31057154e-05 DT = 4.768829326e-05 [Level 0 step 3] ADVANCE at time 9.31057154e-05 with dt = 5.007270792e-05 [Level 0 step 3] Advanced 262144 cells [STEP 3] Coarse TimeStep time: 0.346645894 [STEP 3] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 3 TIME = 0.0001431784233 DT = 5.007270792e-05 [Level 0 step 4] ADVANCE at time 0.0001431784233 with dt = 5.257634331e-05 [Level 0 step 4] Advanced 262144 cells [STEP 4] Coarse TimeStep time: 0.341308128 [STEP 4] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 4 TIME = 0.0001957547666 DT = 5.257634331e-05 [Level 0 step 5] ADVANCE at time 0.0001957547666 with dt = 5.520516048e-05 [Level 0 step 5] Advanced 262144 cells [STEP 5] Coarse TimeStep time: 0.357268087 [STEP 5] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 5 TIME = 0.0002509599271 DT = 5.520516048e-05 [Level 0 step 6] ADVANCE at time 0.0002509599271 with dt = 5.79654185e-05 [Level 0 step 6] Advanced 262144 cells [STEP 6] Coarse TimeStep time: 0.332026879 [STEP 6] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 6 TIME = 0.0003089253456 DT = 5.79654185e-05 [Level 0 step 7] ADVANCE at time 0.0003089253456 with dt = 6.086368943e-05 [Level 0 step 7] Advanced 262144 cells [STEP 7] Coarse TimeStep time: 0.316045967 [STEP 7] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 7 TIME = 0.000369789035 DT = 6.086368943e-05 [Level 0 step 8] ADVANCE at time 0.000369789035 with dt = 6.39068739e-05 [Level 0 step 8] Advanced 262144 cells [STEP 8] Coarse TimeStep time: 0.313252806 [STEP 8] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 8 TIME = 0.000433695909 DT = 6.39068739e-05 [Level 0 step 9] ADVANCE at time 0.000433695909 with dt = 6.71022176e-05 [Level 0 step 9] Advanced 262144 cells [STEP 9] Coarse TimeStep time: 0.319722185 [STEP 9] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 9 TIME = 0.0005007981265 DT = 6.71022176e-05 [Level 0 step 10] ADVANCE at time 0.0005007981265 with dt = 7.045732848e-05 [Level 0 step 10] Advanced 262144 cells [STEP 10] Coarse TimeStep time: 0.314860345 [STEP 10] FAB kilobyte spread across MPI nodes: [57998 ... 57998] STEP = 10 TIME = 0.000571255455 DT = 7.045732848e-05 PLOTFILE: file = dustcollapse-3d-monopole_plt00010 Write plotfile time = 0.048377719 seconds Ending run at 05:21:27 UTC on 2023-09-12. Run time = 3.63601252 Run time without initialization = 3.437715209 Average number of zones advanced per microsecond: 0.763 Average number of zones advanced per microsecond per rank: 0.095 TinyProfiler total time across processes [min...avg...max]: 3.634 ... 3.635 ... 3.636 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 10 2.3019 2.3671 2.4681 67.88% Gravity::make_radial_gravity() 21 0.3249 0.334 0.3492 9.60% check_for_negative_density() 10 0.01988 0.1224 0.1881 5.17% Castro::initData() 1 0.03911 0.08714 0.1327 3.65% Castro::estTimeStep() 21 0.023 0.07528 0.1303 3.58% FabArray::ParallelCopy_finish() 41 0.03253 0.04897 0.08793 2.42% amrex::Copy() 193 0.06281 0.07559 0.08438 2.32% FabArray::setVal() 143 0.05938 0.06828 0.07807 2.15% Castro::reset_internal_energy(Fab) 1008 0.04804 0.04921 0.05009 1.38% Amr::writePlotFile() 2 0.04432 0.04734 0.04944 1.36% StateData::FillBoundary(geom) 41 0.03422 0.03804 0.04359 1.20% Castro::normalize_species() 62 0.04144 0.04196 0.04248 1.17% FabArray::ParallelCopy_nowait() 41 0.03547 0.04018 0.04242 1.17% FillPatchIterator::Initialize 41 0.02767 0.03149 0.03367 0.93% Gravity::interpolate_monopole_grav() 21 0.02829 0.02882 0.02951 0.81% Castro::construct_new_gravity_source() 10 0.02257 0.02464 0.02808 0.77% Castro::computeTemp() 63 0.01862 0.01973 0.0206 0.57% Castro::initialize_do_advance() 10 0.01146 0.01716 0.01993 0.55% VisMF::Write(FabArray) 2 0.01168 0.01284 0.0158 0.43% Castro::initialize_advance() 10 0.004101 0.01018 0.01375 0.38% FabArray::Saxpy() 20 0.01066 0.01271 0.01366 0.38% FabArray::setDomainBndry() 41 0.008948 0.01109 0.01244 0.34% MultiFab::contains_nan() 20 0.00989 0.01064 0.01196 0.33% Castro::construct_old_gravity_source() 10 0.006939 0.009657 0.01014 0.28% Castro::initMFs() 1 0.006302 0.007729 0.009859 0.27% Castro::enforce_min_density() 62 0.007424 0.007612 0.007786 0.21% StateData::define() 4 0.005156 0.006138 0.00778 0.21% Castro::construct_new_source() 50 0.005674 0.006306 0.006574 0.18% Gravity::get_new_grav_vector() 11 0.002697 0.003662 0.005462 0.15% main() 1 0.003362 0.004254 0.005046 0.14% Castro::swap_state_time_levels() 10 0.002973 0.004053 0.004837 0.13% Amr::coarseTimeStep() 10 0.001159 0.002085 0.002477 0.07% Gravity::get_old_grav_vector() 10 0.00159 0.001961 0.00234 0.06% Castro::buildMetrics() 1 0.001465 0.00176 0.002314 0.06% Castro::finalize_do_advance() 10 0.0004451 0.001039 0.00131 0.04% Amr::InitAmr() 1 0.0002801 0.0006135 0.001162 0.03% AmrLevel::FillPatch() 41 0.0006965 0.000756 0.0008939 0.02% DistributionMapping::LeastUsedCPUs() 1 6.273e-06 0.0005164 0.0008771 0.02% Amr::timeStep() 10 0.0001661 0.0002367 0.0004374 0.01% Castro::reset_internal_energy(MultiFab) 63 0.0002889 0.0003087 0.0003293 0.01% Castro::do_advance_ctu() 10 0.0001624 0.000174 0.0002034 0.01% Castro::clean_state() 62 0.0001427 0.0001619 0.0001739 0.00% FillPatchIterator::FillFromLevel0() 41 0.0001083 0.0001236 0.0001456 0.00% Castro::enforce_consistent_e() 1 8.225e-05 9.515e-05 0.0001427 0.00% StateDataPhysBCFunct::() 41 8.733e-05 0.0001014 0.0001223 0.00% Castro::Castro() 1 5.547e-05 8.037e-05 0.0001165 0.00% Castro::do_old_sources() 10 7.169e-05 9.91e-05 0.0001162 0.00% Castro::finalize_advance() 10 7.939e-05 0.0001001 0.0001088 0.00% Amr::writeSmallPlotFile() 1 3.183e-05 5.003e-05 9.909e-05 0.00% Castro::construct_old_source() 50 5.503e-05 7.698e-05 8.964e-05 0.00% Amr::FinalizeInit() 1 3.423e-06 1.45e-05 8.403e-05 0.00% Castro::do_new_sources() 10 5.595e-05 6.927e-05 7.996e-05 0.00% FabArray::ParallelCopy() 41 6.95e-05 7.398e-05 7.98e-05 0.00% FillPatchSingleLevel 41 6.112e-05 6.796e-05 7.422e-05 0.00% Amr::defBaseLevel() 1 2.053e-05 3.2e-05 7.057e-05 0.00% Castro::advance() 10 4.097e-05 4.74e-05 7.027e-05 0.00% Castro::construct_old_gravity() 10 4.418e-05 5.18e-05 6.936e-05 0.00% FabArrayBase::getCPC() 41 5.003e-05 5.574e-05 6.846e-05 0.00% Castro::subcycle_advance_ctu() 10 4.41e-05 5.175e-05 6.124e-05 0.00% Castro::apply_source_to_state() 20 3.837e-05 4.829e-05 5.435e-05 0.00% Castro::construct_new_gravity() 10 3.513e-05 4.078e-05 5.401e-05 0.00% AmrLevel::AmrLevel(dm) 1 1.568e-05 2.277e-05 3.364e-05 0.00% Amr::initSubcycle() 1 9.246e-06 1.534e-05 3.345e-05 0.00% FabArrayBase::CPC::define() 2 2.187e-05 2.606e-05 3.109e-05 0.00% Castro::enforce_speed_limit() 62 2.492e-05 2.724e-05 3.061e-05 0.00% Castro::expand_state() 10 2.126e-05 2.476e-05 2.778e-05 0.00% Castro::check_for_nan() 20 1.949e-05 2.052e-05 2.28e-05 0.00% Castro::post_timestep() 10 1.346e-05 1.609e-05 1.874e-05 0.00% Gravity::swapTimeLevels() 10 9.095e-06 1.469e-05 1.817e-05 0.00% Gravity::set_mass_offset() 10 8.733e-06 1.097e-05 1.375e-05 0.00% Castro::computeNewDt() 9 8.961e-06 1.056e-05 1.25e-05 0.00% Castro::create_source_corrector() 10 6.247e-06 8.218e-06 1.008e-05 0.00% Castro::retry_advance_ctu() 10 6.191e-06 7.563e-06 9.258e-06 0.00% Castro::FluxRegCrseInit 10 6.351e-06 7.177e-06 8.341e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 3.542e-06 4.83e-06 7.701e-06 0.00% Castro::post_init() 1 2.638e-06 3.481e-06 4.823e-06 0.00% Castro::computeInitialDt() 2 3.591e-06 4.005e-06 4.587e-06 0.00% Castro::post_regrid() 1 1.546e-06 2.834e-06 4.234e-06 0.00% Amr::InitializeInit() 1 1.82e-06 2.549e-06 3.683e-06 0.00% Castro::FluxRegFineAdd() 10 2.08e-06 2.379e-06 3.229e-06 0.00% Amr::init() 1 1.719e-06 2.155e-06 3.212e-06 0.00% Amr::initialInit() 1 9.97e-07 1.565e-06 2.274e-06 0.00% DistributionMapping::Distribute() 1 1.025e-06 1.258e-06 1.73e-06 0.00% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 3.634 3.635 3.636 100.00% Amr::coarseTimeStep() 10 3.384 3.384 3.384 93.08% Amr::timeStep() 10 3.371 3.372 3.373 92.75% Castro::advance() 10 3.355 3.355 3.356 92.30% Castro::subcycle_advance_ctu() 10 3.275 3.282 3.295 90.63% Castro::do_advance_ctu() 10 3.275 3.282 3.295 90.63% Castro::construct_ctu_hydro_source() 10 2.508 2.511 2.515 69.17% Gravity::make_radial_gravity() 21 0.341 0.3498 0.3623 9.96% Gravity::get_new_grav_vector() 11 0.228 0.2305 0.2324 6.39% AmrLevel::FillPatch() 41 0.18 0.1999 0.2207 6.07% Castro::construct_old_gravity() 10 0.194 0.202 0.2118 5.83% Gravity::get_old_grav_vector() 10 0.1934 0.2014 0.2112 5.81% Castro::construct_new_gravity() 10 0.2032 0.2048 0.206 5.67% Amr::init() 1 0.197 0.1975 0.1979 5.44% FillPatchIterator::Initialize 41 0.152 0.1702 0.1969 5.41% check_for_negative_density() 10 0.01988 0.1224 0.1881 5.17% Amr::initialInit() 1 0.1828 0.1832 0.1837 5.05% Castro::initialize_do_advance() 10 0.1559 0.1684 0.1769 4.86% FillPatchIterator::FillFromLevel0() 41 0.1135 0.1276 0.1603 4.41% FillPatchSingleLevel 41 0.1134 0.1275 0.1601 4.40% Amr::InitializeInit() 1 0.05903 0.1096 0.1541 4.24% Amr::defBaseLevel() 1 0.05903 0.1095 0.1541 4.24% Castro::initData() 1 0.04302 0.09117 0.1363 3.75% Castro::estTimeStep() 21 0.023 0.07528 0.1303 3.58% Castro::expand_state() 10 0.1 0.1141 0.1258 3.46% Amr::FinalizeInit() 1 0.02919 0.07362 0.1238 3.41% FabArray::ParallelCopy() 41 0.07412 0.0893 0.1236 3.40% Castro::clean_state() 62 0.1157 0.118 0.1198 3.29% Castro::computeInitialDt() 2 0.002335 0.04654 0.09659 2.66% Castro::do_old_sources() 10 0.0891 0.09362 0.09656 2.66% FabArray::ParallelCopy_finish() 41 0.03253 0.04897 0.08793 2.42% amrex::Copy() 193 0.06281 0.07559 0.08438 2.32% FabArray::setVal() 143 0.05938 0.06828 0.07807 2.15% Castro::initialize_advance() 10 0.05594 0.06994 0.07742 2.13% Castro::do_new_sources() 10 0.06025 0.06586 0.07455 2.05% Castro::computeTemp() 63 0.06762 0.06925 0.07099 1.95% Amr::writePlotFile() 2 0.06245 0.06282 0.06305 1.73% Castro::reset_internal_energy(MultiFab) 63 0.04832 0.04952 0.05039 1.39% Castro::reset_internal_energy(Fab) 1008 0.04804 0.04921 0.05009 1.38% StateDataPhysBCFunct::() 41 0.03432 0.03814 0.04369 1.20% StateData::FillBoundary(geom) 41 0.03422 0.03804 0.04359 1.20% FabArray::ParallelCopy_nowait() 41 0.03555 0.04026 0.04249 1.17% Castro::normalize_species() 62 0.04144 0.04196 0.04248 1.17% Castro::construct_new_source() 50 0.02875 0.03095 0.03457 0.95% Gravity::interpolate_monopole_grav() 21 0.02829 0.02882 0.02951 0.81% Castro::construct_new_gravity_source() 10 0.02257 0.02464 0.02808 0.77% Castro::finalize_do_advance() 10 0.01151 0.02056 0.02697 0.74% Castro::post_init() 1 0.02651 0.02674 0.02695 0.74% Castro::post_timestep() 10 0.01578 0.01604 0.0164 0.45% VisMF::Write(FabArray) 2 0.01168 0.01284 0.0158 0.43% Castro::Castro() 1 0.009976 0.01166 0.01425 0.39% Castro::apply_source_to_state() 20 0.0107 0.01275 0.01372 0.38% FabArray::Saxpy() 20 0.01066 0.01271 0.01366 0.38% FabArray::setDomainBndry() 41 0.008948 0.01109 0.01244 0.34% Castro::check_for_nan() 20 0.009909 0.01066 0.01198 0.33% MultiFab::contains_nan() 20 0.00989 0.01064 0.01196 0.33% Castro::construct_old_source() 50 0.006994 0.009734 0.01022 0.28% Castro::construct_old_gravity_source() 10 0.006939 0.009657 0.01014 0.28% Castro::initMFs() 1 0.006302 0.007729 0.009859 0.27% Castro::computeNewDt() 9 0.009227 0.009233 0.009242 0.25% AmrLevel::AmrLevel(dm) 1 0.005175 0.006161 0.007811 0.21% Castro::enforce_min_density() 62 0.007424 0.007612 0.007786 0.21% StateData::define() 4 0.005156 0.006138 0.00778 0.21% Castro::swap_state_time_levels() 10 0.002973 0.004053 0.004837 0.13% Castro::finalize_advance() 10 0.003 0.003443 0.003986 0.11% Castro::buildMetrics() 1 0.001586 0.001927 0.002544 0.07% Castro::post_regrid() 1 0.001564 0.001644 0.001849 0.05% Amr::InitAmr() 1 0.0002899 0.0006288 0.001171 0.03% DistributionMapping::SFCProcessorMapDoIt() 1 1.107e-05 0.0005224 0.0008839 0.02% DistributionMapping::LeastUsedCPUs() 1 6.273e-06 0.0005164 0.0008771 0.02% Castro::enforce_consistent_e() 1 8.225e-05 9.515e-05 0.0001427 0.00% Amr::writeSmallPlotFile() 1 3.183e-05 5.003e-05 9.909e-05 0.00% FabArrayBase::getCPC() 41 7.302e-05 8.18e-05 9.401e-05 0.00% Amr::initSubcycle() 1 9.246e-06 1.534e-05 3.345e-05 0.00% FabArrayBase::CPC::define() 2 2.187e-05 2.606e-05 3.109e-05 0.00% Castro::enforce_speed_limit() 62 2.492e-05 2.724e-05 3.061e-05 0.00% Gravity::swapTimeLevels() 10 9.095e-06 1.469e-05 1.817e-05 0.00% Gravity::set_mass_offset() 10 8.733e-06 1.097e-05 1.375e-05 0.00% Castro::create_source_corrector() 10 6.247e-06 8.218e-06 1.008e-05 0.00% Castro::retry_advance_ctu() 10 6.191e-06 7.563e-06 9.258e-06 0.00% Castro::FluxRegCrseInit 10 6.351e-06 7.177e-06 8.341e-06 0.00% Castro::FluxRegFineAdd() 10 2.08e-06 2.379e-06 3.229e-06 0.00% DistributionMapping::Distribute() 1 1.025e-06 1.258e-06 1.73e-06 0.00% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2] [TOP]::amr.regrid_int(nvals = 4) :: [2, 2, 2, 2] Pinned Memory Usage: --------------------------------------------------------------------------------------------------------------------- Name Nalloc Nfree AvgMem min AvgMem avg AvgMem max MaxMem min MaxMem avg MaxMem max --------------------------------------------------------------------------------------------------------------------- The_Pinned_Arena::Initialize() 8 8 30 B 38 B 63 B 8192 KiB 8192 KiB 8192 KiB --------------------------------------------------------------------------------------------------------------------- AMReX (23.09-5-gb98bdae9fb67) finalized