MPI initialized with 16 MPI processes MPI initialized with thread support level 3 AMReX (22.04-37-g3cc4a66759c9) initialized Starting run at 19:11:08 UTC on 2022-04-28. Successfully read inputs file ... Castro git describe: 22.04-41-gfdb304c1e AMReX git describe: 22.04-37-g3cc4a6675 Microphysics git describe: 22.04-3-g3c498521 reading extern runtime parameters ... 3 Species: C12 O16 Mg24 Successfully read inputs file ... Castro::numpts_1d at level 0 is 118 Setting the gravity type to ConstantGrav Warning: the following castro.* parameters are ignored castro.center = 0.0 0.0 0.0 Initializing the data at level 0 Done initializing the level 0 data ...estimated hydro-limited timestep at level 0: 0.003340435764 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.003340435764 Moment of inertia = 2826.515708 TIME= 0 MASS = 0.5935240411 TIME= 0 XMOM = 1.110223025e-16 TIME= 0 YMOM = -1.665334537e-16 TIME= 0 ZMOM = 0 TIME= 0 ANG MOM X = -6.938893904e-18 TIME= 0 ANG MOM Y = -5.551115123e-17 TIME= 0 ANG MOM Z = 4.289006454 TIME= 0 HYB MOM R = 0 TIME= 0 HYB MOM L = 4.289006454 TIME= 0 HYB MOM P = 0 TIME= 0 RHO*e = 0.7105397598 TIME= 0 RHO*K = 13.47431117 TIME= 0 RHO*E = 14.18485093 TIME= 0 RHO*PHI = 0 TIME= 0 TOTAL ENERGY= 14.18485093 INITIAL GRIDS Level 0 16 grids 262144 cells 100 % of domain smallest grid: 32 x 32 x 16 biggest grid: 32 x 32 x 16 CHECKPOINT: file = rotating_torus-restart_chk00000 checkPoint() time = 0.017323999 secs. PLOTFILE: file = rotating_torus-restart_plt00000 Write plotfile time = 0.162907239 seconds ...estimated hydro-limited timestep at level 0: 0.003340435764 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.003340435764 [Level 0 step 1] ADVANCE with dt = 0.0003340435764  Beginning subcycle 1 starting at time 0 with dt = 0.0003340435764 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.008522633 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.270783822 Castro::do_new_sources() time = 0.009163689 ...estimated hydro-limited timestep at level 0: 0.003354140473 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.003354140473  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.7236653504 [Level 0 step 1] Advanced 262144 cells Moment of inertia = 2826.373585 Moment of inertia / initial moment of inertia = 0.9999497183 TIME= 0.0003340435764 MASS = 0.5935240411 TIME= 0.0003340435764 XMOM = -3.330669074e-16 TIME= 0.0003340435764 YMOM = -3.497202528e-15 TIME= 0.0003340435764 ZMOM = -6.350499824e-13 TIME= 0.0003340435764 ANG MOM X = 1.42247325e-16 TIME= 0.0003340435764 ANG MOM Y = 2.775557562e-17 TIME= 0.0003340435764 ANG MOM Z = 4.289006454 TIME= 0.0003340435764 HYB MOM R = 0.001323091462 TIME= 0.0003340435764 HYB MOM L = 4.289006454 TIME= 0.0003340435764 HYB MOM P = -6.350499824e-13 TIME= 0.0003340435764 RHO*e = 0.711807891 TIME= 0.0003340435764 RHO*K = 13.47504927 TIME= 0.0003340435764 RHO*E = 14.18685716 TIME= 0.0003340435764 RHO*PHI = 0 TIME= 0.0003340435764 TOTAL ENERGY= 14.18685716 [STEP 1] Coarse TimeStep time: 0.370904472 [STEP 1] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 1 TIME = 0.0003340435764 DT = 0.0003340435764 ...estimated hydro-limited timestep at level 0: 0.003354140473 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.003354140473 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.003354140473 ... but limiting to: 0.0003507457552 = 1.05 * 0.0003340435764 [Level 0 step 2] ADVANCE with dt = 0.0003507457552  Beginning subcycle 1 starting at time 0.0003340435764 with dt = 0.0003507457552 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.008004434 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.265278194 Castro::do_new_sources() time = 0.009116074 ...estimated hydro-limited timestep at level 0: 0.003373674841 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.003373674841  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.7575714687 [Level 0 step 2] Advanced 262144 cells Moment of inertia = 2826.287536 Moment of inertia / initial moment of inertia = 0.9999192746 TIME= 0.0006847893316 MASS = 0.5935240411 TIME= 0.0006847893316 XMOM = -1.221245327e-15 TIME= 0.0006847893316 YMOM = 9.159339953e-15 TIME= 0.0006847893316 ZMOM = -1.965399313e-12 TIME= 0.0006847893316 ANG MOM X = 1.908195824e-16 TIME= 0.0006847893316 ANG MOM Y = -9.714451465e-17 TIME= 0.0006847893316 ANG MOM Z = 4.289006454 TIME= 0.0006847893316 HYB MOM R = 0.002712341525 TIME= 0.0006847893316 HYB MOM L = 4.289006454 TIME= 0.0006847893316 HYB MOM P = -1.965399313e-12 TIME= 0.0006847893316 RHO*e = 0.712189286 TIME= 0.0006847893316 RHO*K = 13.47551885 TIME= 0.0006847893316 RHO*E = 14.18770813 TIME= 0.0006847893316 RHO*PHI = 0 TIME= 0.0006847893316 TOTAL ENERGY= 14.18770813 [STEP 2] Coarse TimeStep time: 0.359965332 [STEP 2] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 2 TIME = 0.0006847893316 DT = 0.0003507457552 ...estimated hydro-limited timestep at level 0: 0.003373674841 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.003373674841 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.003373674841 ... but limiting to: 0.000368283043 = 1.05 * 0.0003507457552 [Level 0 step 3] ADVANCE with dt = 0.000368283043  Beginning subcycle 1 starting at time 0.0006847893316 with dt = 0.000368283043 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.005994855 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.258689556 Castro::do_new_sources() time = 0.008685568 ...estimated hydro-limited timestep at level 0: 0.00339481947 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.00339481947  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.770629537 [Level 0 step 3] Advanced 262144 cells Moment of inertia = 2826.201161 Moment of inertia / initial moment of inertia = 0.9998887157 TIME= 0.001053072375 MASS = 0.5935240411 TIME= 0.001053072375 XMOM = -1.665334537e-15 TIME= 0.001053072375 YMOM = 3.441691376e-15 TIME= 0.001053072375 ZMOM = -3.356804404e-12 TIME= 0.001053072375 ANG MOM X = 1.040834086e-16 TIME= 0.001053072375 ANG MOM Y = 8.326672685e-17 TIME= 0.001053072375 ANG MOM Z = 4.289006454 TIME= 0.001053072375 HYB MOM R = 0.00417099308 TIME= 0.001053072375 HYB MOM L = 4.289006454 TIME= 0.001053072375 HYB MOM P = -3.356804404e-12 TIME= 0.001053072375 RHO*e = 0.7125668076 TIME= 0.001053072375 RHO*K = 13.47601781 TIME= 0.001053072375 RHO*E = 14.18858462 TIME= 0.001053072375 RHO*PHI = 0 TIME= 0.001053072375 TOTAL ENERGY= 14.18858462 [STEP 3] Coarse TimeStep time: 0.352734959 [STEP 3] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 3 TIME = 0.001053072375 DT = 0.000368283043 ...estimated hydro-limited timestep at level 0: 0.00339481947 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.00339481947 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.00339481947 ... but limiting to: 0.0003866971951 = 1.05 * 0.000368283043 [Level 0 step 4] ADVANCE with dt = 0.0003866971951  Beginning subcycle 1 starting at time 0.001053072375 with dt = 0.0003866971951 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.005381246 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.172680913 Castro::do_new_sources() time = 0.009590874 ...estimated hydro-limited timestep at level 0: 0.002981054165 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002981054165  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.05800997 [Level 0 step 4] Advanced 262144 cells Moment of inertia = 2826.122103 Moment of inertia / initial moment of inertia = 0.9998607457 TIME= 0.00143976957 MASS = 0.5935240411 TIME= 0.00143976957 XMOM = 8.881784197e-16 TIME= 0.00143976957 YMOM = 5.440092821e-15 TIME= 0.00143976957 ZMOM = -4.815166034e-12 TIME= 0.00143976957 ANG MOM X = 6.938893904e-17 TIME= 0.00143976957 ANG MOM Y = 1.387778781e-17 TIME= 0.00143976957 ANG MOM Z = 4.289006454 TIME= 0.00143976957 HYB MOM R = 0.005702512117 TIME= 0.00143976957 HYB MOM L = 4.289006454 TIME= 0.00143976957 HYB MOM P = -4.815166034e-12 TIME= 0.00143976957 RHO*e = 0.7129260318 TIME= 0.00143976957 RHO*K = 13.47651236 TIME= 0.00143976957 RHO*E = 14.18943839 TIME= 0.00143976957 RHO*PHI = 0 TIME= 0.00143976957 TOTAL ENERGY= 14.18943839 [STEP 4] Coarse TimeStep time: 0.257797655 [STEP 4] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 4 TIME = 0.00143976957 DT = 0.0003866971951 ...estimated hydro-limited timestep at level 0: 0.002981054165 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002981054165 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.002981054165 ... but limiting to: 0.0004060320549 = 1.05 * 0.0003866971951 [Level 0 step 5] ADVANCE with dt = 0.0004060320549  Beginning subcycle 1 starting at time 0.00143976957 with dt = 0.0004060320549 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.007236307 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.176055624 Castro::do_new_sources() time = 0.009700666 ...estimated hydro-limited timestep at level 0: 0.002422517818 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002422517818  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.065899424 [Level 0 step 5] Advanced 262144 cells Moment of inertia = 2826.060054 Moment of inertia / initial moment of inertia = 0.9998387932 TIME= 0.001845801625 MASS = 0.5935240411 TIME= 0.001845801625 XMOM = 8.881784197e-16 TIME= 0.001845801625 YMOM = 7.827072324e-15 TIME= 0.001845801625 ZMOM = -6.346287035e-12 TIME= 0.001845801625 ANG MOM X = -6.938893904e-18 TIME= 0.001845801625 ANG MOM Y = 1.873501354e-16 TIME= 0.001845801625 ANG MOM Z = 4.289006454 TIME= 0.001845801625 HYB MOM R = 0.007310517316 TIME= 0.001845801625 HYB MOM L = 4.289006454 TIME= 0.001845801625 HYB MOM P = -6.346287035e-12 TIME= 0.001845801625 RHO*e = 0.7132495392 TIME= 0.001845801625 RHO*K = 13.47695837 TIME= 0.001845801625 RHO*E = 14.19020791 TIME= 0.001845801625 RHO*PHI = 0 TIME= 0.001845801625 TOTAL ENERGY= 14.19020791 [STEP 5] Coarse TimeStep time: 0.25488582 [STEP 5] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 5 TIME = 0.001845801625 DT = 0.0004060320549 CHECKPOINT: file = rotating_torus-restart_chk00005 checkPoint() time = 0.017166835 secs. ...estimated hydro-limited timestep at level 0: 0.002422517818 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002422517818 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.002422517818 ... but limiting to: 0.0004263336576 = 1.05 * 0.0004060320549 [Level 0 step 6] ADVANCE with dt = 0.0004263336576  Beginning subcycle 1 starting at time 0.001845801625 with dt = 0.0004263336576 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.006328846 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.16766808 Castro::do_new_sources() time = 0.009193554 ...estimated hydro-limited timestep at level 0: 0.002054026253 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002054026253  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.117895454 [Level 0 step 6] Advanced 262144 cells Moment of inertia = 2826.017073 Moment of inertia / initial moment of inertia = 0.9998235867 TIME= 0.002272135282 MASS = 0.5935240411 TIME= 0.002272135282 XMOM = 4.440892099e-16 TIME= 0.002272135282 YMOM = 6.827871601e-15 TIME= 0.002272135282 ZMOM = -7.954161418e-12 TIME= 0.002272135282 ANG MOM X = -1.387778781e-17 TIME= 0.002272135282 ANG MOM Y = -1.179611964e-16 TIME= 0.002272135282 ANG MOM Z = 4.289006454 TIME= 0.002272135282 HYB MOM R = 0.008998825225 TIME= 0.002272135282 HYB MOM L = 4.289006454 TIME= 0.002272135282 HYB MOM P = -7.954161418e-12 TIME= 0.002272135282 RHO*e = 0.7135428282 TIME= 0.002272135282 RHO*K = 13.47734917 TIME= 0.002272135282 RHO*E = 14.190892 TIME= 0.002272135282 RHO*PHI = 0 TIME= 0.002272135282 TOTAL ENERGY= 14.190892 [STEP 6] Coarse TimeStep time: 0.242759321 [STEP 6] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 6 TIME = 0.002272135282 DT = 0.0004263336576 ...estimated hydro-limited timestep at level 0: 0.002054026253 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002054026253 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.002054026253 ... but limiting to: 0.0004476503405 = 1.05 * 0.0004263336576 [Level 0 step 7] ADVANCE with dt = 0.0004476503405  Beginning subcycle 1 starting at time 0.002272135282 with dt = 0.0004476503405 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.004131656 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.157701087 Castro::do_new_sources() time = 0.007305682 ...estimated hydro-limited timestep at level 0: 0.001791126456 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001791126456  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.262153732 [Level 0 step 7] Advanced 262144 cells Moment of inertia = 2825.989879 Moment of inertia / initial moment of inertia = 0.999813966 TIME= 0.002719785623 MASS = 0.5935240411 TIME= 0.002719785623 XMOM = -1.887379142e-15 TIME= 0.002719785623 YMOM = 3.774758284e-15 TIME= 0.002719785623 ZMOM = -9.642922067e-12 TIME= 0.002719785623 ANG MOM X = -9.36750677e-17 TIME= 0.002719785623 ANG MOM Y = -2.775557562e-17 TIME= 0.002719785623 ANG MOM Z = 4.289006454 TIME= 0.002719785623 HYB MOM R = 0.01077138894 TIME= 0.002719785623 HYB MOM L = 4.289006454 TIME= 0.002719785623 HYB MOM P = -9.642922067e-12 TIME= 0.002719785623 RHO*e = 0.7138174014 TIME= 0.002719785623 RHO*K = 13.47770671 TIME= 0.002719785623 RHO*E = 14.19152411 TIME= 0.002719785623 RHO*PHI = 0 TIME= 0.002719785623 TOTAL ENERGY= 14.19152411 [STEP 7] Coarse TimeStep time: 0.216014377 [STEP 7] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 7 TIME = 0.002719785623 DT = 0.0004476503405 ...estimated hydro-limited timestep at level 0: 0.001791126456 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001791126456 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.001791126456 ... but limiting to: 0.0004700328575 = 1.05 * 0.0004476503405 [Level 0 step 8] ADVANCE with dt = 0.0004700328575  Beginning subcycle 1 starting at time 0.002719785623 with dt = 0.0004700328575 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.004011609 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.157386748 Castro::do_new_sources() time = 0.005116751 ...estimated hydro-limited timestep at level 0: 0.001590548666 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001590548666  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.311209763 [Level 0 step 8] Advanced 262144 cells Moment of inertia = 2825.975647 Moment of inertia / initial moment of inertia = 0.9998089305 TIME= 0.00318981848 MASS = 0.5935240411 TIME= 0.00318981848 XMOM = 9.992007222e-16 TIME= 0.00318981848 YMOM = 1.665334537e-16 TIME= 0.00318981848 ZMOM = -1.141697121e-11 TIME= 0.00318981848 ANG MOM X = 6.938893904e-18 TIME= 0.00318981848 ANG MOM Y = -6.245004514e-17 TIME= 0.00318981848 ANG MOM Z = 4.289006454 TIME= 0.00318981848 HYB MOM R = 0.01263236378 TIME= 0.00318981848 HYB MOM L = 4.289006454 TIME= 0.00318981848 HYB MOM P = -1.141697121e-11 TIME= 0.00318981848 RHO*e = 0.7140922963 TIME= 0.00318981848 RHO*K = 13.47805002 TIME= 0.00318981848 RHO*E = 14.19214232 TIME= 0.00318981848 RHO*PHI = 0 TIME= 0.00318981848 TOTAL ENERGY= 14.19214232 [STEP 8] Coarse TimeStep time: 0.207405105 [STEP 8] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 8 TIME = 0.00318981848 DT = 0.0004700328575 ...estimated hydro-limited timestep at level 0: 0.001590548666 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001590548666 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.001590548666 ... but limiting to: 0.0004935345004 = 1.05 * 0.0004700328575 [Level 0 step 9] ADVANCE with dt = 0.0004935345004  Beginning subcycle 1 starting at time 0.00318981848 with dt = 0.0004935345004 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.003973611 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.156303298 Castro::do_new_sources() time = 0.005158557 ...estimated hydro-limited timestep at level 0: 0.001432337647 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001432337647  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.312184286 [Level 0 step 9] Advanced 262144 cells Moment of inertia = 2825.983195 Moment of inertia / initial moment of inertia = 0.999811601 TIME= 0.003683352981 MASS = 0.5935240411 TIME= 0.003683352981 XMOM = -5.551115123e-16 TIME= 0.003683352981 YMOM = -2.498001805e-15 TIME= 0.003683352981 ZMOM = -1.32791597e-11 TIME= 0.003683352981 ANG MOM X = 9.020562075e-17 TIME= 0.003683352981 ANG MOM Y = 1.387778781e-17 TIME= 0.003683352981 ANG MOM Z = 4.289006454 TIME= 0.003683352981 HYB MOM R = 0.01458610189 TIME= 0.003683352981 HYB MOM L = 4.289006454 TIME= 0.003683352981 HYB MOM P = -1.32791597e-11 TIME= 0.003683352981 RHO*e = 0.7143408385 TIME= 0.003683352981 RHO*K = 13.47834086 TIME= 0.003683352981 RHO*E = 14.19268169 TIME= 0.003683352981 RHO*PHI = 0 TIME= 0.003683352981 TOTAL ENERGY= 14.19268169 [STEP 9] Coarse TimeStep time: 0.207299982 [STEP 9] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 9 TIME = 0.003683352981 DT = 0.0004935345004 ...estimated hydro-limited timestep at level 0: 0.001432337647 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001432337647 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.001432337647 ... but limiting to: 0.0005182112254 = 1.05 * 0.0004935345004 [Level 0 step 10] ADVANCE with dt = 0.0005182112254  Beginning subcycle 1 starting at time 0.003683352981 with dt = 0.0005182112254 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.004026712 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.155982489 Castro::do_new_sources() time = 0.005194621 ...estimated hydro-limited timestep at level 0: 0.001305227986 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001305227986  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.313067318 [Level 0 step 10] Advanced 262144 cells Moment of inertia = 2826.014035 Moment of inertia / initial moment of inertia = 0.9998225119 TIME= 0.004201564206 MASS = 0.5935240411 TIME= 0.004201564206 XMOM = -4.329869796e-15 TIME= 0.004201564206 YMOM = -5.939693182e-15 TIME= 0.004201564206 ZMOM = -1.523249403e-11 TIME= 0.004201564206 ANG MOM X = -1.630640067e-16 TIME= 0.004201564206 ANG MOM Y = 4.163336342e-17 TIME= 0.004201564206 ANG MOM Z = 4.289006454 TIME= 0.004201564206 HYB MOM R = 0.01663716124 TIME= 0.004201564206 HYB MOM L = 4.289006454 TIME= 0.004201564206 HYB MOM P = -1.523249403e-11 TIME= 0.004201564206 RHO*e = 0.7145679066 TIME= 0.004201564206 RHO*K = 13.47857813 TIME= 0.004201564206 RHO*E = 14.19314603 TIME= 0.004201564206 RHO*PHI = 0 TIME= 0.004201564206 TOTAL ENERGY= 14.19314603 [STEP 10] Coarse TimeStep time: 0.207154614 [STEP 10] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 10 TIME = 0.004201564206 DT = 0.0005182112254 CHECKPOINT: file = rotating_torus-restart_chk00010 checkPoint() time = 0.014204428 secs. PLOTFILE: file = rotating_torus-restart_plt00010 Write plotfile time = 0.199536069 seconds Ending run at 19:11:11 UTC on 2022-04-28. Run time = 3.131452607 Run time without initialization = 2.909758609 Average number of zones advanced per microsecond: 0.901 Average number of zones advanced per microsecond per rank: 0.056 TinyProfiler total time across processes [min...avg...max]: 3.131 ... 3.131 ... 3.132 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 10 1.4674 1.7334 1.9495 62.25% Castro::do_advance_ctu() 10 0.02924 0.248 0.5239 16.73% Amr::writePlotFile() 2 0.3429 0.35 0.3512 11.22% FabArray::ParallelCopy_finish() 251 0.05607 0.09095 0.1356 4.33% FabArray::setVal() 164 0.07938 0.0866 0.09167 2.93% Castro::reset_internal_energy(Fab) 504 0.05342 0.05873 0.06673 2.13% VisMF::Write(FabArray) 11 0.04389 0.04769 0.06134 1.96% Castro::estTimeStep() 21 0.01908 0.03611 0.05447 1.74% FabArray::ParallelCopy_nowait() 251 0.03149 0.04101 0.0516 1.65% StateData::FillBoundary(geom) 20 0.02055 0.03226 0.05024 1.60% Castro::do_old_sources() 10 0.0001686 0.01201 0.04546 1.45% FillPatchIterator::Initialize 251 0.03157 0.03666 0.04196 1.34% Castro::post_timestep() 10 0.004727 0.02127 0.04138 1.32% Castro::computeTemp() 63 0.03305 0.03659 0.04114 1.31% Castro::finalize_do_advance() 10 0.002006 0.02579 0.03775 1.21% Castro::normalize_species() 62 0.02384 0.02722 0.03203 1.02% Castro::initialize_do_advance() 10 0.02063 0.02564 0.02943 0.94% Castro::expand_state() 10 0.01722 0.02219 0.02917 0.93% Castro::construct_new_gravity_source() 10 0.02054 0.02222 0.0245 0.78% Castro::initialize_advance() 10 0.01549 0.01858 0.02365 0.76% Castro::hybrid_to_linear_momentum() 62 0.01594 0.01852 0.02207 0.70% Amr::checkPoint() 3 0.00852 0.01555 0.01884 0.60% MultiFab::contains_nan() 20 0.00961 0.01265 0.01703 0.54% MultiFab::Saxpy() 20 0.01195 0.01434 0.01633 0.52% Castro::do_new_sources() 10 0.0001463 0.00481 0.01383 0.44% Gravity::get_new_grav_vector() 11 0.00716 0.009007 0.01358 0.43% FabArray::setDomainBndry() 251 0.006278 0.008855 0.01301 0.42% Castro::derive() 187 0.007966 0.009446 0.01161 0.37% Gravity::add_pointmass_to_gravity() 21 0.007696 0.008697 0.00992 0.32% Castro::construct_old_gravity_source() 10 0.005199 0.007439 0.009489 0.30% Gravity::get_old_grav_vector() 10 0.004571 0.005641 0.00739 0.24% Castro::fill_hybrid_hydro_source() 30 0.005072 0.006178 0.006994 0.22% Castro::initMFs() 1 0.003668 0.005419 0.00636 0.20% Castro::volWgtSum() 176 0.005169 0.005511 0.005889 0.19% Castro::enforce_min_density() 62 0.004217 0.004716 0.005373 0.17% Castro::post_init() 1 0.0007646 0.002103 0.00519 0.17% StateData::define() 4 0.003396 0.004628 0.005128 0.16% Castro::swap_state_time_levels() 10 0.004036 0.004346 0.004899 0.16% Castro::initData() 1 0.001797 0.002493 0.002971 0.09% main() 1 0.0009304 0.001155 0.001581 0.05% Castro::buildMetrics() 1 0.0008311 0.001152 0.0013 0.04% Castro::finalize_advance() 10 0.0009452 0.001073 0.001203 0.04% Castro::subcycle_advance_ctu() 10 5.906e-05 0.0001402 0.001129 0.04% Amr::InitAmr() 1 0.0002677 0.0004135 0.0006294 0.02% Castro::locSquaredSum() 11 0.0004021 0.0004682 0.0005551 0.02% Amr::coarseTimeStep() 10 0.000271 0.0003463 0.0004578 0.01% DistributionMapping::LeastUsedCPUs() 1 1.256e-05 0.0002672 0.0004496 0.01% Castro::reset_internal_energy(MultiFab) 63 0.0003172 0.0003711 0.000448 0.01% Castro::clean_state() 62 0.0002336 0.0002869 0.0003406 0.01% StateData::checkPoint() 12 2.083e-05 4.73e-05 0.0003011 0.01% StateDataPhysBCFunct::() 251 0.0002243 0.0002525 0.0002894 0.01% Castro::linear_to_hybrid_momentum() 1 0.0002088 0.0002315 0.0002548 0.01% Castro::Castro() 1 8.293e-05 0.0001416 0.0002333 0.01% Castro::computeNewDt() 9 1.136e-05 2.534e-05 0.0002117 0.01% FillPatchSingleLevel 251 0.000154 0.0001824 0.0002109 0.01% FabArray::ParallelCopy() 251 0.0001437 0.0001673 0.0001933 0.01% Castro::enforce_consistent_e() 1 5.98e-05 9.356e-05 0.0001782 0.01% Amr::timeStep() 10 5.545e-05 6.872e-05 0.0001678 0.01% Castro::construct_new_source() 70 8.708e-05 0.0001243 0.0001586 0.01% Castro::construct_old_gravity() 10 6.13e-05 8.447e-05 0.0001472 0.00% AmrLevel::checkPoint() 3 5.038e-05 7.411e-05 0.0001432 0.00% Castro::construct_new_gravity() 10 3.82e-05 6.677e-05 0.0001099 0.00% Castro::construct_old_source() 70 6.459e-05 8.451e-05 0.0001094 0.00% FabArrayBase::CPC::define() 2 3.552e-05 5.16e-05 9.896e-05 0.00% Amr::writeSmallPlotFile() 1 2.492e-05 7.041e-05 9.678e-05 0.00% Castro::advance() 10 4.922e-05 6.204e-05 8.603e-05 0.00% Castro::apply_source_to_state() 20 5.062e-05 5.933e-05 7.185e-05 0.00% FabArrayBase::getCPC() 20 3.768e-05 4.693e-05 5.897e-05 0.00% Castro::check_for_nan() 20 2.945e-05 4.182e-05 5.806e-05 0.00% Castro::enforce_speed_limit() 62 4.071e-05 4.973e-05 5.72e-05 0.00% Amr::FinalizeInit() 1 4.642e-06 8.688e-06 5.308e-05 0.00% Amr::defBaseLevel() 1 3.125e-05 4.092e-05 4.794e-05 0.00% Castro::construct_new_hybrid_source() 10 2.175e-05 3.199e-05 4.25e-05 0.00% Gravity::swapTimeLevels() 10 1.866e-05 2.515e-05 3.435e-05 0.00% Castro::create_source_corrector() 10 1.022e-05 1.911e-05 2.831e-05 0.00% Castro::construct_old_hybrid_source() 10 1.389e-05 2.059e-05 2.737e-05 0.00% AmrLevel::AmrLevel(dm) 1 9.522e-06 1.234e-05 1.724e-05 0.00% Amr::initSubcycle() 1 9.052e-06 1.433e-05 1.71e-05 0.00% Amr::InitializeInit() 1 2.864e-06 4.344e-06 1.622e-05 0.00% Castro::construct_new_rotation_source() 10 8.705e-06 1.209e-05 1.468e-05 0.00% Castro::retry_advance_ctu() 10 6.113e-06 1.006e-05 1.297e-05 0.00% Castro::FluxRegCrseInit 10 7.901e-06 9.701e-06 1.149e-05 0.00% Castro::construct_old_rotation_source() 10 4.514e-06 8.354e-06 1.115e-05 0.00% Castro::gwstrain() 11 5.845e-06 7.788e-06 1.004e-05 0.00% Gravity::set_mass_offset() 10 6.695e-06 8.16e-06 9.275e-06 0.00% AmrLevel::checkPointPost() 3 4.89e-06 6.32e-06 8.273e-06 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 5.507e-06 6.739e-06 8.168e-06 0.00% AmrLevel::checkPointPre() 3 3.147e-06 5.295e-06 7.295e-06 0.00% Castro::computeInitialDt() 2 3.486e-06 4.15e-06 5.506e-06 0.00% Castro::FluxRegFineAdd() 10 3.21e-06 4.433e-06 5.379e-06 0.00% Castro::post_regrid() 1 2.711e-06 3.814e-06 4.754e-06 0.00% Amr::init() 1 2.742e-06 3.408e-06 4.136e-06 0.00% Amr::initialInit() 1 1.459e-06 2.299e-06 3.382e-06 0.00% DistributionMapping::Distribute() 1 1.561e-06 2.265e-06 3.036e-06 0.00% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 3.131 3.131 3.131 100.00% Amr::coarseTimeStep() 10 2.709 2.709 2.709 86.50% Amr::timeStep() 10 2.668 2.668 2.668 85.19% Castro::advance() 10 2.584 2.608 2.62 83.66% Castro::subcycle_advance_ctu() 10 2.484 2.511 2.525 80.63% Castro::do_advance_ctu() 10 2.483 2.511 2.525 80.63% Castro::construct_ctu_hydro_source() 10 1.467 1.733 1.949 62.25% Amr::writePlotFile() 2 0.3627 0.3627 0.3627 11.58% FillPatchIterator::Initialize 251 0.1739 0.2104 0.2394 7.65% Castro::initialize_do_advance() 10 0.1997 0.2141 0.2379 7.60% Amr::init() 1 0.2209 0.2211 0.2213 7.07% FillPatchSingleLevel 251 0.129 0.1649 0.2005 6.40% FabArray::ParallelCopy() 251 0.09112 0.1322 0.1795 5.73% Castro::clean_state() 62 0.1306 0.1447 0.166 5.30% Castro::expand_state() 10 0.1321 0.1445 0.1587 5.07% FabArray::ParallelCopy_finish() 251 0.05607 0.09095 0.1356 4.33% Castro::computeTemp() 63 0.08682 0.09569 0.1082 3.46% Castro::initialize_advance() 10 0.08763 0.09538 0.1053 3.36% Castro::do_old_sources() 10 0.03969 0.05721 0.09478 3.03% FabArray::setVal() 164 0.07938 0.0866 0.09167 2.93% Castro::do_new_sources() 10 0.06474 0.07448 0.08512 2.72% Castro::post_timestep() 10 0.04792 0.05994 0.0838 2.68% Castro::reset_internal_energy(MultiFab) 63 0.05376 0.0591 0.06717 2.15% Castro::reset_internal_energy(Fab) 504 0.05342 0.05873 0.06673 2.13% VisMF::Write(FabArray) 11 0.04389 0.04769 0.06134 1.96% Castro::estTimeStep() 21 0.01908 0.03611 0.05447 1.74% FabArray::ParallelCopy_nowait() 251 0.03158 0.04111 0.05171 1.65% Amr::checkPoint() 3 0.05049 0.05061 0.05063 1.62% StateDataPhysBCFunct::() 251 0.02079 0.03252 0.05052 1.61% StateData::FillBoundary(geom) 20 0.02055 0.03226 0.05024 1.60% AmrLevel::checkPoint() 3 0.03178 0.03505 0.04195 1.34% StateData::checkPoint() 12 0.03173 0.03498 0.04181 1.34% Amr::initialInit() 1 0.03889 0.03903 0.03921 1.25% Castro::finalize_do_advance() 10 0.002006 0.02579 0.03775 1.21% Castro::normalize_species() 62 0.02384 0.02722 0.03203 1.02% Castro::construct_new_source() 70 0.02404 0.02655 0.02891 0.92% Castro::volWgtSum() 176 0.02146 0.02415 0.02758 0.88% Amr::InitializeInit() 1 0.01692 0.02401 0.02735 0.87% Amr::defBaseLevel() 1 0.01692 0.02401 0.02735 0.87% Castro::construct_new_gravity_source() 10 0.02054 0.02222 0.0245 0.78% Castro::derive() 187 0.01692 0.01939 0.02261 0.72% Amr::FinalizeInit() 1 0.01161 0.01502 0.02228 0.71% Castro::hybrid_to_linear_momentum() 62 0.01594 0.01852 0.02207 0.70% Gravity::get_new_grav_vector() 11 0.0121 0.01495 0.02099 0.67% Castro::construct_new_gravity() 10 0.01142 0.01414 0.0206 0.66% Castro::check_for_nan() 20 0.009644 0.0127 0.01709 0.55% MultiFab::contains_nan() 20 0.00961 0.01265 0.01703 0.54% Castro::apply_source_to_state() 20 0.012 0.01439 0.0164 0.52% MultiFab::Saxpy() 20 0.01195 0.01434 0.01633 0.52% Castro::computeInitialDt() 2 0.002397 0.005794 0.01306 0.42% FabArray::setDomainBndry() 251 0.006278 0.008855 0.01301 0.42% Castro::construct_old_gravity() 10 0.01023 0.01148 0.01293 0.41% Gravity::get_old_grav_vector() 10 0.00952 0.01071 0.01194 0.38% Castro::construct_old_source() 70 0.006935 0.009563 0.0118 0.38% Castro::Castro() 1 0.005944 0.00977 0.01152 0.37% Castro::initData() 1 0.007069 0.009279 0.01048 0.33% Gravity::add_pointmass_to_gravity() 21 0.007696 0.008697 0.00992 0.32% Castro::construct_old_gravity_source() 10 0.005199 0.007439 0.009489 0.30% Castro::post_init() 1 0.007125 0.007661 0.008587 0.27% Castro::computeNewDt() 9 0.00772 0.007823 0.007883 0.25% Castro::fill_hybrid_hydro_source() 30 0.005072 0.006178 0.006994 0.22% Castro::initMFs() 1 0.003668 0.005419 0.00636 0.20% Castro::enforce_min_density() 62 0.004217 0.004716 0.005373 0.17% AmrLevel::AmrLevel(dm) 1 0.003407 0.00464 0.005141 0.16% StateData::define() 4 0.003396 0.004628 0.005128 0.16% Castro::swap_state_time_levels() 10 0.004036 0.004346 0.004899 0.16% Castro::construct_new_hybrid_source() 10 0.003327 0.004199 0.004829 0.15% Castro::post_regrid() 1 0.001728 0.002658 0.003184 0.10% Castro::construct_old_hybrid_source() 10 0.001667 0.002031 0.002351 0.08% Castro::locSquaredSum() 11 0.001039 0.001221 0.001589 0.05% Castro::buildMetrics() 1 0.0009043 0.001294 0.001455 0.05% Castro::finalize_advance() 10 0.0009593 0.001087 0.001218 0.04% Amr::InitAmr() 1 0.0002768 0.0004278 0.000646 0.02% DistributionMapping::SFCProcessorMapDoIt() 1 2.237e-05 0.0002762 0.0004595 0.01% DistributionMapping::LeastUsedCPUs() 1 1.256e-05 0.0002672 0.0004496 0.01% Castro::linear_to_hybrid_momentum() 1 0.0002088 0.0002315 0.0002548 0.01% Castro::enforce_consistent_e() 1 5.98e-05 9.356e-05 0.0001782 0.01% FabArrayBase::getCPC() 20 7.694e-05 9.853e-05 0.0001526 0.00% FabArrayBase::CPC::define() 2 3.552e-05 5.16e-05 9.896e-05 0.00% Amr::writeSmallPlotFile() 1 2.492e-05 7.041e-05 9.678e-05 0.00% Castro::enforce_speed_limit() 62 4.071e-05 4.973e-05 5.72e-05 0.00% Gravity::swapTimeLevels() 10 1.866e-05 2.515e-05 3.435e-05 0.00% Castro::create_source_corrector() 10 1.022e-05 1.911e-05 2.831e-05 0.00% Amr::initSubcycle() 1 9.052e-06 1.433e-05 1.71e-05 0.00% Castro::construct_new_rotation_source() 10 8.705e-06 1.209e-05 1.468e-05 0.00% Castro::retry_advance_ctu() 10 6.113e-06 1.006e-05 1.297e-05 0.00% Castro::FluxRegCrseInit 10 7.901e-06 9.701e-06 1.149e-05 0.00% Castro::construct_old_rotation_source() 10 4.514e-06 8.354e-06 1.115e-05 0.00% Castro::gwstrain() 11 5.845e-06 7.788e-06 1.004e-05 0.00% Gravity::set_mass_offset() 10 6.695e-06 8.16e-06 9.275e-06 0.00% AmrLevel::checkPointPost() 3 4.89e-06 6.32e-06 8.273e-06 0.00% AmrLevel::checkPointPre() 3 3.147e-06 5.295e-06 7.295e-06 0.00% Castro::FluxRegFineAdd() 10 3.21e-06 4.433e-06 5.379e-06 0.00% DistributionMapping::Distribute() 1 1.561e-06 2.265e-06 3.036e-06 0.00% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::castro.center(nvals = 3) :: [0.0, 0.0, 0.0] [TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2] [TOP]::amr.regrid_int(nvals = 1) :: [-1] [TOP]::problme.outer_radius(nvals = 1) :: [1.50] AMReX (22.04-37-g3cc4a66759c9) finalized MPI initialized with 16 MPI processes MPI initialized with thread support level 3 AMReX (22.04-37-g3cc4a66759c9) initialized Starting run at 19:11:11 UTC on 2022-04-28. Successfully read inputs file ... Castro git describe: 22.04-41-gfdb304c1e AMReX git describe: 22.04-37-g3cc4a6675 Microphysics git describe: 22.04-3-g3c498521 reading extern runtime parameters ... 3 Species: C12 O16 Mg24 restarting calculation from file: rotating_torus-restart_chk00005 Successfully read inputs file ... read CPU time: 29.3602669 Castro::numpts_1d at level 0 is 118 Moment of inertia = 2826.060054 Restart time = 0.037693385 seconds. ...estimated hydro-limited timestep at level 0: 0.002422517818 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002422517818 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.002422517818 ... but limiting to: 0.0004263336576 = 1.05 * 0.0004060320549 [Level 0 step 6] ADVANCE with dt = 0.0004263336576  Beginning subcycle 1 starting at time 0.001845801625 with dt = 0.0004263336576 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.008200387 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.271812571 Castro::do_new_sources() time = 0.009175896 ...estimated hydro-limited timestep at level 0: 0.002054026253 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002054026253  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.7164671579 [Level 0 step 6] Advanced 262144 cells Moment of inertia = 2826.017073 Moment of inertia / initial moment of inertia = 0.9999847911 TIME= 0.002272135282 MASS = 0.5935240411 TIME= 0.002272135282 XMOM = 4.440892099e-16 TIME= 0.002272135282 YMOM = 6.827871601e-15 TIME= 0.002272135282 ZMOM = -7.954161418e-12 TIME= 0.002272135282 ANG MOM X = -1.387778781e-17 TIME= 0.002272135282 ANG MOM Y = -1.179611964e-16 TIME= 0.002272135282 ANG MOM Z = 4.289006454 TIME= 0.002272135282 HYB MOM R = 0.008998825225 TIME= 0.002272135282 HYB MOM L = 4.289006454 TIME= 0.002272135282 HYB MOM P = -7.954161418e-12 TIME= 0.002272135282 RHO*e = 0.7135428282 TIME= 0.002272135282 RHO*K = 13.47734917 TIME= 0.002272135282 RHO*E = 14.190892 TIME= 0.002272135282 RHO*PHI = 0 TIME= 0.002272135282 TOTAL ENERGY= 14.190892 [STEP 6] Coarse TimeStep time: 0.422213018 [STEP 6] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 6 TIME = 0.002272135282 DT = 0.0004263336576 ...estimated hydro-limited timestep at level 0: 0.002054026253 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.002054026253 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.002054026253 ... but limiting to: 0.0004476503405 = 1.05 * 0.0004263336576 [Level 0 step 7] ADVANCE with dt = 0.0004476503405  Beginning subcycle 1 starting at time 0.002272135282 with dt = 0.0004476503405 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.006001353 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.26180404 Castro::do_new_sources() time = 0.009127288 ...estimated hydro-limited timestep at level 0: 0.001791126456 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001791126456  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.7515128789 [Level 0 step 7] Advanced 262144 cells Moment of inertia = 2825.989879 Moment of inertia / initial moment of inertia = 0.9999751688 TIME= 0.002719785623 MASS = 0.5935240411 TIME= 0.002719785623 XMOM = -1.887379142e-15 TIME= 0.002719785623 YMOM = 3.774758284e-15 TIME= 0.002719785623 ZMOM = -9.642922067e-12 TIME= 0.002719785623 ANG MOM X = -9.36750677e-17 TIME= 0.002719785623 ANG MOM Y = -2.775557562e-17 TIME= 0.002719785623 ANG MOM Z = 4.289006454 TIME= 0.002719785623 HYB MOM R = 0.01077138894 TIME= 0.002719785623 HYB MOM L = 4.289006454 TIME= 0.002719785623 HYB MOM P = -9.642922067e-12 TIME= 0.002719785623 RHO*e = 0.7138174014 TIME= 0.002719785623 RHO*K = 13.47770671 TIME= 0.002719785623 RHO*E = 14.19152411 TIME= 0.002719785623 RHO*PHI = 0 TIME= 0.002719785623 TOTAL ENERGY= 14.19152411 [STEP 7] Coarse TimeStep time: 0.359775928 [STEP 7] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 7 TIME = 0.002719785623 DT = 0.0004476503405 ...estimated hydro-limited timestep at level 0: 0.001791126456 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001791126456 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.001791126456 ... but limiting to: 0.0004700328575 = 1.05 * 0.0004476503405 [Level 0 step 8] ADVANCE with dt = 0.0004700328575  Beginning subcycle 1 starting at time 0.002719785623 with dt = 0.0004700328575 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.005749811 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.258728195 Castro::do_new_sources() time = 0.008700895 ...estimated hydro-limited timestep at level 0: 0.001590548666 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001590548666  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 0.7676113458 [Level 0 step 8] Advanced 262144 cells Moment of inertia = 2825.975647 Moment of inertia / initial moment of inertia = 0.9999701325 TIME= 0.00318981848 MASS = 0.5935240411 TIME= 0.00318981848 XMOM = 9.992007222e-16 TIME= 0.00318981848 YMOM = 1.665334537e-16 TIME= 0.00318981848 ZMOM = -1.141697121e-11 TIME= 0.00318981848 ANG MOM X = 6.938893904e-18 TIME= 0.00318981848 ANG MOM Y = -6.245004514e-17 TIME= 0.00318981848 ANG MOM Z = 4.289006454 TIME= 0.00318981848 HYB MOM R = 0.01263236378 TIME= 0.00318981848 HYB MOM L = 4.289006454 TIME= 0.00318981848 HYB MOM P = -1.141697121e-11 TIME= 0.00318981848 RHO*e = 0.7140922963 TIME= 0.00318981848 RHO*K = 13.47805002 TIME= 0.00318981848 RHO*E = 14.19214232 TIME= 0.00318981848 RHO*PHI = 0 TIME= 0.00318981848 TOTAL ENERGY= 14.19214232 [STEP 8] Coarse TimeStep time: 0.350222767 [STEP 8] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 8 TIME = 0.00318981848 DT = 0.0004700328575 ...estimated hydro-limited timestep at level 0: 0.001590548666 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001590548666 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.001590548666 ... but limiting to: 0.0004935345004 = 1.05 * 0.0004700328575 [Level 0 step 9] ADVANCE with dt = 0.0004935345004  Beginning subcycle 1 starting at time 0.00318981848 with dt = 0.0004935345004 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.00719915 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.172225697 Castro::do_new_sources() time = 0.009465562 ...estimated hydro-limited timestep at level 0: 0.001432337647 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001432337647  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.057105343 [Level 0 step 9] Advanced 262144 cells Moment of inertia = 2825.983195 Moment of inertia / initial moment of inertia = 0.9999728034 TIME= 0.003683352981 MASS = 0.5935240411 TIME= 0.003683352981 XMOM = -5.551115123e-16 TIME= 0.003683352981 YMOM = -2.498001805e-15 TIME= 0.003683352981 ZMOM = -1.32791597e-11 TIME= 0.003683352981 ANG MOM X = 9.020562075e-17 TIME= 0.003683352981 ANG MOM Y = 1.387778781e-17 TIME= 0.003683352981 ANG MOM Z = 4.289006454 TIME= 0.003683352981 HYB MOM R = 0.01458610189 TIME= 0.003683352981 HYB MOM L = 4.289006454 TIME= 0.003683352981 HYB MOM P = -1.32791597e-11 TIME= 0.003683352981 RHO*e = 0.7143408385 TIME= 0.003683352981 RHO*K = 13.47834086 TIME= 0.003683352981 RHO*E = 14.19268169 TIME= 0.003683352981 RHO*PHI = 0 TIME= 0.003683352981 TOTAL ENERGY= 14.19268169 [STEP 9] Coarse TimeStep time: 0.255631246 [STEP 9] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 9 TIME = 0.003683352981 DT = 0.0004935345004 ...estimated hydro-limited timestep at level 0: 0.001432337647 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001432337647 Castro::compute_new_dt : limiting dt at level 0 ... new dt computed: 0.001432337647 ... but limiting to: 0.0005182112254 = 1.05 * 0.0004935345004 [Level 0 step 10] ADVANCE with dt = 0.0005182112254  Beginning subcycle 1 starting at time 0.003683352981 with dt = 0.0005182112254 Estimated number of subcycles remaining: 1 Castro::do_old_sources() time = 0.006683249 ... Entering construct_ctu_hydro_source() ... Leaving construct_ctu_hydro_source() Castro::construct_ctu_hydro_source() time = 0.163365545 Castro::do_new_sources() time = 0.009617346 ...estimated hydro-limited timestep at level 0: 0.001305227986 Castro::estTimeStep (hydro-limited) at level 0: estdt = 0.001305227986  Subcycle completed Subcycling complete Zones advanced per microsecond at this level: 1.126673612 [Level 0 step 10] Advanced 262144 cells Moment of inertia = 2826.014035 Moment of inertia / initial moment of inertia = 0.9999837161 TIME= 0.004201564206 MASS = 0.5935240411 TIME= 0.004201564206 XMOM = -4.329869796e-15 TIME= 0.004201564206 YMOM = -5.939693182e-15 TIME= 0.004201564206 ZMOM = -1.523249403e-11 TIME= 0.004201564206 ANG MOM X = -1.630640067e-16 TIME= 0.004201564206 ANG MOM Y = 4.163336342e-17 TIME= 0.004201564206 ANG MOM Z = 4.289006454 TIME= 0.004201564206 HYB MOM R = 0.01663716124 TIME= 0.004201564206 HYB MOM L = 4.289006454 TIME= 0.004201564206 HYB MOM P = -1.523249403e-11 TIME= 0.004201564206 RHO*e = 0.7145679066 TIME= 0.004201564206 RHO*K = 13.47857813 TIME= 0.004201564206 RHO*E = 14.19314603 TIME= 0.004201564206 RHO*PHI = 0 TIME= 0.004201564206 TOTAL ENERGY= 14.19314603 [STEP 10] Coarse TimeStep time: 0.239377963 [STEP 10] FAB kilobyte spread across MPI nodes: [50507 ... 50507] STEP = 10 TIME = 0.004201564206 DT = 0.0005182112254 PLOTFILE: file = rotating_torus-restart_plt00010 Write plotfile time = 0.071878487 seconds Ending run at 19:11:13 UTC on 2022-04-28. Run time = 1.740541338 Run time without initialization = 1.702399689 Average number of zones advanced per microsecond: 0.770 Average number of zones advanced per microsecond per rank: 0.048 TinyProfiler total time across processes [min...avg...max]: 1.74 ... 1.74 ... 1.741 ------------------------------------------------------------------------------------------- Name NCalls Excl. Min Excl. Avg Excl. Max Max % ------------------------------------------------------------------------------------------- Castro::construct_ctu_hydro_source() 5 0.8338 1.0100 1.1385 65.41% Castro::do_advance_ctu() 5 0.01896 0.1445 0.3343 19.21% FabArray::ParallelCopy_finish() 116 0.02723 0.05831 0.09568 5.50% Castro::post_timestep() 5 0.04912 0.06056 0.07089 4.07% Amr::writePlotFile() 1 0.06259 0.06444 0.06598 3.79% FabArray::setVal() 80 0.04269 0.04834 0.05539 3.18% Castro::do_old_sources() 5 7.858e-05 0.01104 0.04221 2.43% Castro::reset_internal_energy(Fab) 240 0.0257 0.02996 0.03849 2.21% FabArray::ParallelCopy_nowait() 116 0.0177 0.02306 0.03248 1.87% StateData::FillBoundary(geom) 10 0.01157 0.01986 0.03158 1.81% FillPatchIterator::Initialize 116 0.01732 0.02276 0.02647 1.52% Castro::estTimeStep() 10 0.009598 0.01793 0.02625 1.51% Castro::computeTemp() 30 0.0162 0.01873 0.02391 1.37% Castro::do_new_sources() 5 7.026e-05 0.004918 0.0219 1.26% Castro::finalize_do_advance() 5 0.009553 0.01597 0.02177 1.25% VisMF::Read() 3 0.0157 0.0184 0.02115 1.21% Castro::normalize_species() 30 0.01182 0.01441 0.01906 1.10% Castro::expand_state() 5 0.01031 0.01387 0.01872 1.08% Castro::initialize_do_advance() 5 0.01189 0.01374 0.0165 0.95% Castro::construct_new_gravity_source() 5 0.01304 0.01432 0.01579 0.91% Castro::initialize_advance() 5 0.009498 0.01177 0.01512 0.87% Castro::hybrid_to_linear_momentum() 30 0.00887 0.01089 0.01509 0.87% MultiFab::contains_nan() 10 0.005875 0.007892 0.01205 0.69% MultiFab::Saxpy() 10 0.006695 0.009142 0.01082 0.62% VisMF::Write(FabArray) 1 0.006594 0.008138 0.009988 0.57% Amr::restart() 1 0.004305 0.007153 0.007779 0.45% Castro::derive() 86 0.005044 0.006015 0.00777 0.45% Gravity::get_new_grav_vector() 5 0.003665 0.004861 0.007039 0.40% Castro::post_restart() 1 5.234e-05 0.002013 0.006452 0.37% FabArray::setDomainBndry() 116 0.003122 0.004941 0.006438 0.37% Castro::construct_old_gravity_source() 5 0.003586 0.004777 0.006261 0.36% Gravity::add_pointmass_to_gravity() 10 0.004106 0.00476 0.005697 0.33% Castro::swap_state_time_levels() 5 0.004155 0.00453 0.004968 0.29% Castro::initMFs() 1 0.003152 0.004447 0.004863 0.28% Gravity::get_old_grav_vector() 5 0.002499 0.003375 0.004547 0.26% Castro::fill_hybrid_hydro_source() 15 0.002857 0.003682 0.00431 0.25% StateData::restartDoit() 4 0.002755 0.003711 0.004237 0.24% DistributionMapping::LeastUsedCPUs() 1 1.316e-05 0.0007516 0.003758 0.22% Castro::enforce_min_density() 30 0.002233 0.002635 0.003289 0.19% Castro::volWgtSum() 80 0.00249 0.002702 0.003161 0.18% main() 1 0.001629 0.002265 0.002571 0.15% Castro::finalize_advance() 5 0.0008405 0.001935 0.002337 0.13% Castro::subcycle_advance_ctu() 5 2.694e-05 6.847e-05 0.0005378 0.03% Castro::buildMetrics() 1 0.0001804 0.0003584 0.0004864 0.03% Amr::InitAmr() 1 0.0002674 0.0003825 0.000476 0.03% Castro::locSquaredSum() 6 0.0002209 0.000284 0.0003748 0.02% Amr::coarseTimeStep() 5 0.0001901 0.000232 0.0003006 0.02% Castro::reset_internal_energy(MultiFab) 30 0.0001413 0.000175 0.0002829 0.02% AmrLevel::restart() 1 0.000133 0.000218 0.0002553 0.01% Castro::clean_state() 30 0.0001195 0.0001555 0.0002218 0.01% StateDataPhysBCFunct::() 116 8.16e-05 0.0001178 0.0001613 0.01% FillPatchSingleLevel 116 7.423e-05 9.91e-05 0.000135 0.01% Castro::construct_new_source() 35 4.146e-05 7.706e-05 0.000111 0.01% FabArray::ParallelCopy() 116 6.812e-05 8.433e-05 0.0001109 0.01% Castro::construct_new_gravity() 5 2.461e-05 4.583e-05 8.658e-05 0.00% Castro::computeNewDt() 5 4.477e-06 1.031e-05 8.122e-05 0.00% FabArrayBase::CPC::define() 2 2.838e-05 5.216e-05 8.083e-05 0.00% Castro::construct_old_source() 35 3.202e-05 4.762e-05 8.006e-05 0.00% Castro::construct_old_gravity() 5 2.353e-05 4.688e-05 7.529e-05 0.00% Castro::apply_source_to_state() 10 1.66e-05 3.097e-05 5.487e-05 0.00% Amr::timeStep() 5 2.982e-05 3.602e-05 5.059e-05 0.00% Castro::check_for_nan() 10 1.542e-05 2.781e-05 4.609e-05 0.00% Amr::writeSmallPlotFile() 1 2.537e-05 3.333e-05 4.229e-05 0.00% Castro::advance() 5 2.273e-05 2.926e-05 3.787e-05 0.00% Castro::construct_new_hybrid_source() 5 7.929e-06 2.249e-05 3.57e-05 0.00% FabArrayBase::getCPC() 10 1.881e-05 2.505e-05 3.273e-05 0.00% Castro::enforce_speed_limit() 30 1.237e-05 1.614e-05 2.915e-05 0.00% Gravity::swapTimeLevels() 5 8.288e-06 1.416e-05 2.592e-05 0.00% Castro::construct_old_hybrid_source() 5 6.89e-06 1.19e-05 2.158e-05 0.00% Castro::create_source_corrector() 5 7.546e-06 1.311e-05 1.782e-05 0.00% Amr::initSubcycle() 1 8.112e-06 1.309e-05 1.688e-05 0.00% Castro::construct_new_rotation_source() 5 4.489e-06 7.406e-06 1.164e-05 0.00% DistributionMapping::SFCProcessorMapDoIt() 1 5.986e-06 7.659e-06 1.084e-05 0.00% Castro::retry_advance_ctu() 5 4.642e-06 6.191e-06 9.203e-06 0.00% Castro::construct_old_rotation_source() 5 3.247e-06 5.394e-06 8.563e-06 0.00% Castro::FluxRegCrseInit 5 4.454e-06 5.072e-06 6.689e-06 0.00% Gravity::set_mass_offset() 6 3.452e-06 4.973e-06 6.456e-06 0.00% Amr::init() 1 8.67e-07 1.564e-06 5.748e-06 0.00% Castro::gwstrain() 5 2.796e-06 4.488e-06 5.457e-06 0.00% Castro::FluxRegFineAdd() 5 1.981e-06 2.472e-06 3.645e-06 0.00% DistributionMapping::Distribute() 1 1.424e-06 2.411e-06 3.024e-06 0.00% AmrLevel::AmrLevel() 1 1.283e-06 1.833e-06 2.81e-06 0.00% ------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- Name NCalls Incl. Min Incl. Avg Incl. Max Max % ------------------------------------------------------------------------------------------- main() 1 1.74 1.74 1.741 100.00% Amr::coarseTimeStep() 5 1.627 1.627 1.627 93.50% Amr::timeStep() 5 1.622 1.622 1.622 93.18% Castro::advance() 5 1.53 1.537 1.543 88.67% Castro::subcycle_advance_ctu() 5 1.471 1.477 1.484 85.27% Castro::do_advance_ctu() 5 1.471 1.477 1.484 85.26% Castro::construct_ctu_hydro_source() 5 0.8338 1.01 1.139 65.41% FillPatchIterator::Initialize 116 0.1088 0.1293 0.1523 8.75% Castro::initialize_do_advance() 5 0.1149 0.1279 0.1445 8.30% FillPatchSingleLevel 116 0.07827 0.1016 0.1318 7.57% FabArray::ParallelCopy() 116 0.04644 0.08153 0.1184 6.80% Castro::expand_state() 5 0.07991 0.08935 0.1008 5.79% Castro::clean_state() 30 0.06535 0.07697 0.09971 5.73% FabArray::ParallelCopy_finish() 116 0.02723 0.05831 0.09568 5.50% Castro::post_timestep() 5 0.07849 0.08449 0.09151 5.26% Amr::writePlotFile() 1 0.07257 0.07258 0.07258 4.17% Castro::initialize_advance() 5 0.05077 0.05861 0.06808 3.91% Castro::do_old_sources() 5 0.02419 0.03762 0.0648 3.72% Castro::computeTemp() 30 0.04215 0.04886 0.06224 3.58% Castro::do_new_sources() 5 0.03969 0.04671 0.05903 3.39% FabArray::setVal() 80 0.04269 0.04834 0.05539 3.18% Castro::reset_internal_energy(MultiFab) 30 0.02585 0.03013 0.03877 2.23% Castro::reset_internal_energy(Fab) 240 0.0257 0.02996 0.03849 2.21% Amr::init() 1 0.03749 0.03759 0.03774 2.17% Amr::restart() 1 0.03748 0.03759 0.03774 2.17% FabArray::ParallelCopy_nowait() 116 0.01777 0.02314 0.03257 1.87% StateDataPhysBCFunct::() 116 0.01167 0.01997 0.03171 1.82% StateData::FillBoundary(geom) 10 0.01157 0.01986 0.03158 1.81% AmrLevel::restart() 1 0.0192 0.02318 0.02786 1.60% Castro::estTimeStep() 10 0.009598 0.01793 0.02625 1.51% StateData::restartDoit() 4 0.01864 0.0222 0.02395 1.38% Castro::finalize_do_advance() 5 0.009553 0.01597 0.02177 1.25% VisMF::Read() 3 0.0157 0.0184 0.02115 1.21% Castro::normalize_species() 30 0.01182 0.01441 0.01906 1.10% Castro::construct_new_source() 35 0.01494 0.01697 0.01852 1.06% Castro::volWgtSum() 80 0.01214 0.01447 0.01786 1.03% Castro::construct_new_gravity_source() 5 0.01304 0.01432 0.01579 0.91% Castro::derive() 86 0.01026 0.01233 0.01529 0.88% Castro::hybrid_to_linear_momentum() 30 0.00887 0.01089 0.01509 0.87% Castro::construct_new_gravity() 5 0.006687 0.008604 0.01253 0.72% Castro::check_for_nan() 10 0.005899 0.00792 0.01209 0.69% MultiFab::contains_nan() 10 0.005875 0.007892 0.01205 0.69% Gravity::get_new_grav_vector() 5 0.006262 0.008106 0.01168 0.67% Castro::apply_source_to_state() 10 0.00672 0.009173 0.01087 0.62% MultiFab::Saxpy() 10 0.006695 0.009142 0.01082 0.62% VisMF::Write(FabArray) 1 0.006594 0.008138 0.009988 0.57% Castro::construct_old_gravity() 5 0.005604 0.0068 0.008602 0.49% Gravity::get_old_grav_vector() 5 0.005287 0.00635 0.00807 0.46% Castro::construct_old_source() 35 0.004615 0.005982 0.007638 0.44% Castro::post_restart() 1 0.0003269 0.002316 0.006743 0.39% FabArray::setDomainBndry() 116 0.003122 0.004941 0.006438 0.37% Castro::construct_old_gravity_source() 5 0.003586 0.004777 0.006261 0.36% Gravity::add_pointmass_to_gravity() 10 0.004106 0.00476 0.005697 0.33% Castro::computeNewDt() 5 0.005209 0.005307 0.005359 0.31% Castro::swap_state_time_levels() 5 0.004155 0.00453 0.004968 0.29% Castro::initMFs() 1 0.003152 0.004447 0.004863 0.28% Castro::fill_hybrid_hydro_source() 15 0.002857 0.003682 0.00431 0.25% DistributionMapping::SFCProcessorMapDoIt() 1 2.291e-05 0.0007617 0.003771 0.22% DistributionMapping::LeastUsedCPUs() 1 1.316e-05 0.0007516 0.003758 0.22% Castro::enforce_min_density() 30 0.002233 0.002635 0.003289 0.19% Castro::construct_new_hybrid_source() 5 0.001833 0.002565 0.003021 0.17% Castro::finalize_advance() 5 0.0008484 0.001942 0.002344 0.13% Castro::construct_old_hybrid_source() 5 0.0009798 0.001151 0.001327 0.08% Castro::locSquaredSum() 6 0.0007449 0.0008463 0.0009641 0.06% Castro::buildMetrics() 1 0.0002368 0.000491 0.0006412 0.04% Amr::InitAmr() 1 0.0002755 0.0003956 0.0004926 0.03% FabArrayBase::getCPC() 10 5.051e-05 7.721e-05 0.0001018 0.01% FabArrayBase::CPC::define() 2 2.838e-05 5.216e-05 8.083e-05 0.00% Amr::writeSmallPlotFile() 1 2.537e-05 3.333e-05 4.229e-05 0.00% Castro::enforce_speed_limit() 30 1.237e-05 1.614e-05 2.915e-05 0.00% Gravity::swapTimeLevels() 5 8.288e-06 1.416e-05 2.592e-05 0.00% Castro::create_source_corrector() 5 7.546e-06 1.311e-05 1.782e-05 0.00% Amr::initSubcycle() 1 8.112e-06 1.309e-05 1.688e-05 0.00% Castro::construct_new_rotation_source() 5 4.489e-06 7.406e-06 1.164e-05 0.00% Castro::retry_advance_ctu() 5 4.642e-06 6.191e-06 9.203e-06 0.00% Castro::construct_old_rotation_source() 5 3.247e-06 5.394e-06 8.563e-06 0.00% Castro::FluxRegCrseInit 5 4.454e-06 5.072e-06 6.689e-06 0.00% Gravity::set_mass_offset() 6 3.452e-06 4.973e-06 6.456e-06 0.00% Castro::gwstrain() 5 2.796e-06 4.488e-06 5.457e-06 0.00% Castro::FluxRegFineAdd() 5 1.981e-06 2.472e-06 3.645e-06 0.00% DistributionMapping::Distribute() 1 1.424e-06 2.411e-06 3.024e-06 0.00% AmrLevel::AmrLevel() 1 1.283e-06 1.833e-06 2.81e-06 0.00% ------------------------------------------------------------------------------------------- Unused ParmParse Variables: [TOP]::castro.center(nvals = 3) :: [0.0, 0.0, 0.0] [TOP]::amr.ref_ratio(nvals = 4) :: [2, 2, 2, 2] [TOP]::amr.regrid_int(nvals = 1) :: [-1] [TOP]::problme.outer_radius(nvals = 1) :: [1.50] AMReX (22.04-37-g3cc4a66759c9) finalized