Initializing CUDA...
CUDA initialized with 1 device.
AMReX (23.06-17-g0375e78e7838) initialized

Starting run at 09:30:16 UTC on 2023-06-13.
Successfully read inputs file ... 

Castro git describe: 23.06-17-g448ec9434
AMReX git describe: 23.06-17-g0375e78e7
Microphysics git describe: 23.06-12-g8c5b63da

reading extern runtime parameters ...
13 Species: 
He4  C12  O16  Ne20  Mg24  Si28  S32  Ar36  Ca40  Ti44  Cr48  Fe52  Ni56  
Successfully read inputs file ... 

Generated initial model for primary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm.

Generated initial model for secondary WD of mass 0.9 solar masses, central density 1.574e+07 g cm**-3, and radius 6.747e+08 cm.

Generated binary orbit of distance 1.781e+09 cm = 1.190e-04 AU.
The primary orbits the center of mass at distance -8.903e+08 cm = -5.951e-05 AU.
The secondary orbits the center of mass at distance 8.903e+08 cm = 5.951e-05 AU.
The initial orbital period is 30.543687 s.
The initial orbital speed of the primary is -1.831e+08 cm/s.
The initial orbital speed of the secondary is 1.831e+08 cm/s.

Castro::numpts_1d at level  0 is 174
Setting the gravity type to PoissonGrav
Initializing the data at level 0
Done initializing the level 0 data 
...estimated hydro-limited timestep at level 0: 0.05426072637
...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.05426072637

grid_places() time: 0.00267933 new finest: 0
Now regridding at level lbase = 0
grid_places() time: 0.001617231 new finest: 0
STEP = 0 TIME = 0 : REGRID  with lbase = 0

Gravity::fill_multipole_BCs() time = 0.006979244

INITIAL GRIDS 
  Level 0   27 grids  884736 cells  100 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

PLOTFILE: file = wdmerger_3D_plt00000
Write plotfile time = 0.242900244  seconds

...estimated hydro-limited timestep at level 0: 0.05426072637
...hydro-limited CFL timestep constrained at (i,j,k) = (0,0,0)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.05426072637

[Level 0 step 1] ADVANCE with dt = 0.005426072637

  Beginning subcycle 1 starting at time 0 with dt = 0.005426072637
  Estimated number of subcycles remaining: 1

Castro::do_old_sources() time = 0.00835164

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.152571611

... new-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006253289

Gravity::solve_for_phi() time = 0.038729917

Castro::do_new_sources() time = 0.007032996

...estimated hydro-limited timestep at level 0: 0.05461684804
...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.05461684804

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 3.77297812

[Level 0 step 1] Advanced 884736 cells

  Updating the rotational period from 30.54368673 s to 30.46017574 s.


[STEP 1] Coarse TimeStep time: 0.244224091
[STEP 1] FAB kilobyte spread across MPI nodes: [2005477 ... 2005477]

STEP = 1 TIME = 0.005426072637 DT = 0.005426072637

...estimated hydro-limited timestep at level 0: 0.05461684804
...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.05461684804

Castro::compute_new_dt : limiting dt at level 0
 ... new dt computed: 0.05461684804
 ... but limiting to: 0.0059686799 = 1.1 * 0.005426072637
Now regridding at level lbase = 0
grid_places() time: 0.002003103 new finest: 1
Castro::numpts_1d at level  1 is 340
Gravity::fill_multipole_BCs() time = 0.006905348

STEP = 1 TIME = 0.005426072637 : REGRID  with lbase = 0
  Level 1   2 grids  65536 cells  0.9259259259 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

...estimated hydro-limited timestep at level 0: 0.05461684804
...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.05461684804

...estimated hydro-limited timestep at level 1: 2.526135647e-09
...hydro-limited CFL timestep constrained at (i,j,k) = (104,88,99)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 2.526135647e-09

[Level 0 step 2] ADVANCE with dt = 2.526135647e-09

  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135647e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006226788

Gravity::solve_for_phi() time = 0.042046997

Castro::do_old_sources() time = 0.006609103

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.075677303

... new-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006272361

Gravity::solve_for_phi() time = 0.02073237

Castro::do_new_sources() time = 0.006266596

...estimated hydro-limited timestep at level 0: 0.05461716053
...hydro-limited CFL timestep constrained at (i,j,k) = (0,29,94)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.05461716053

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 4.892893002

[Level 0 step 2] Advanced 884736 cells
[Level 1 step 2] ADVANCE with dt = 2.526135647e-09

  Beginning subcycle 1 starting at time 0.005426072637 with dt = 2.526135647e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.002122798

Castro::do_old_sources() time = 0.002048893

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.007613532

... new-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.011323661

Castro::do_new_sources() time = 0.002619437

...estimated hydro-limited timestep at level 1: 6.697001619e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 6.697001619e-07

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 2.04321753

[Level 1 step 2] Advanced 65536 cells
Castro::do_new_sources() time = 0.001603683

Castro::do_new_sources() time = 0.00580843

Castro::reflux() at level 0 : time = 0.027095367

  Updating the rotational period from 30.46017574 s to 30.52759479 s.


[STEP 2] Coarse TimeStep time: 0.319515982
[STEP 2] FAB kilobyte spread across MPI nodes: [2209725 ... 2209725]

STEP = 2 TIME = 0.005426075163 DT = 2.526135647e-09

...estimated hydro-limited timestep at level 0: 0.0008044208529
...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0008044208529

...estimated hydro-limited timestep at level 1: 6.697001619e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 6.697001619e-07

Castro::compute_new_dt : limiting dt at level 0
 ... new dt computed: 0.0008044208529
 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09
Castro::compute_new_dt : limiting dt at level 1
 ... new dt computed: 6.697001619e-07
 ... but limiting to: 2.778749212e-09 = 1.1 * 2.526135647e-09
Now regridding at level lbase = 0
grid_places() time: 0.001965311 new finest: 1
Castro::numpts_1d at level  1 is 340
Gravity::fill_multipole_BCs() time = 0.006955262

STEP = 2 TIME = 0.005426075163 : REGRID  with lbase = 0
  Level 1   2 grids  65536 cells  0.9259259259 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

...estimated hydro-limited timestep at level 0: 0.0008044208529
...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0008044208529

...estimated hydro-limited timestep at level 1: 6.697001619e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 6.697001619e-07

[Level 0 step 3] ADVANCE with dt = 2.778749212e-09

  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778749212e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.00622725

Gravity::solve_for_phi() time = 0.042731993

Castro::do_old_sources() time = 0.006905272

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.146399293

... new-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006246718

Gravity::solve_for_phi() time = 0.020881836

Castro::do_new_sources() time = 0.006283401

...estimated hydro-limited timestep at level 0: 0.0008044237461
...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0008044237461

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 3.501305235

[Level 0 step 3] Advanced 884736 cells
[Level 1 step 3] ADVANCE with dt = 2.778749212e-09

  Beginning subcycle 1 starting at time 0.005426075163 with dt = 2.778749212e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.001928213

Castro::do_old_sources() time = 0.002171304

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.007003335

... new-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.006985781

Castro::do_new_sources() time = 0.002239279

...estimated hydro-limited timestep at level 1: 6.965184638e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 6.965184638e-07

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 2.499227476

[Level 1 step 3] Advanced 65536 cells
Castro::do_new_sources() time = 0.001401818

Castro::do_new_sources() time = 0.009450069

Castro::reflux() at level 0 : time = 0.034609941

  Updating the rotational period from 30.52759479 s to 30.52761298 s.


[STEP 3] Coarse TimeStep time: 0.371785444
[STEP 3] FAB kilobyte spread across MPI nodes: [2209725 ... 2209725]

STEP = 3 TIME = 0.005426077942 DT = 2.778749212e-09

...estimated hydro-limited timestep at level 0: 0.000804308035
...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.000804308035

...estimated hydro-limited timestep at level 1: 6.965184638e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 6.965184638e-07

Castro::compute_new_dt : limiting dt at level 0
 ... new dt computed: 0.000804308035
 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09
Castro::compute_new_dt : limiting dt at level 1
 ... new dt computed: 6.965184638e-07
 ... but limiting to: 3.056624133e-09 = 1.1 * 2.778749212e-09
Now regridding at level lbase = 0
grid_places() time: 0.002030044 new finest: 1
Castro::numpts_1d at level  1 is 340
Gravity::fill_multipole_BCs() time = 0.006848892

STEP = 3 TIME = 0.005426077942 : REGRID  with lbase = 0
  Level 1   2 grids  65536 cells  0.9259259259 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

...estimated hydro-limited timestep at level 0: 0.000804308035
...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.000804308035

...estimated hydro-limited timestep at level 1: 6.965184638e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 6.965184638e-07

[Level 0 step 4] ADVANCE with dt = 3.056624133e-09

  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056624133e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006233242

Gravity::solve_for_phi() time = 0.042530189

Castro::do_old_sources() time = 0.006271761

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.13973375

... new-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006251065

Gravity::solve_for_phi() time = 0.020787682

Castro::do_new_sources() time = 0.006221701

...estimated hydro-limited timestep at level 0: 0.0008043112215
...hydro-limited CFL timestep constrained at (i,j,k) = (37,49,42)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0008043112215

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 3.440120712

[Level 0 step 4] Advanced 884736 cells
[Level 1 step 4] ADVANCE with dt = 3.056624133e-09

  Beginning subcycle 1 starting at time 0.005426077942 with dt = 3.056624133e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.001934884

Castro::do_old_sources() time = 0.002469598

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.006764308

... new-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.007014723

Castro::do_new_sources() time = 0.002271931

...estimated hydro-limited timestep at level 1: 7.255166705e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 7.255166705e-07

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 2.448509876

[Level 1 step 4] Advanced 65536 cells
Castro::do_new_sources() time = 0.00212703

Castro::do_new_sources() time = 0.006014634

Castro::reflux() at level 0 : time = 0.028223523

  Updating the rotational period from 30.52761298 s to 30.52761297 s.


[STEP 4] Coarse TimeStep time: 0.366781227
[STEP 4] FAB kilobyte spread across MPI nodes: [2209725 ... 2209725]

STEP = 4 TIME = 0.005426080998 DT = 3.056624133e-09

...estimated hydro-limited timestep at level 0: 0.0002497928903
...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0002497928903

...estimated hydro-limited timestep at level 1: 7.255166705e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 7.255166705e-07

Castro::compute_new_dt : limiting dt at level 0
 ... new dt computed: 0.0002497928903
 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09
Castro::compute_new_dt : limiting dt at level 1
 ... new dt computed: 7.255166705e-07
 ... but limiting to: 3.362286546e-09 = 1.1 * 3.056624133e-09
Now regridding at level lbase = 0
grid_places() time: 0.001986966 new finest: 1
Castro::numpts_1d at level  1 is 340
Gravity::fill_multipole_BCs() time = 0.006943711

STEP = 4 TIME = 0.005426080998 : REGRID  with lbase = 0
  Level 1   2 grids  65536 cells  0.9259259259 % of domain
            smallest grid: 32 x 32 x 32  biggest grid: 32 x 32 x 32

...estimated hydro-limited timestep at level 0: 0.0002497928903
...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0002497928903

...estimated hydro-limited timestep at level 1: 7.255166705e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 7.255166705e-07

[Level 0 step 5] ADVANCE with dt = 3.362286546e-09

  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286546e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006244989

Gravity::solve_for_phi() time = 0.042753405

Castro::do_old_sources() time = 0.006489144

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.075935585

... new-time level Poisson gravity solve at level 0

Gravity::fill_multipole_BCs() time = 0.006269773

Gravity::solve_for_phi() time = 0.020969995

Castro::do_new_sources() time = 0.006282482

...estimated hydro-limited timestep at level 0: 0.0006805523364
...hydro-limited CFL timestep constrained at (i,j,k) = (57,41,46)
Castro::estTimeStep (hydro-limited) at level 0:  estdt = 0.0006805523364

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 4.871231373

[Level 0 step 5] Advanced 884736 cells
[Level 1 step 5] ADVANCE with dt = 3.362286546e-09

  Beginning subcycle 1 starting at time 0.005426080998 with dt = 3.362286546e-09
  Estimated number of subcycles remaining: 1

... old-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.001971712

Castro::do_old_sources() time = 0.00150535

... Entering construct_ctu_hydro_source()

... Leaving construct_ctu_hydro_source()

Castro::construct_ctu_hydro_source() time = 0.010891084

... new-time level Poisson gravity solve at level 1

Gravity::solve_for_phi() time = 0.00702602

Castro::do_new_sources() time = 0.001928556

...estimated hydro-limited timestep at level 1: 7.553385904e-07
...hydro-limited CFL timestep constrained at (i,j,k) = (121,90,90)
Castro::estTimeStep (hydro-limited) at level 1:  estdt = 7.553385904e-07

  Subcycle completed

  Subcycling complete

  Zones advanced per microsecond at this level: 2.25735236

[Level 1 step 5] Advanced 65536 cells
Castro::do_new_sources() time = 0.002248554

Castro::do_new_sources() time = 0.007290634

Castro::reflux() at level 0 : time = 0.030392161

  Updating the rotational period from 30.52761297 s to 30.52761296 s.


[STEP 5] Coarse TimeStep time: 0.27960482
[STEP 5] FAB kilobyte spread across MPI nodes: [2209725 ... 2209725]

STEP = 5 TIME = 0.005426084361 DT = 3.362286546e-09

PLOTFILE: file = wdmerger_3D_plt00005
Write plotfile time = 0.218391274  seconds


Ending run at 09:30:20 UTC on 2023-06-13.
Run time = 3.367905477
Run time without initialization = 1.801199076

  Average number of zones advanced per microsecond: 2.602
  Average number of zones advanced per microsecond per rank: 2.602

CPU(0): Heap Space (bytes) used by Coalescing FAB Arena: 9481961472


TinyProfiler total time across processes [min...avg...max]: 3.368 ... 3.368 ... 3.368

--------------------------------------------------------------------------------------------
Name                                         NCalls  Excl. Min  Excl. Avg  Excl. Max   Max %
--------------------------------------------------------------------------------------------
Amr::InitAmr()                                    1     1.0866     1.0866     1.0866  32.26%
Castro::construct_ctu_hydro_source()              9     0.6048     0.6048     0.6048  17.96%
VisMF::Write(FabArray)                            3     0.4368     0.4368     0.4368  12.97%
Castro::initMFs()                                 5     0.1608     0.1608     0.1608   4.77%
Gravity::fill_multipole_BCs()                    14    0.08969    0.08969    0.08969   2.66%
MLCellLinOp::applyBC()                         3868    0.08747    0.08747    0.08747   2.60%
Castro::computeTemp()                           104    0.08262    0.08262    0.08262   2.45%
FillBoundary_nowait()                          3516    0.07765    0.07765    0.07765   2.31%
Castro::normalize_species()                     103    0.06833    0.06833    0.06833   2.03%
FabArray::setVal()                             1493    0.04288    0.04288    0.04288   1.27%
amrex::Copy()                                  1129    0.03404    0.03404    0.03404   1.01%
StateData::FillBoundary(geom)                   650    0.03317    0.03317    0.03317   0.98%
FabArray::ParallelCopy_nowait()                1194    0.03287    0.03287    0.03287   0.98%
Castro::reset_internal_energy(MultiFab)         104    0.03246    0.03246    0.03246   0.96%
MLCellLinOp::defineAuxData()                     22    0.03092    0.03092    0.03092   0.92%
Castro::volInBoundary()                          42    0.02771    0.02771    0.02771   0.82%
MLPoisson::Fsmooth()                           2800    0.02463    0.02463    0.02463   0.73%
FabArray::Saxpy()                               755    0.02349    0.02349    0.02349   0.70%
Amr::writePlotFile()                              2    0.02236    0.02236    0.02236   0.66%
FluxRegister::Reflux()                           24    0.02164    0.02164    0.02164   0.64%
amrex::Add()                                    227     0.0216     0.0216     0.0216   0.64%
Castro::enforce_min_density()                   103    0.02098    0.02098    0.02098   0.62%
Castro::estTimeStep()                            26    0.01935    0.01935    0.01935   0.57%
Castro::construct_new_source()                  119    0.01888    0.01888    0.01888   0.56%
amrex::Dot()                                    972    0.01821    0.01821    0.01821   0.54%
BndryData::define()                              30    0.01634    0.01634    0.01634   0.49%
FabArray::norminf()                             670    0.01438    0.01438    0.01438   0.43%
StateDataPhysBCFunct::()                         95    0.01317    0.01317    0.01317   0.39%
MLPoisson::Fapply()                            1025    0.01231    0.01231    0.01231   0.37%
FabArray::Xpay()                                731    0.01155    0.01155    0.01155   0.34%
Castro::problem_post_timestep()                  10    0.01034    0.01034    0.01034   0.31%
amrex::average_down                             392    0.00914    0.00914    0.00914   0.27%
MLMG::addInterpCorrection()                     340   0.008829   0.008829   0.008829   0.26%
Castro::construct_new_gravity_source()           17   0.008617   0.008617   0.008617   0.26%
Castro::FluxRegCrseInit                           9   0.007881   0.007881   0.007881   0.23%
Castro::construct_old_gravity_source()           18   0.007557   0.007557   0.007557   0.22%
Castro::reset_internal_energy(Fab)             1708   0.007406   0.007406   0.007406   0.22%
Castro::reflux()                                  4   0.007242   0.007242   0.007242   0.22%
AmrMesh::MakeNewGrids()                           6   0.005376   0.005376   0.005376   0.16%
Gravity::actual_solve_with_mlmg()                22   0.005254   0.005254   0.005254   0.16%
MLCellLinOp::reflux()                            17    0.00523    0.00523    0.00523   0.16%
Castro::enforce_speed_limit()                   103    0.00506    0.00506    0.00506   0.15%
Castro::initData()                                1   0.004642   0.004642   0.004642   0.14%
FabArray::setVal(val, thecmd, scomp, ncomp)     956   0.004561   0.004561   0.004561   0.14%
MLCellLinOp::setLevelBC()                        26   0.004371   0.004371   0.004371   0.13%
FabArrayBase::CPC::define()                     910   0.004315   0.004315   0.004315   0.13%
Castro::wd_update()                               6   0.004247   0.004247   0.004247   0.13%
MLCellLinOp::compGrad()                          26   0.003911   0.003911   0.003911   0.12%
FabArray::LinComb()                             486   0.003895   0.003895   0.003895   0.12%
Castro::construct_old_source()                   63     0.0036     0.0036     0.0036   0.11%
FabArray::setDomainBndry()                       92   0.003194   0.003194   0.003194   0.09%
MultiFab::contains_nan()                         18   0.003168   0.003168   0.003168   0.09%
FillPatchInterp(MF)                              30   0.003095   0.003095   0.003095   0.09%
MLCellLinOp::prepareForSolve()                   22   0.003049   0.003049   0.003049   0.09%
MLMG::prepareForSolve()                          22   0.002835   0.002835   0.002835   0.08%
FabArray::mult()                                 63   0.002634   0.002634   0.002634   0.08%
Castro::fill_hybrid_hydro_source()               43   0.002359   0.002359   0.002359   0.07%
check_for_negative_density()                      9   0.002148   0.002148   0.002148   0.06%
MLCGSolver::bicgstab                             70    0.00202    0.00202    0.00202   0.06%
Castro::apply_tagging_restrictions()              6   0.001666   0.001666   0.001666   0.05%
MLCellLinOp::defineBC()                          22   0.001562   0.001562   0.001562   0.05%
FabArrayBase::getCPC()                         2138   0.001549   0.001549   0.001549   0.05%
TagBoxArray::mapPRD                               6   0.001196   0.001196   0.001196   0.04%
Gravity::get_new_grav_vector()                   10   0.001102   0.001102   0.001102   0.03%
MultiFab::Subtract()                             20   0.001007   0.001007   0.001007   0.03%
MLCellLinOp::smooth()                          1400  0.0009853  0.0009853  0.0009853   0.03%
FabArrayBase::FB::FB()                           84  0.0009419  0.0009419  0.0009419   0.03%
OwnerMask()                                       6  0.0008896  0.0008896  0.0008896   0.03%
Castro::build_fine_mask()                        40  0.0008828  0.0008828  0.0008828   0.03%
Gravity::get_old_grav_vector()                    9  0.0008631  0.0008631  0.0008631   0.03%
main()                                            1  0.0008247  0.0008247  0.0008247   0.02%
FabArray::FillBoundary()                       3516  0.0008244  0.0008244  0.0008244   0.02%
Castro::post_timestep()                           9  0.0007697  0.0007697  0.0007697   0.02%
MLCellLinOp::updateSolBC()                       14  0.0007582  0.0007582  0.0007582   0.02%
FillPatchTwoLevels                               31  0.0006858  0.0006858  0.0006858   0.02%
MLPoisson::FFlux()                               85  0.0006776  0.0006776  0.0006776   0.02%
TagBoxArray::collate()                            6  0.0006557  0.0006557  0.0006557   0.02%
FabArray::norminf(mask)                          11  0.0006097  0.0006097  0.0006097   0.02%
FillPatchIterator::Initialize                    62  0.0005964  0.0005964  0.0005964   0.02%
FabArrayBase::getFB()                          3516  0.0005887  0.0005887  0.0005887   0.02%
MLCellLinOp::updateCorBC()                       10  0.0005862  0.0005862  0.0005862   0.02%
Castro::FluxRegFineAdd()                          9  0.0005642  0.0005642  0.0005642   0.02%
Gravity::solve_for_phi()                         17  0.0005415  0.0005415  0.0005415   0.02%
MultiFab::max()                                  18  0.0005325  0.0005325  0.0005325   0.02%
Castro::apply_problem_tags()                      6  0.0004716  0.0004716  0.0004716   0.01%
Castro::construct_old_gravity()                   9  0.0004626  0.0004626  0.0004626   0.01%
Amr::regrid()                                     5  0.0004184  0.0004184  0.0004184   0.01%
MLCellLinOp::apply()                           1025  0.0004047  0.0004047  0.0004047   0.01%
Castro::Castro()                                  5   0.000398   0.000398   0.000398   0.01%
FabArray::ParallelCopy()                       1194  0.0003882  0.0003882  0.0003882   0.01%
MLLinOp::defineGrids()                           22   0.000381   0.000381   0.000381   0.01%
Castro::subcycle_advance_ctu()                    9   0.000378   0.000378   0.000378   0.01%
Castro::construct_new_rotation_source()          17   0.000373   0.000373   0.000373   0.01%
average_down_faces                               24  0.0003659  0.0003659  0.0003659   0.01%
Castro::clean_state()                           103  0.0003475  0.0003475  0.0003475   0.01%
MLMG::mgVcycle()                                 90  0.0003222  0.0003222  0.0003222   0.01%
Gravity::multilevel_solve_for_new_phi()           5  0.0002799  0.0002799  0.0002799   0.01%
Gravity::update_max_rhs()                        10  0.0002581  0.0002581  0.0002581   0.01%
Castro::initialize_do_advance()                   9  0.0002571  0.0002571  0.0002571   0.01%
MLCGSolver::ParallelAllReduce                  1304   0.000255   0.000255   0.000255   0.01%
Amr::coarseTimeStep()                             5   0.000254   0.000254   0.000254   0.01%
Castro::do_new_sources()                         17  0.0002338  0.0002338  0.0002338   0.01%
MultiFab::Multiply()                              4  0.0002321  0.0002321  0.0002321   0.01%
Castro::do_old_sources()                          9  0.0002258  0.0002258  0.0002258   0.01%
Gravity::create_comp_minus_level_grad_phi()       4  0.0002245  0.0002245  0.0002245   0.01%
Castro::initialize_advance()                      9   0.000224   0.000224   0.000224   0.01%
Castro::construct_old_rotation_source()           9  0.0002114  0.0002114  0.0002114   0.01%
MLCellLinOp::correctionResidual()               430  0.0001982  0.0001982  0.0001982   0.01%
MLMG::interpCorrection_1                         10  0.0001905  0.0001905  0.0001905   0.01%
Castro::finalize_advance()                        9  0.0001845  0.0001845  0.0001845   0.01%
AMRErrorTag::operator()                           6  0.0001834  0.0001834  0.0001834   0.01%
Castro::construct_new_gravity()                   9  0.0001757  0.0001757  0.0001757   0.01%
Gravity::solve_phi_with_mlmg()                   22  0.0001635  0.0001635  0.0001635   0.00%
Castro::derive()                                  6  0.0001483  0.0001483  0.0001483   0.00%
Castro::do_advance_ctu()                          9  0.0001456  0.0001456  0.0001456   0.00%
Gravity::GetCrsePhi()                             8  0.0001425  0.0001425  0.0001425   0.00%
Castro::expand_state()                           13  0.0001379  0.0001379  0.0001379   0.00%
Castro::swap_state_time_levels()                  9  0.0001342  0.0001342  0.0001342   0.00%
Castro::post_regrid()                            10  0.0001335  0.0001335  0.0001335   0.00%
Castro::finalize_do_advance()                     9  0.0001325  0.0001325  0.0001325   0.00%
StateData::define()                              25  0.0001279  0.0001279  0.0001279   0.00%
Castro::enforce_consistent_e()                    1  0.0001168  0.0001168  0.0001168   0.00%
Castro::linear_to_hybrid_momentum()               1  0.0001162  0.0001162  0.0001162   0.00%
Castro::advance()                                 9  0.0001123  0.0001123  0.0001123   0.00%
MLMG:computeResOfCorrection()                   340  0.0001043  0.0001043  0.0001043   0.00%
Amr::grid_places()                                6  9.941e-05  9.941e-05  9.941e-05   0.00%
Gravity::actual_multilevel_solve()                5  9.856e-05  9.856e-05  9.856e-05   0.00%
Castro::create_source_corrector()                 9  9.333e-05  9.333e-05  9.333e-05   0.00%
MLMG::solve()                                    22  8.881e-05  8.881e-05  8.881e-05   0.00%
Castro::buildMetrics()                            5  8.726e-05  8.726e-05  8.726e-05   0.00%
MLMG::actualBottomSolve()                        70  8.633e-05  8.633e-05  8.633e-05   0.00%
MLMG::mgVcycle_down::0                           70  7.872e-05  7.872e-05  7.872e-05   0.00%
MLMG::mgVcycle_down::1                           70   7.22e-05   7.22e-05   7.22e-05   0.00%
MLMG::mgVcycle_down::3                           70  7.085e-05  7.085e-05  7.085e-05   0.00%
MLMG::mgVcycle_down::2                           70  6.988e-05  6.988e-05  6.988e-05   0.00%
Amr::timeStep()                                   9  6.984e-05  6.984e-05  6.984e-05   0.00%
MLMG::oneIter()                                  70  6.835e-05  6.835e-05  6.835e-05   0.00%
MLCellLinOp::solutionResidual()                 109  6.517e-05  6.517e-05  6.517e-05   0.00%
Gravity::average_fine_ec_onto_crse_ec()           4  6.387e-05  6.387e-05  6.387e-05   0.00%
Castro::init(old)                                 3  5.574e-05  5.574e-05  5.574e-05   0.00%
FillPatchSingleLevel                             92  5.571e-05  5.571e-05  5.571e-05   0.00%
MLMG::mgVcycle_down::4                           60  5.542e-05  5.542e-05  5.542e-05   0.00%
makeSFC                                         102  5.218e-05  5.218e-05  5.218e-05   0.00%
MLMG::mgVcycle_bottom                            90   5.01e-05   5.01e-05   5.01e-05   0.00%
Castro::errorEst()                                6  4.955e-05  4.955e-05  4.955e-05   0.00%
MLMG::mgVcycle_up::4                             60  4.784e-05  4.784e-05  4.784e-05   0.00%
AmrLevel::AmrLevel(dm)                            5    4.7e-05    4.7e-05    4.7e-05   0.00%
MLMG::mgVcycle_up::0                             70  4.595e-05  4.595e-05  4.595e-05   0.00%
MLMG::mgVcycle_up::3                             70  4.571e-05  4.571e-05  4.571e-05   0.00%
MLMG::mgVcycle_up::2                             70  4.412e-05  4.412e-05  4.412e-05   0.00%
AmrLevel::FillCoarsePatch()                       5  4.357e-05  4.357e-05  4.357e-05   0.00%
MLMG::mgVcycle_up::1                             70  4.285e-05  4.285e-05  4.285e-05   0.00%
DistributionMapping::Distribute()               104  4.204e-05  4.204e-05  4.204e-05   0.00%
MLMG::ResNormInf()                               99  3.976e-05  3.976e-05  3.976e-05   0.00%
MLPoisson::define()                              22  3.622e-05  3.622e-05  3.622e-05   0.00%
Amr::writeSmallPlotFile()                         1  3.511e-05  3.511e-05  3.511e-05   0.00%
Amr::defBaseLevel()                               1  3.042e-05  3.042e-05  3.042e-05   0.00%
MLMG::computeResidual()                          70  2.917e-05  2.917e-05  2.917e-05   0.00%
MLLinOp::makeAgglomeratedDMap                    22  2.883e-05  2.883e-05  2.883e-05   0.00%
Castro::computeNewDt()                            8  2.769e-05  2.769e-05  2.769e-05   0.00%
MLPoisson::prepareForSolve()                     22  2.386e-05  2.386e-05  2.386e-05   0.00%
FPinfo::FPinfo()                                  3  2.002e-05  2.002e-05  2.002e-05   0.00%
FabArrayBase::TheFPinfo()                        25  1.878e-05  1.878e-05  1.878e-05   0.00%
MLLinOp::define()                                22  1.818e-05  1.818e-05  1.818e-05   0.00%
FabArray::RecvLayoutMask()                        4  1.749e-05  1.749e-05  1.749e-05   0.00%
MLMG::computeMLResidual()                        25  1.668e-05  1.668e-05  1.668e-05   0.00%
Castro::apply_source_to_state()                  34  1.601e-05  1.601e-05  1.601e-05   0.00%
Amr::FinalizeInit()                               1  1.586e-05  1.586e-05  1.586e-05   0.00%
MLMG::computeResWithCrseSolFineCor()             10   1.49e-05   1.49e-05   1.49e-05   0.00%
ClusterList::intersect()                          4  1.333e-05  1.333e-05  1.333e-05   0.00%
Castro::construct_new_hybrid_source()            17  1.307e-05  1.307e-05  1.307e-05   0.00%
MLMG::getGradSolution()                          22  1.204e-05  1.204e-05  1.204e-05   0.00%
Castro::react_state()                            18  1.061e-05  1.061e-05  1.061e-05   0.00%
Castro::check_for_nan()                          18  1.028e-05  1.028e-05  1.028e-05   0.00%
Amr::initSubcycle()                               1   1.01e-05   1.01e-05   1.01e-05   0.00%
MLMG::MLResNormInf()                             25  9.716e-06  9.716e-06  9.716e-06   0.00%
Gravity::swapTimeLevels()                         9  9.247e-06  9.247e-06  9.247e-06   0.00%
MLMG::MLRhsNormInf()                             22  8.206e-06  8.206e-06  8.206e-06   0.00%
MLMG::computeResWithCrseCorFineCor()             10  7.381e-06  7.381e-06  7.381e-06   0.00%
Castro::avgDown(state_indx)                      20  7.323e-06  7.323e-06  7.323e-06   0.00%
MLMG::miniCycle()                                20  6.963e-06  6.963e-06  6.963e-06   0.00%
amrex::average_down_w_geom                       20  6.636e-06  6.636e-06  6.636e-06   0.00%
DistributionMapping::SFCProcessorMapDoIt()        2  6.232e-06  6.232e-06  6.232e-06   0.00%
Castro::construct_old_hybrid_source()             9  6.117e-06  6.117e-06  6.117e-06   0.00%
AmrMesh-cluster                                   4  6.037e-06  6.037e-06  6.037e-06   0.00%
BoxList::parallelComplementIn()                  12  5.872e-06  5.872e-06  5.872e-06   0.00%
Castro::retry_advance_ctu()                       9  5.211e-06  5.211e-06  5.211e-06   0.00%
Castro::post_init()                               1  4.721e-06  4.721e-06  4.721e-06   0.00%
Amr::init()                                       1  4.402e-06  4.402e-06  4.402e-06   0.00%
Amr::InitializeInit()                             1  4.122e-06  4.122e-06  4.122e-06   0.00%
Gravity::set_mass_offset()                        6  3.967e-06  3.967e-06  3.967e-06   0.00%
MLLinOp::makeSubCommunicator()                   22  3.811e-06  3.811e-06  3.811e-06   0.00%
Castro::avgDown()                                 4  3.804e-06  3.804e-06  3.804e-06   0.00%
BoxList::complementIn                            12  3.785e-06  3.785e-06  3.785e-06   0.00%
Castro::computeInitialDt()                        2  2.601e-06  2.601e-06  2.601e-06   0.00%
Amr::bldFineLevels()                              1   2.25e-06   2.25e-06   2.25e-06   0.00%
Castro::init()                                    1  2.029e-06  2.029e-06  2.029e-06   0.00%
ClusterList::chop()                               4    1.4e-06    1.4e-06    1.4e-06   0.00%
Amr::initialInit()                                1  1.073e-06  1.073e-06  1.073e-06   0.00%
--------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------
Name                                         NCalls  Incl. Min  Incl. Avg  Incl. Max   Max %
--------------------------------------------------------------------------------------------
main()                                            1      3.368      3.368      3.368 100.00%
Amr::coarseTimeStep()                             5      1.582      1.582      1.582  46.98%
Amr::timeStep()                                   9      1.575      1.575      1.575  46.76%
Castro::advance()                                 9      1.221      1.221      1.221  36.25%
Castro::subcycle_advance_ctu()                    9      1.183      1.183      1.183  35.13%
Castro::do_advance_ctu()                          9      1.183      1.183      1.183  35.12%
Amr::InitAmr()                                    1      1.087      1.087      1.087  32.26%
Castro::construct_ctu_hydro_source()              9     0.6226     0.6226     0.6226  18.49%
Gravity::solve_phi_with_mlmg()                   22     0.4991     0.4991     0.4991  14.82%
Amr::init()                                       1     0.4798     0.4798     0.4798  14.25%
Amr::writePlotFile()                              2     0.4615     0.4615     0.4615  13.70%
VisMF::Write(FabArray)                            3     0.4368     0.4368     0.4368  12.97%
Gravity::actual_solve_with_mlmg()                22      0.406      0.406      0.406  12.05%
MLMG::solve()                                    22     0.3336     0.3336     0.3336   9.91%
Gravity::solve_for_phi()                         17     0.3327     0.3327     0.3327   9.88%
MLMG::oneIter()                                  70     0.2956     0.2956     0.2956   8.78%
MLMG::mgVcycle()                                 90     0.2768     0.2768     0.2768   8.22%
Amr::initialInit()                                1     0.2368     0.2368     0.2368   7.03%
Castro::clean_state()                           103     0.2159     0.2159     0.2159   6.41%
Castro::construct_old_gravity()                   9     0.1995     0.1995     0.1995   5.92%
Castro::post_timestep()                           9     0.1857     0.1857     0.1857   5.51%
Castro::construct_new_gravity()                   9     0.1742     0.1742     0.1742   5.17%
Amr::InitializeInit()                             1     0.1706     0.1706     0.1706   5.07%
Amr::defBaseLevel()                               1     0.1706     0.1706     0.1706   5.07%
Gravity::multilevel_solve_for_new_phi()           5     0.1692     0.1692     0.1692   5.02%
Gravity::actual_multilevel_solve()                5      0.169      0.169      0.169   5.02%
Amr::regrid()                                     5     0.1671     0.1671     0.1671   4.96%
MLCellLinOp::applyBC()                         3868     0.1667     0.1667     0.1667   4.95%
Castro::Castro()                                  5      0.163      0.163      0.163   4.84%
Castro::initMFs()                                 5     0.1608     0.1608     0.1608   4.77%
Castro::post_regrid()                            10     0.1538     0.1538     0.1538   4.57%
MLCellLinOp::smooth()                          1400     0.1526     0.1526     0.1526   4.53%
Castro::computeTemp()                           104     0.1225     0.1225     0.1225   3.64%
Castro::reflux()                                  4     0.1203     0.1203     0.1203   3.57%
Gravity::fill_multipole_BCs()                    14    0.09102    0.09102    0.09102   2.70%
FabArray::FillBoundary()                       3516       0.08       0.08       0.08   2.38%
FillBoundary_nowait()                          3516    0.07918    0.07918    0.07918   2.35%
Castro::do_new_sources()                         17     0.0773     0.0773     0.0773   2.30%
FillPatchIterator::Initialize                    62    0.07325    0.07325    0.07325   2.17%
MLMG::mgVcycle_bottom                            90    0.06916    0.06916    0.06916   2.05%
Castro::normalize_species()                     103    0.06833    0.06833    0.06833   2.03%
Amr::FinalizeInit()                               1    0.06614    0.06614    0.06614   1.96%
FillPatchSingleLevel                             92    0.06531    0.06531    0.06531   1.94%
MLMG::actualBottomSolve()                        70    0.06517    0.06517    0.06517   1.93%
MLCGSolver::bicgstab                             70    0.06457    0.06457    0.06457   1.92%
MLMG::mgVcycle_down::0                           70      0.057      0.057      0.057   1.69%
MLPoisson::define()                              22    0.05625    0.05625    0.05625   1.67%
Castro::post_init()                               1    0.05612    0.05612    0.05612   1.67%
Castro::initialize_do_advance()                   9    0.05246    0.05246    0.05246   1.56%
MLCellLinOp::apply()                           1025    0.04989    0.04989    0.04989   1.48%
Castro::problem_post_timestep()                  10    0.04816    0.04816    0.04816   1.43%
StateDataPhysBCFunct::()                         95    0.04634    0.04634    0.04634   1.38%
MLMG::mgVcycle_up::0                             70    0.04324    0.04324    0.04324   1.28%
Castro::do_old_sources()                          9    0.04293    0.04293    0.04293   1.27%
FabArray::setVal()                             1493    0.04288    0.04288    0.04288   1.27%
Castro::construct_new_source()                  119    0.04061    0.04061    0.04061   1.21%
Castro::reset_internal_energy(MultiFab)         104    0.03986    0.03986    0.03986   1.18%
Castro::expand_state()                           13    0.03956    0.03956    0.03956   1.17%
FabArray::ParallelCopy()                       1194    0.03754    0.03754    0.03754   1.11%
FabArray::ParallelCopy_nowait()                1194    0.03716    0.03716    0.03716   1.10%
MLCellLinOp::defineAuxData()                     22    0.03517    0.03517    0.03517   1.04%
amrex::Copy()                                  1129    0.03404    0.03404    0.03404   1.01%
StateData::FillBoundary(geom)                   650    0.03317    0.03317    0.03317   0.98%
Castro::wd_update()                               6     0.0324     0.0324     0.0324   0.96%
MLCellLinOp::correctionResidual()               430    0.03221    0.03221    0.03221   0.96%
FluxRegister::Reflux()                           24    0.02921    0.02921    0.02921   0.87%
Castro::volInBoundary()                          42    0.02771    0.02771    0.02771   0.82%
Castro::initialize_advance()                      9     0.0274     0.0274     0.0274   0.81%
MLMG:computeResOfCorrection()                   340    0.02737    0.02737    0.02737   0.81%
MLPoisson::Fsmooth()                           2800    0.02463    0.02463    0.02463   0.73%
FabArray::Saxpy()                               755    0.02349    0.02349    0.02349   0.70%
MLMG::mgVcycle_down::1                           70     0.0234     0.0234     0.0234   0.69%
amrex::Add()                                    227     0.0216     0.0216     0.0216   0.64%
Gravity::get_new_grav_vector()                   10    0.02152    0.02152    0.02152   0.64%
Castro::enforce_min_density()                   103    0.02098    0.02098    0.02098   0.62%
MLCellLinOp::defineBC()                          22    0.02053    0.02053    0.02053   0.61%
Castro::estTimeStep()                            26    0.01935    0.01935    0.01935   0.57%
BndryData::define()                              30     0.0189     0.0189     0.0189   0.56%
MLCellLinOp::solutionResidual()                 109     0.0188     0.0188     0.0188   0.56%
amrex::Dot()                                    972    0.01821    0.01821    0.01821   0.54%
Gravity::get_old_grav_vector()                    9    0.01779    0.01779    0.01779   0.53%
MLMG::mgVcycle_up::1                             70    0.01702    0.01702    0.01702   0.51%
amrex::average_down                             392    0.01517    0.01517    0.01517   0.45%
FabArray::norminf()                             670    0.01438    0.01438    0.01438   0.43%
MLMG::addInterpCorrection()                     340     0.0139     0.0139     0.0139   0.41%
MLMG::mgVcycle_down::2                           70    0.01381    0.01381    0.01381   0.41%
MLMG::mgVcycle_down::3                           70    0.01298    0.01298    0.01298   0.39%
MLMG::prepareForSolve()                          22    0.01296    0.01296    0.01296   0.38%
Amr::grid_places()                                6    0.01235    0.01235    0.01235   0.37%
MLPoisson::Fapply()                            1025    0.01231    0.01231    0.01231   0.37%
AmrMesh::MakeNewGrids()                           6    0.01225    0.01225    0.01225   0.36%
MLMG::computeResidual()                          70    0.01184    0.01184    0.01184   0.35%
FabArray::Xpay()                                731    0.01155    0.01155    0.01155   0.34%
Castro::construct_old_source()                   63    0.01112    0.01112    0.01112   0.33%
Castro::computeNewDt()                            8     0.0111     0.0111     0.0111   0.33%
MLMG::mgVcycle_down::4                           60    0.01087    0.01087    0.01087   0.32%
MLCGSolver::ParallelAllReduce                  1304    0.01038    0.01038    0.01038   0.31%
Castro::finalize_advance()                        9    0.01031    0.01031    0.01031   0.31%
MLMG::mgVcycle_up::2                             70    0.01009    0.01009    0.01009   0.30%
MLMG::mgVcycle_up::3                             70    0.01008    0.01008    0.01008   0.30%
Castro::initData()                                1   0.009331   0.009331   0.009331   0.28%
Castro::FluxRegCrseInit                           9   0.008892   0.008892   0.008892   0.26%
MLMG::computeMLResidual()                        25   0.008831   0.008831   0.008831   0.26%
MLMG::mgVcycle_up::4                             60   0.008818   0.008818   0.008818   0.26%
Castro::construct_new_gravity_source()           17   0.008617   0.008617   0.008617   0.26%
MLCellLinOp::reflux()                            17   0.008391   0.008391   0.008391   0.25%
MLMG::computeResWithCrseSolFineCor()             10   0.007612   0.007612   0.007612   0.23%
Castro::apply_source_to_state()                  34    0.00757    0.00757    0.00757   0.22%
Castro::construct_old_gravity_source()           18   0.007557   0.007557   0.007557   0.22%
Castro::reset_internal_energy(Fab)             1708   0.007406   0.007406   0.007406   0.22%
FillPatchTwoLevels                               31   0.007041   0.007041   0.007041   0.21%
Amr::bldFineLevels()                              1   0.006794   0.006794   0.006794   0.20%
Castro::finalize_do_advance()                     9   0.006391   0.006391   0.006391   0.19%
FabArrayBase::getCPC()                         2138   0.005864   0.005864   0.005864   0.17%
MLMG::getGradSolution()                          22   0.005802   0.005802   0.005802   0.17%
MLCellLinOp::compGrad()                          26    0.00579    0.00579    0.00579   0.17%
Castro::enforce_speed_limit()                   103    0.00506    0.00506    0.00506   0.15%
MLCellLinOp::setLevelBC()                        26   0.005011   0.005011   0.005011   0.15%
FabArray::setVal(val, thecmd, scomp, ncomp)     956   0.004561   0.004561   0.004561   0.14%
FabArrayBase::CPC::define()                     910   0.004315   0.004315   0.004315   0.13%
MLMG::miniCycle()                                20   0.003968   0.003968   0.003968   0.12%
FabArray::LinComb()                             486   0.003895   0.003895   0.003895   0.12%
MLMG::ResNormInf()                               99   0.003585   0.003585   0.003585   0.11%
Castro::errorEst()                                6   0.003198   0.003198   0.003198   0.09%
FabArray::setDomainBndry()                       92   0.003194   0.003194   0.003194   0.09%
Castro::check_for_nan()                          18   0.003178   0.003178   0.003178   0.09%
MultiFab::contains_nan()                         18   0.003168   0.003168   0.003168   0.09%
FillPatchInterp(MF)                              30   0.003095   0.003095   0.003095   0.09%
MLPoisson::prepareForSolve()                     22   0.003073   0.003073   0.003073   0.09%
MLCellLinOp::prepareForSolve()                   22   0.003049   0.003049   0.003049   0.09%
Castro::init(old)                                 3   0.002815   0.002815   0.002815   0.08%
TagBoxArray::mapPRD                               6   0.002742   0.002742   0.002742   0.08%
FabArray::mult()                                 63   0.002634   0.002634   0.002634   0.08%
Castro::fill_hybrid_hydro_source()               43   0.002359   0.002359   0.002359   0.07%
check_for_negative_density()                      9   0.002148   0.002148   0.002148   0.06%
Castro::computeInitialDt()                        2   0.002017   0.002017   0.002017   0.06%
MLMG::computeResWithCrseCorFineCor()             10   0.002002   0.002002   0.002002   0.06%
Castro::construct_new_hybrid_source()            17   0.001855   0.001855   0.001855   0.06%
MLCellLinOp::updateSolBC()                       14   0.001847   0.001847   0.001847   0.05%
Gravity::update_max_rhs()                        10   0.001835   0.001835   0.001835   0.05%
Gravity::create_comp_minus_level_grad_phi()       4   0.001738   0.001738   0.001738   0.05%
Castro::apply_tagging_restrictions()              6   0.001666   0.001666   0.001666   0.05%
FabArrayBase::getFB()                          3516   0.001531   0.001531   0.001531   0.05%
Gravity::GetCrsePhi()                             8   0.001335   0.001335   0.001335   0.04%
MLCellLinOp::updateCorBC()                       10   0.001328   0.001328   0.001328   0.04%
Castro::avgDown()                                 4   0.001195   0.001195   0.001195   0.04%
Castro::avgDown(state_indx)                      20   0.001191   0.001191   0.001191   0.04%
amrex::average_down_w_geom                       20   0.001183   0.001183   0.001183   0.04%
MLMG::MLResNormInf()                             25   0.001108   0.001108   0.001108   0.03%
MultiFab::Subtract()                             20   0.001007   0.001007   0.001007   0.03%
FabArrayBase::FB::FB()                           84  0.0009419  0.0009419  0.0009419   0.03%
Gravity::average_fine_ec_onto_crse_ec()           4  0.0009068  0.0009068  0.0009068   0.03%
OwnerMask()                                       6  0.0008896  0.0008896  0.0008896   0.03%
Castro::build_fine_mask()                        40  0.0008828  0.0008828  0.0008828   0.03%
Castro::derive()                                  6  0.0008282  0.0008282  0.0008282   0.02%
MLMG::MLRhsNormInf()                             22  0.0007623  0.0007623  0.0007623   0.02%
Castro::init()                                    1  0.0006809  0.0006809  0.0006809   0.02%
AmrLevel::FillCoarsePatch()                       5  0.0006788  0.0006788  0.0006788   0.02%
MLPoisson::FFlux()                               85  0.0006776  0.0006776  0.0006776   0.02%
TagBoxArray::collate()                            6  0.0006557  0.0006557  0.0006557   0.02%
Gravity::swapTimeLevels()                         9  0.0006298  0.0006298  0.0006298   0.02%
Castro::buildMetrics()                            5  0.0006098  0.0006098  0.0006098   0.02%
FabArray::norminf(mask)                          11  0.0006097  0.0006097  0.0006097   0.02%
average_down_faces                               24   0.000578   0.000578   0.000578   0.02%
Castro::FluxRegFineAdd()                          9  0.0005642  0.0005642  0.0005642   0.02%
MultiFab::max()                                  18  0.0005325  0.0005325  0.0005325   0.02%
Castro::construct_old_hybrid_source()             9  0.0005239  0.0005239  0.0005239   0.02%
MLLinOp::define()                                22  0.0005175  0.0005175  0.0005175   0.02%
MLMG::interpCorrection_1                         10  0.0005158  0.0005158  0.0005158   0.02%
MLLinOp::defineGrids()                           22  0.0004993  0.0004993  0.0004993   0.01%
Castro::apply_problem_tags()                      6  0.0004716  0.0004716  0.0004716   0.01%
Castro::construct_new_rotation_source()          17   0.000373   0.000373   0.000373   0.01%
Castro::react_state()                            18  0.0003289  0.0003289  0.0003289   0.01%
MultiFab::Multiply()                              4  0.0002321  0.0002321  0.0002321   0.01%
Castro::construct_old_rotation_source()           9  0.0002114  0.0002114  0.0002114   0.01%
AMRErrorTag::operator()                           6  0.0001834  0.0001834  0.0001834   0.01%
AmrLevel::AmrLevel(dm)                            5  0.0001749  0.0001749  0.0001749   0.01%
Castro::swap_state_time_levels()                  9  0.0001342  0.0001342  0.0001342   0.00%
StateData::define()                              25  0.0001279  0.0001279  0.0001279   0.00%
Castro::enforce_consistent_e()                    1  0.0001168  0.0001168  0.0001168   0.00%
Castro::linear_to_hybrid_momentum()               1  0.0001162  0.0001162  0.0001162   0.00%
MLLinOp::makeAgglomeratedDMap                    22  0.0001145  0.0001145  0.0001145   0.00%
Castro::create_source_corrector()                 9  9.333e-05  9.333e-05  9.333e-05   0.00%
makeSFC                                         102  8.568e-05  8.568e-05  8.568e-05   0.00%
DistributionMapping::Distribute()               104  4.204e-05  4.204e-05  4.204e-05   0.00%
FabArrayBase::TheFPinfo()                        25   3.88e-05   3.88e-05   3.88e-05   0.00%
Amr::writeSmallPlotFile()                         1  3.511e-05  3.511e-05  3.511e-05   0.00%
AmrMesh-cluster                                   4  2.077e-05  2.077e-05  2.077e-05   0.00%
FPinfo::FPinfo()                                  3  2.002e-05  2.002e-05  2.002e-05   0.00%
FabArray::RecvLayoutMask()                        4  1.749e-05  1.749e-05  1.749e-05   0.00%
DistributionMapping::SFCProcessorMapDoIt()        2  1.477e-05  1.477e-05  1.477e-05   0.00%
ClusterList::intersect()                          4  1.333e-05  1.333e-05  1.333e-05   0.00%
Amr::initSubcycle()                               1   1.01e-05   1.01e-05   1.01e-05   0.00%
BoxList::parallelComplementIn()                  12  9.657e-06  9.657e-06  9.657e-06   0.00%
Castro::retry_advance_ctu()                       9  5.211e-06  5.211e-06  5.211e-06   0.00%
Gravity::set_mass_offset()                        6  3.967e-06  3.967e-06  3.967e-06   0.00%
MLLinOp::makeSubCommunicator()                   22  3.811e-06  3.811e-06  3.811e-06   0.00%
BoxList::complementIn                            12  3.785e-06  3.785e-06  3.785e-06   0.00%
ClusterList::chop()                               4    1.4e-06    1.4e-06    1.4e-06   0.00%
--------------------------------------------------------------------------------------------

Unused ParmParse Variables:
  [TOP]::mg.v(nvals = 1)  :: [2]

Device Memory Usage:
------------------------------------------------------------------------------
Name                                         Nalloc  Nfree    AvgMem    MaxMem
------------------------------------------------------------------------------
The_Arena::Initialize()                           1      1    24 MiB  9042 MiB
Castro::construct_ctu_hydro_source()           5148   5148   382 MiB  4449 MiB
Castro::reflux()                                436    436    10 MiB   666 MiB
Castro::initMFs()                               258    258   356 MiB   576 MiB
Castro::swap_state_time_levels()                175    175   173 MiB   340 MiB
StateData::define()                             175    175   225 MiB   340 MiB
FillPatchIterator::Initialize                   899    899  3597 KiB   303 MiB
Castro::initialize_do_advance()                 143    143    97 MiB   303 MiB
Amr::writePlotFile()                             56     56    23 MiB   189 MiB
Castro::problem_post_timestep()                 162    162   715 KiB   167 MiB
Castro::FluxRegCrseInit                         660    660   415 KiB   167 MiB
FluxRegister::Reflux()                          648    648  1408 KiB   160 MiB
Castro::initialize_advance()                    143    143    37 MiB   113 MiB
Castro::construct_old_source()                  143    143   110 KiB    67 MiB
Castro::construct_new_source()                  259    259   330 KiB    67 MiB
Castro::buildMetrics()                          140    140    38 MiB    61 MiB
Gravity::multilevel_solve_for_new_phi()         429    429    27 MiB    58 MiB
Castro::Castro()                                210    210  5337 KiB    50 MiB
MLMG::prepareForSolve()                        2806   2806  3858 KiB    42 MiB
Gravity::get_new_grav_vector()                  173    173   208 KiB    33 MiB
Gravity::get_old_grav_vector()                  143    143   167 KiB    33 MiB
ResizeRandomSeed                                  1      1    30 MiB    30 MiB
Castro::construct_old_gravity()                 432    432  1525 KiB    28 MiB
Gravity::create_comp_minus_level_grad_phi()     432    432  4844 KiB    27 MiB
MLCellLinOp::defineAuxData()                   7592   7592   598 KiB    16 MiB
FillPatchTwoLevels                              300    300  6209   B    11 MiB
Gravity::GetCrsePhi()                           216    216    98 KiB  8290 KiB
FillPatchInterp(MF)                             330    330  3383   B  7955 KiB
Gravity::fill_multipole_BCs()                   470    470   154 KiB  7469 KiB
Gravity::update_max_rhs()                       286    286  3759   B  7424 KiB
Gravity::actual_multilevel_solve()              143    143   364 KiB  7424 KiB
Castro::build_fine_mask()                       224    224   856 KiB  6912 KiB
Gravity::solve_for_phi()                        259    259   605 KiB  6912 KiB
Castro::derive()                                162    162  2066   B  6912 KiB
BndryData::define()                            4920   4920   375 KiB  4243 KiB
MLCellLinOp::reflux()                           255    255  2818   B  2376 KiB
Castro::wd_update()                               6      6    21 KiB  2240 KiB
AmrLevel::FillCoarsePatch()                      10     10   262   B  2095 KiB
amrex::average_down                            2631   2631  2082   B  1473 KiB
AmrMesh::MakeNewGrids()                         324    324  3819   B  1041 KiB
MLMG::addInterpCorrection()                    2460   2460  1402   B   867 KiB
MLCellLinOp::defineBC()                         232    232  2140   B   689 KiB
Castro::estTimeStep()                            26     26  2728   B   480 KiB
Castro::volInBoundary()                          70     70  2669   B   320 KiB
VisMF::Write(FabArray)                         1568   1568  2108   B   320 KiB
Castro::normalize_species()                     103    103  6610   B   320 KiB
Gravity::average_fine_ec_onto_crse_ec()          24     24    53   B   204 KiB
FabArray::norminf(mask)                          15     15    93   B   163 KiB
amrex::Dot()                                   1182   1182   796   B   160 KiB
FabArray::norminf()                             810    810   636   B   160 KiB
check_for_negative_density()                      9      9   102   B   160 KiB
Castro::initData()                                1      1    20   B   160 KiB
MultiFab::max()                                  18     18    24   B   160 KiB
FabArray::setVal()                              460    460    45 KiB    96 KiB
MLMG::interpCorrection_1                         20     20    13   B    91 KiB
average_down_faces                               51     51   443   B    69 KiB
amrex::Copy()                                   244    244    20 KiB    64 KiB
FabArray::ParallelCopy_nowait()                1182   1182   253   B    57 KiB
MLPoisson::Fsmooth()                            396    396  4658   B    55 KiB
FillBoundary_nowait()                          1280   1280   789   B    53 KiB
TagBoxArray::mapPRD                             324    324    33   B    45 KiB
OwnerMask()                                     168    168     9   B    38 KiB
MLCellLinOp::setLevelBC()                        96     96   353   B    37 KiB
MLCellLinOp::applyBC()                         7736   7736   242   B    30 KiB
MLCellLinOp::prepareForSolve()                  128    128     7   B    25 KiB
MultiFab::contains_nan()                         18     18    19   B    20 KiB
StateData::FillBoundary(geom)                  2538   2538    32   B    11 KiB
FabArray::setVal(val, thecmd, scomp, ncomp)     316    316     4   B  9808   B
MLCGSolver::bicgstab                            630    630    65   B  3584   B
MultiFab::Multiply()                              4      4   147   B  3456   B
TagBoxArray::collate()                           14     14     0   B   672   B
------------------------------------------------------------------------------

Managed Memory Usage:
-----------------------------------------------------------------
Name                             Nalloc  Nfree   AvgMem    MaxMem
-----------------------------------------------------------------
The_Managed_Arena::Initialize()       1      1  172   B  8192 KiB
-----------------------------------------------------------------

Pinned Memory Usage:
------------------------------------------------------------------------------
Name                                         Nalloc  Nfree    AvgMem    MaxMem
------------------------------------------------------------------------------
The_Pinned_Arena::Initialize()                    1      1    12 KiB  8192 KiB
VisMF::Write(FabArray)                         1624   1624   823 KiB  7168 KiB
Gravity::fill_multipole_BCs()                    42     42  1623   B   278 KiB
FabArray::setVal()                              460    460    45 KiB    96 KiB
amrex::Copy()                                   244    244    20 KiB    64 KiB
FabArray::ParallelCopy_nowait()                1182   1182   253   B    57 KiB
MLPoisson::Fsmooth()                            396    396  4658   B    55 KiB
FillBoundary_nowait()                          1280   1280   790   B    53 KiB
MLCellLinOp::applyBC()                         3868   3868   242   B    30 KiB
MLCellLinOp::prepareForSolve()                  128    128     7   B    25 KiB
OwnerMask()                                       6      6     0   B    15 KiB
StateData::FillBoundary(geom)                  2538   2538    32   B    11 KiB
Gravity::get_new_grav_vector()                    3      3  2519   B    10 KiB
FabArray::setVal(val, thecmd, scomp, ncomp)     316    316     4   B  9808   B
amrex::average_down                             197    197   608   B  4688   B
MLMG::prepareForSolve()                          26     26   324   B  3712   B
FabArray::norminf(mask)                          15     15    66   B  3472   B
MLMG::addInterpCorrection()                     130    130     3   B  3456   B
MultiFab::Multiply()                              4      4   147   B  3456   B
Castro::FluxRegCrseInit                          12     12     8   B  3456   B
FillPatchInterp(MF)                             140    140     2   B  2720   B
MLCellLinOp::setLevelBC()                       128    128     0   B  2592   B
average_down_faces                               27     27   432   B  1264   B
TagBoxArray::collate()                           16     16     0   B   784   B
amrex::Dot()                                   1182   1182     5   B   400   B
FabArray::norminf()                             810    810     4   B   272   B
Castro::build_fine_mask()                         8      8     0   B   208   B
MLCellLinOp::defineAuxData()                      4      4     0   B   208   B
Castro::reflux()                                  4      4     0   B   208   B
Castro::wd_update()                               6      6     1   B   112   B
Castro::problem_post_timestep()                  19     19     0   B    48   B
Castro::estTimeStep()                            26     26     0   B    32   B
MultiFab::max()                                  18     18     0   B    16   B
MultiFab::contains_nan()                         18     18     0   B    16   B
Castro::volInBoundary()                          70     70     0   B    16   B
Castro::normalize_species()                     103    103     0   B    16   B
Castro::initData()                                1      1     0   B    16   B
check_for_negative_density()                      9      9     0   B    16   B
------------------------------------------------------------------------------

Total GPU global memory (MB): 12056
Free  GPU global memory (MB): 2412
[The         Arena] space allocated (MB): 9042
[The         Arena] space used      (MB): 0
[The Managed Arena] space allocated (MB): 8
[The Managed Arena] space used      (MB): 0
[The  Pinned Arena] space allocated (MB): 8
[The  Pinned Arena] space used      (MB): 0
AMReX (23.06-17-g0375e78e7838) finalized