=== If no file names and line numbers are shown below, one can run addr2line -Cpfie my_exefile my_line_address to convert `my_line_address` (e.g., 0x4a6b) into file name and line number. Or one can use amrex/Tools/Backtrace/parse_bt.py. === Please note that the line number reported by addr2line may not be accurate. One can use readelf -wl my_exefile | grep my_line_address' to find out the offset for that line. 0: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x6b3cc6] amrex::BLBackTrace::print_backtrace_info(_IO_FILE*) /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_BLBackTrace.cpp:179:39 1: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x6b5e3d] amrex::BLBackTrace::handler(int) /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_BLBackTrace.cpp:85:15 2: /lib64/libc.so.6(+0x3da70) [0x7f7430522a70] __restore_rt ??:0 3: /lib64/libc.so.6(gsignal+0x145) [0x7f74305229e5] raise ??:0 4: /lib64/libc.so.6(abort+0x116) [0x7f743050b8a4] abort ??:0 5: /lib64/libc.so.6(+0x80007) [0x7f7430565007] __libc_message ??:0 6: /lib64/libc.so.6(+0x87cdc) [0x7f743056ccdc] ?? ??:0 7: /lib64/libc.so.6(+0x8956c) [0x7f743056e56c] _int_free ??:0 8: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x4403a0] amrex::BaseFab::clear() /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_BaseFab.H:2121:40 9: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x4ada72] Castro::estdt_rad() /raid/testing/castro-gfortran/Castro//Source/driver/timestep.cpp:515:64 10: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x46be10] amrex::ParallelDescriptor::Communicator() inlined at /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_ParallelDescriptor.H:1204:5 in Castro::estTimeStep() /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_ParallelDescriptor.H:212:55 void amrex::ParallelDescriptor::detail::DoAllReduce(double*, int, int) /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_ParallelDescriptor.H:1204:5 std::enable_if::value, void>::type amrex::ParallelDescriptor::ReduceRealMin(double&) /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_ParallelDescriptor.H:1335:31 Castro::estTimeStep() /raid/testing/castro-gfortran/Castro//Source/driver/Castro.cpp:1570:42 11: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x483972] Castro::do_advance_ctu(double, double, int, int) /raid/testing/castro-gfortran/Castro//Source/driver/Castro_advance_ctu.cpp:405:28 12: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x4842dd] advance_status::operator=(advance_status&&) inlined at /raid/testing/castro-gfortran/Castro//Source/driver/Castro_advance_ctu.cpp:673:90 in Castro::subcycle_advance_ctu(double, double, int, int) /raid/testing/castro-gfortran/Castro//Source/driver/Castro.H:131:8 Castro::subcycle_advance_ctu(double, double, int, int) /raid/testing/castro-gfortran/Castro//Source/driver/Castro_advance_ctu.cpp:673:90 13: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x48253a] double const& std::min(double const&, double const&) inlined at /raid/testing/castro-gfortran/Castro//Source/driver/Castro_advance.cpp:57:92 in Castro::advance(double, double, int, int) /usr/include/c++/10/bits/stl_algobase.h:236:9 Castro::advance(double, double, int, int) /raid/testing/castro-gfortran/Castro//Source/driver/Castro_advance.cpp:57:92 14: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x73e3b9] std::vector >::operator[](unsigned long) inlined at /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_Vector.H:37:59 in amrex::Amr::timeStep(int, double, int, int, double) /usr/include/c++/10/bits/stl_vector.h:1046:39 amrex::Vector >::operator[](unsigned long) /raid/testing/castro-gfortran/amrex//Src/Base/AMReX_Vector.H:37:59 amrex::Amr::timeStep(int, double, int, int, double) /raid/testing/castro-gfortran/amrex//Src/Amr/AMReX_Amr.cpp:2018:68 15: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x73e512] amrex::Amr::timeStep(int, double, int, int, double) /raid/testing/castro-gfortran/amrex//Src/Amr/AMReX_Amr.cpp:2061:13 16: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x73e512] amrex::Amr::timeStep(int, double, int, int, double) /raid/testing/castro-gfortran/amrex//Src/Amr/AMReX_Amr.cpp:2061:13 17: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x735360] amrex::Amr::coarseTimeStep(double) /raid/testing/castro-gfortran/amrex//Src/Amr/AMReX_Amr.cpp:2125:13 18: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x42c9ed] main /raid/testing/castro-gfortran/Castro//Source/driver/main.cpp:136:33 19: /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f743050d1e2] __libc_start_main ??:0 20: ./Castro3d.gnu.TEST.TPROF.MPI.ex() [0x4351ae] _start ??:0 ===== TinyProfilers ====== main() Amr::coarseTimeStep() Amr::timeStep() Amr::timeStep() Amr::timeStep() Castro::advance() Castro::subcycle_advance_ctu() Castro::do_advance_ctu() Castro::estTimeStep()