=== If no file names and line numbers are shown below, one can run addr2line -Cpfie my_exefile my_line_address to convert `my_line_address` (e.g., 0x4a6b) into file name and line number. Or one can use amrex/Tools/Backtrace/parse_bt.py. === Please note that the line number reported by addr2line may not be accurate. One can use readelf -wl my_exefile | grep my_line_address' to find out the offset for that line. 0: ./Castro3d.gnu.TPROF.CUDA.ex() [0x6de706] amrex::BLBackTrace::print_backtrace_info(_IO_FILE*) /raid/testing/castro-gpu/amrex//Src/Base/AMReX_BLBackTrace.cpp:179:35 1: ./Castro3d.gnu.TPROF.CUDA.ex() [0x6e087d] amrex::BLBackTrace::handler(int) /raid/testing/castro-gpu/amrex//Src/Base/AMReX_BLBackTrace.cpp:85:7 2: ./Castro3d.gnu.TPROF.CUDA.ex() [0x5f8d28] std::__cxx11::basic_string, std::allocator >::_M_data() const inlined at /usr/include/c++/10/bits/basic_string.h:222:17 in amrex::Gpu::Device::synchronize() /usr/include/c++/10/bits/basic_string.h:187:24 std::__cxx11::basic_string, std::allocator >::_M_is_local() const /usr/include/c++/10/bits/basic_string.h:222:17 std::__cxx11::basic_string, std::allocator >::_M_dispose() /usr/include/c++/10/bits/basic_string.h:231:17 std::__cxx11::basic_string, std::allocator >::~basic_string() /usr/include/c++/10/bits/basic_string.h:658:13 amrex::Gpu::Device::synchronize() /raid/testing/castro-gpu/amrex//Src/Base/AMReX_GpuDevice.cpp:604:423 3: ./Castro3d.gnu.TPROF.CUDA.ex() [0x6940a5] amrex::Gpu::synchronize() inlined at /raid/testing/castro-gpu/amrex//Src/Base/AMReX_MFIter.cpp:225:36 in amrex::MFIter::~MFIter() /raid/testing/castro-gpu/amrex//Src/Base/AMReX_GpuDevice.H:201:1 amrex::MFIter::~MFIter() /raid/testing/castro-gpu/amrex//Src/Base/AMReX_MFIter.cpp:225:36 4: ./Castro3d.gnu.TPROF.CUDA.ex() [0x4df5ba] amrex::BaseFab::~BaseFab() inlined at /raid/testing/castro-gpu/amrex//Src/Base/AMReX_FArrayBox.H:272:40 in Castro::construct_ctu_hydro_source(double, double) /raid/testing/castro-gpu/amrex//Src/Base/AMReX_BaseFab.H:2020:6 amrex::FArrayBox::~FArrayBox() /raid/testing/castro-gpu/amrex//Src/Base/AMReX_FArrayBox.H:272:40 Castro::construct_ctu_hydro_source(double, double) /raid/testing/castro-gpu/Castro//Source/hydro/Castro_ctu_hydro.cpp:118:24 5: ./Castro3d.gnu.TPROF.CUDA.ex() [0x475c47] Castro::do_advance_ctu(double, double, int, int) /raid/testing/castro-gpu/Castro//Source/driver/Castro_advance_ctu.cpp:182:36 6: ./Castro3d.gnu.TPROF.CUDA.ex() [0x476aa6] advance_status::operator=(advance_status&&) inlined at /raid/testing/castro-gpu/Castro//Source/driver/Castro_advance_ctu.cpp:655:87 in Castro::subcycle_advance_ctu(double, double, int, int) /raid/testing/castro-gpu/Castro//Source/driver/Castro.H:151:8 Castro::subcycle_advance_ctu(double, double, int, int) /raid/testing/castro-gpu/Castro//Source/driver/Castro_advance_ctu.cpp:655:87 7: ./Castro3d.gnu.TPROF.CUDA.ex() [0x47496a] double const& std::min(double const&, double const&) inlined at /raid/testing/castro-gpu/Castro//Source/driver/Castro_advance.cpp:55:90 in Castro::advance(double, double, int, int) /usr/include/c++/10/bits/stl_algobase.h:237:8 Castro::advance(double, double, int, int) /raid/testing/castro-gpu/Castro//Source/driver/Castro_advance.cpp:55:90 8: ./Castro3d.gnu.TPROF.CUDA.ex() [0x7667c9] std::vector >::operator[](unsigned long) inlined at /raid/testing/castro-gpu/amrex//Src/Base/AMReX_Vector.H:37:54 in amrex::Amr::timeStep(int, double, int, int, double) /usr/include/c++/10/bits/stl_vector.h:1046:42 amrex::Vector >::operator[](unsigned long) /raid/testing/castro-gpu/amrex//Src/Base/AMReX_Vector.H:37:54 amrex::Amr::timeStep(int, double, int, int, double) /raid/testing/castro-gpu/amrex//Src/Amr/AMReX_Amr.cpp:2019:71 9: ./Castro3d.gnu.TPROF.CUDA.ex() [0x759aa9] amrex::Amr::coarseTimeStep(double) /raid/testing/castro-gpu/amrex//Src/Amr/AMReX_Amr.cpp:2126:11 10: ./Castro3d.gnu.TPROF.CUDA.ex() [0x42ae25] main /raid/testing/castro-gpu/Castro//Source/driver/main.cpp:136:29 11: /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f0d8b9451e2] __libc_start_main ??:0 12: ./Castro3d.gnu.TPROF.CUDA.ex() [0x4342de] _start ??:0 ===== TinyProfilers ====== main() Amr::coarseTimeStep() Amr::timeStep() Castro::advance() Castro::subcycle_advance_ctu() Castro::do_advance_ctu() Castro::construct_ctu_hydro_source()