commit bef02a4c6394d24578ceed82ab9d73ede5a637e2 Author: Weiqun Zhang Date: Sat Jul 10 15:09:36 2021 -0700 Parser Precision (#2154) Use double internally. If the arguments are all floats, the final result will be converted to float. This is necessary to avoid overflow in some WarpX single precision tests. Src/Base/Parser/AMReX_Parser.H Src/Base/Parser/AMReX_Parser.cpp Src/Base/Parser/AMReX_Parser_Exe.H Src/Base/Parser/AMReX_Parser_Exe.cpp Src/Base/Parser/AMReX_Parser_Y.H Src/Base/Parser/AMReX_Parser_Y.cpp Src/Base/Parser/amrex_parser.tab.h Src/Base/Parser/amrex_parser.y commit f8c2fe41d3187e69f747816f1f6ab0d38335c11e Author: asalmgren Date: Sat Jul 10 14:11:49 2021 -0700 Add to particlemeshtest (#2159) Tests/Particles/ParticleMeshMultiLevel/CMakeLists.txt Tests/Particles/ParticleMeshMultiLevel/Make.package Tests/Particles/ParticleMeshMultiLevel/main.cpp Tests/Particles/ParticleMeshMultiLevel/mypc.H Tests/Particles/ParticleMeshMultiLevel/trilinear_deposition_K.H commit d325cf8eab0d78bb356d5bcf6c90b9bef7ef63ae Author: Weiqun Zhang Date: Sat Jul 10 14:09:06 2021 -0700 IParser: Integer Parser (#2158) Add a parser for integers. Because there is rounding in integer division, it's not safe to use the real number parser for integer. The two parsers have a lot of similarity, but floating point number specific functions (e.g., `sqrt`, `sin`, etc.) are not supported in `IParser`. In addition to `/` whose result truncates towards zero, the integer parser also supports `//` whose result truncates towards negative infinity. Docs/sphinx_documentation/source/Basics.rst Src/Base/CMakeLists.txt Src/Base/Make.package Src/Base/Parser/AMReX_IParser.H Src/Base/Parser/AMReX_IParser.cpp Src/Base/Parser/AMReX_IParser_Exe.H Src/Base/Parser/AMReX_IParser_Exe.cpp Src/Base/Parser/AMReX_IParser_Y.H Src/Base/Parser/AMReX_IParser_Y.cpp Src/Base/Parser/AMReX_Parser.cpp Src/Base/Parser/AMReX_Parser_Y.cpp Src/Base/Parser/GNUmakefile Src/Base/Parser/amrex_iparser.l Src/Base/Parser/amrex_iparser.lex.cpp Src/Base/Parser/amrex_iparser.lex.h Src/Base/Parser/amrex_iparser.tab.cpp Src/Base/Parser/amrex_iparser.tab.h Src/Base/Parser/amrex_iparser.y Src/Base/Parser/amrex_parser.l Src/Base/Parser/amrex_parser.lex.cpp Src/Base/Parser/amrex_parser.y Tests/Parser/main.cpp commit fb41ccbb852a89026cda4b1d26eee6f1fab56f9e Author: asalmgren Date: Fri Jul 9 14:26:29 2021 -0700 Remove single level assertions for NeighborParticleContainers on GPUs (#2157) * Remove single-level assertions for NeighborParticleContainer on GPUs Src/Particle/AMReX_NeighborParticlesGPUImpl.H commit cebf3df6c3e068a0c4f69b04198c0ba9af94d50a Author: Candace Gilet Date: Fri Jul 9 15:08:59 2021 -0400 Docs: fix typo in author name (#2156) Docs/sphinx_documentation/source/GridCreation.rst commit b0ffd259a5bf7eba82fc47a3871ae6b61c8860a5 Author: Erik Date: Thu Jul 8 15:57:11 2021 -0400 correct sp typo (#2153) Docs/sphinx_documentation/source/GettingStarted.rst commit 91ea9a0243b48a5d66c0c09e18df16e2db60e27f Author: Erik Schnetter Date: Thu Jul 8 13:44:41 2021 -0400 Disable `jn` on Windows (#2152) Disable `jn` on Windows – not just when `dpcpp` is used, but also when building with other compilers in MINGW. Src/Base/Parser/AMReX_Parser_Y.H commit 6b2ef0b507a24d2d90a04c81895c33aaf315454a Author: Mu-Hua Chien Date: Thu Jul 8 11:24:46 2021 -0400 add an option for hypre solver for Tests/Linearsolvers/NodalPoisson (#2139) Add input-rt.hypre for NodalPoisson example Tests/LinearSolvers/NodalPoisson/GNUmakefile Tests/LinearSolvers/NodalPoisson/MyTest.H Tests/LinearSolvers/NodalPoisson/MyTest.cpp Tests/LinearSolvers/NodalPoisson/inputs-rt.hypre commit 49b6ad4a85f09f86e7601a07744ed6e63f29a164 Author: Erik Date: Thu Jul 8 11:20:46 2021 -0400 Doxygen Side-Bar Navigation, Doxygen Logo and Mainpage Content (#2150) Enables side-bar navigation for Doxygen, and adds logo and content to main splash page. Docs/Doxygen/customdoxygen.css Docs/Doxygen/doxygen.conf Docs/Doxygen/footer.html Docs/Doxygen/header.html Docs/Doxygen/main.dox Docs/Doxygen/resize_AMReX_logo.png commit e7591aabfe4578ddf9339688632700a70ac17af0 Author: Weiqun Zhang Date: Wed Jul 7 20:38:29 2021 -0700 Allow empty string Parser (#2151) Src/Base/Parser/AMReX_Parser.H Src/Base/Parser/AMReX_Parser.cpp commit 1d6b01af423bf0f39d9efba8ac59ebff75179de0 Author: Erik Date: Wed Jul 7 12:13:38 2021 -0400 Doxygen comment formatting (#2148) Src/Particle/AMReX_Particles.H commit fffd3d976c7e8a3fa1e9a59e35e7cbfbd37a2493 Author: Andrew Myers Date: Tue Jul 6 21:53:51 2021 -0700 Fix the size of the BC Vector in the multi-level version of ParticleToMesh. (#2149) Src/AmrCore/AMReX_AmrParticles.H commit 7c185502a57fb263f0819635c14f7ee84e96f0b7 Author: Andrew Myers Date: Tue Jul 6 10:56:01 2021 -0700 Make ParticleToMesh and MeshToParticle support iMultifab. (#2137) Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_MultiFab.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_iMultiFab.H Src/Base/AMReX_iMultiFab.cpp Src/Particle/AMReX_ParGDB.H Src/Particle/AMReX_ParticleContainerBase.H Src/Particle/AMReX_ParticleMesh.H Tests/Particles/ParticleMesh/main.cpp commit 901bac18bdc3635b992121ba1023b0b9d14f8729 Author: Nicholas Brunhart-Lupo Date: Fri Jul 2 15:42:16 2021 -0600 Fix crashes with Conduit Blueprint Code (#2144) ## Summary This PR contributes two fixes to the Conduit Blueprint code. ### Nestsets fix (Issue AMReX-Codes/amrex#2135) The first fix here is within `Nestsets`, which resolves a one-past the end access on `ref_ratio`. The problematic lines are 116 to 120, of the form `window["ratio/i"] = ref_ratio[level+1][0];` For some inputs, this is one-past the end of the `ref_ratio` array. It was advised in https://github.com/AMReX-Codes/amrex/issues/2135#issuecomment-872692909 to resolve this by removing the `+1`. ### MultiLevelToBlueprint fix (Issue AMReX-Codes/amrex#2133) The second fix resolves a bug inside `MultiLevelToBlueprint`, where the `box_offsets` array is not built correctly. The problematic code is here: https://github.com/AMReX-Codes/amrex/blob/14dde9aaf741f84cf85adb67a7cdc5bef4c0764a/Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp#L363-L376 During the construction of the `box_offsets` array the first round of the loop at line 364, with `i == 0` will size the `box_offsets` array to 1. Subsequent rounds through the loop will attempt to write to non-existent slots of the vector here: https://github.com/AMReX-Codes/amrex/blob/14dde9aaf741f84cf85adb67a7cdc5bef4c0764a/Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp#L374 This fix sizes `box_offsets` before the loop starts, and changes all accesses to the vector by index, not by appending. This appears to be the right fix as advised here: https://github.com/AMReX-Codes/amrex/issues/2133#issuecomment-871501258 Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp commit 85021c840b13c0f89c0db7039cda66585e9b239a Author: kngott <30483578+kngott@users.noreply.github.com> Date: Fri Jul 2 08:30:43 2021 -0700 Perlmutter Build: Adjust to allow CPU build + cleanup. (#2143) Tested on PrgEnv-nvidia, PrgEnv-gnu for USE_CUDA={TRUE,FALSE} and BL_NO_FORT={TRUE,FALSE}. Tools/GNUMake/sites/Make.nersc commit ab69825ddee58a2ca834a2437376a05c0b8a76af Author: Weiqun Zhang Date: Thu Jul 1 17:06:29 2021 -0700 Fix mismatched tag warning (#2141) AmrInfo is defined as a struct, and it was declared as class in forward declaration. This is valid C++. But the Intel oneAPI compiler gives a warning that this may result in linker errors under the Microsoft C++ ABI. Src/AmrCore/AMReX_AmrCoreFwd.H commit 83087f64af364dd7e80a191fc905a890cf1f635e Author: Weiqun Zhang Date: Thu Jul 1 17:05:57 2021 -0700 Default values for members of ParserExecutor (#2142) Set the default values of pointers inside ParserExecutor to nullptr and add a conversion operator so that it can be tested for null. Src/Base/Parser/AMReX_Parser.H commit d7360e84bb68505316645c346abd1bbe3662d7ce Author: Candace Gilet Date: Thu Jul 1 13:37:23 2021 -0400 Docs: Add labels to chapters that were missing one. (#2134) Docs/sphinx_documentation/source/Basics.rst Docs/sphinx_documentation/source/GettingStarted_Chapter.rst Docs/sphinx_documentation/source/ManagingGridHierarchy_Chapter.rst commit bee2568d831aa1c84e6a9fe24c3e69f3ec5b49a0 Author: Axel Huebl Date: Thu Jul 1 10:36:08 2021 -0700 CMake: HIP Modernizing & RDC (#2031) ## Summary Add `-fgpu-rdc` flags to HIP if requested via `AMReX_GPU_RDC`. Modernize HIP logic with recommended targets that avoid the flaky `hipcc` compiler scripts: https://rocmdocs.amd.com/en/latest/Installation_Guide/Using-CMake-with-AMD-ROCm.html#using-hip-in-cmake Add support for AMDs `clang++`/`clang` compiler for HIP instead of using the legacy `hipcc` perl wrapper as C++ compiler. This also increases support towards Cray compiler wrappers, which also refer to `clang++`/`clang` underneath (Spock/OLCF). Close #1688 ## Additional background Relocatable-device-code (RDC) flags are needed for `extern` device variable support (for codes that use global variables on device). Also needed when linking with Ascent. Follow-up to #2029 With HIP GPU RDC, static libs emitting & linking does get more fancy: https://github.com/ROCmSoftwarePlatform/rccl/pull/260 .github/workflows/hip.yml Docs/sphinx_documentation/source/GPU.rst Tools/CMake/AMReXParallelBackends.cmake commit 5c83ba4d6f5f9c4db9812bbbf0b032f94aac7205 Author: Weiqun Zhang Date: Thu Jul 1 08:10:58 2021 -0700 Update CHANGES for 21.07 (#2138) CHANGES commit 14dde9aaf741f84cf85adb67a7cdc5bef4c0764a Author: Weiqun Zhang Date: Mon Jun 28 20:29:50 2021 -0700 Parser::compileHost (#2132) Add a function that compiles the parser expression into host code only. Src/Base/Parser/AMReX_Parser.H commit 97d907762f776ff4400868029fcc53709703291c Author: Andrew Myers Date: Mon Jun 28 14:03:43 2021 -0700 Fix MR bug in NL construction - the number of cells needs to be increased by a factor of ref_fac. (#2131) Src/Particle/AMReX_NeighborParticlesI.H commit fd1fd11302601a085b2e7ff01f07d36ffa0add66 Author: Weiqun Zhang Date: Mon Jun 28 10:11:55 2021 -0700 EB Implicit Function using Parser (#2129) Add ParserIF to EB. One can use a parser function to generate geometry. For example, the following ParmParse parameters can be used in 3D to place two cylinders with a radius of 0.1 at (-0.5,0,) and (0.5,0,). ``` eb2.geom_type = parser eb2.parser_function = "max(0.01-(x+0.5)^2-y^2, 0.01-(x-0.5)^2-y^2)" ``` Put the internal data of Parser in a shared_ptr for memory safety. Docs/sphinx_documentation/source/Basics.rst Src/Base/Parser/AMReX_Parser.H Src/Base/Parser/AMReX_Parser.cpp Src/EB/AMReX_EB2.cpp Src/EB/AMReX_EB2_GeometryShop.H Src/EB/AMReX_EB2_IF.H Src/EB/AMReX_EB2_IF_Parser.H Src/EB/CMakeLists.txt Src/EB/Make.package commit 55cb40092910183729ca81e6eef3d0ce386af317 Author: Weiqun Zhang Date: Sun Jun 27 10:52:09 2021 -0700 AMReX Version (#2128) In git fails to obtain AMReX version in the make system, grep for the version number in the CHANGES file. CHANGES Tools/GNUMake/Make.defs commit 6ca4306167ed432399a41aaa7a4a80a3f15f65e9 Author: Candace Gilet Date: Fri Jun 25 21:55:09 2021 -0400 Fix typo in 3d NodeLaplacian. (#2126) This fixes an error for cases with for dz != dy Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H commit 81535291686e88a4d2cc3bf36a77bca6a6bf48c3 Author: Weiqun Zhang Date: Fri Jun 25 14:48:53 2021 -0700 Parser (#2121) Add amrex::Parser that can be used at runtime to evaluate math expressions given by strings. It works on both host and device. This is moved from WarpX with additional performance optimization of eliminating recursion. Support for local automatic variables are added. .github/workflows/linux.yml .github/workflows/style/check_tabs.sh .github/workflows/style/check_trailing_whitespaces.sh Docs/sphinx_documentation/source/Basics.rst Src/Base/AMReX_BaseFwd.H Src/Base/CMakeLists.txt Src/Base/Make.package Src/Base/Parser/AMReX_Parser.H Src/Base/Parser/AMReX_Parser.cpp Src/Base/Parser/AMReX_Parser_Exe.H Src/Base/Parser/AMReX_Parser_Exe.cpp Src/Base/Parser/AMReX_Parser_Y.H Src/Base/Parser/AMReX_Parser_Y.cpp Src/Base/Parser/GNUmakefile Src/Base/Parser/README Src/Base/Parser/amrex_parser.l Src/Base/Parser/amrex_parser.lex.cpp Src/Base/Parser/amrex_parser.lex.h Src/Base/Parser/amrex_parser.tab.cpp Src/Base/Parser/amrex_parser.tab.h Src/Base/Parser/amrex_parser.y Tests/CMakeLists.txt Tests/Parser/CMakeLists.txt Tests/Parser/GNUmakefile Tests/Parser/Make.package Tests/Parser/main.cpp commit 67b0794d02c2f13e2c0156b752c486da6863deea Author: Andrew Myers Date: Thu Jun 24 17:52:45 2021 -0700 Remove incorrect call to coarsen. (#2125) Src/Particle/AMReX_NeighborParticlesI.H commit 8575439bdbc52bda3eb38f12660a5d3c1e0ae584 Author: Weiqun Zhang Date: Wed Jun 23 22:59:37 2021 -0700 Avoid std inclusive and exclusive scan for _GLIBCXX_RELEASE < 10 (#2123) Src/Base/AMReX_Scan.H commit 5db3b4fff09400a7c040b6792e06b2d9051fd01a Author: Andrew Myers Date: Tue Jun 22 12:37:48 2021 -0700 Fix Arena docs. (#2118) * Fix Arena docs. * remove device arena from printstats output in docs Docs/sphinx_documentation/source/GPU.rst commit 91a4ff2ce1f0a950ee7092697ea48f9e814391d3 Author: c-wetterer-nelson <78513275+c-wetterer-nelson@users.noreply.github.com> Date: Tue Jun 22 14:28:39 2021 -0400 Add SENSEI Adaptor for Particles Containers (#2016) Docs/sphinx_documentation/source/BuildingAMReX.rst Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp Src/Extern/SENSEI/AMReX_AmrDataAdaptor.cpp Src/Extern/SENSEI/AMReX_AmrInSituBridge.H Src/Extern/SENSEI/AMReX_AmrInSituBridge.cpp Src/Extern/SENSEI/AMReX_AmrMeshDataAdaptor.cpp Src/Extern/SENSEI/AMReX_AmrMeshInSituBridge.H Src/Extern/SENSEI/AMReX_AmrMeshInSituBridge.cpp Src/Extern/SENSEI/AMReX_AmrParticleDataAdaptor.H Src/Extern/SENSEI/AMReX_AmrParticleDataAdaptorI.H Src/Extern/SENSEI/AMReX_AmrParticleInSituBridge.H Src/Extern/SENSEI/AMReX_InSituBridge.H Src/Extern/SENSEI/AMReX_InSituBridge.cpp Src/Extern/SENSEI/AMReX_InSituUtils.H Src/Extern/SENSEI/AMReX_InSituUtils.cpp Src/Extern/SENSEI/AMReX_ParticleDataAdaptor.H Src/Extern/SENSEI/AMReX_ParticleDataAdaptorI.H Src/Extern/SENSEI/AMReX_ParticleInSituBridge.H Src/Extern/SENSEI/CMakeLists.txt Src/Extern/SENSEI/Make.package Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReXTargetHelpers.cmake Tools/CMake/AMReX_Config.H.in Tools/GNUMake/Make.defs Tools/GNUMake/tools/Make.sensei Tools/libamrex/configure.py commit c797f6b6979aea63cc6bda574765c29ba89e7b9c Author: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com> Date: Thu Jun 17 15:45:09 2021 -0500 CI: Add job to test SENSEI build (#2115) .github/workflows/docker/sensei/Dockerfile .github/workflows/docker/sensei/install_deps.sh .github/workflows/docker/sensei/install_sensei.sh .github/workflows/docker/sensei/install_vtk_minimal.sh .github/workflows/docker/sensei/tools.sh .github/workflows/linux.yml commit 4eb054cf1d37f3e2fa79f01a17116072ccfbbcb8 Author: Weiqun Zhang Date: Thu Jun 17 10:22:09 2021 -0700 Fix warning in algoim (#2117) Split the constructor of ImplicitIntegral into two templates to avoid an unreachable loop warning. Src/EB/AMReX_algoim_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Tools/GNUMake/Make.defs commit 04464a4f6d1201dfd01fd248c3ad64c5a6809a74 Author: Andy Nonaka Date: Thu Jun 17 09:46:31 2021 -0700 Enhance DiffSameGrid2.cpp (#2114) This utility, when using L0 norm, now outputs an additional column with the absolute error divided by the maximum value on the coarse grid. This is useful for comparing/verifying data with local regions of very small (close to zero in a machine precision sense) values that may fluctuate based on roundoff errors. Tools/C_util/Convergence/DiffSameGrid2.cpp commit 501626901f500b45fce5102c01d15cd8f74c7187 Author: Andrew Myers Date: Wed Jun 16 15:49:57 2021 -0700 Don't rely on managed memory in ParticleCopyPlan (#2116) Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_ParticleCommunication.cpp commit baf0d4cd96fa81ace1c964a2e5601685e97d41f6 Author: Axel Huebl Date: Tue Jun 15 16:10:11 2021 -0700 HIP: Increase to C++17 (#2113) Bump HIP requirements to C++17 or newer (like DPC++). At the moment, we only test against AMD's clang++/hipcc, which supports C++17 anyway. .github/workflows/hip.yml Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/CMake/AMReX_Config.cmake Tools/GNUMake/comps/hip.mak commit f7ae247580204b1f607e22b09c046120d2cc6e38 Author: Weiqun Zhang Date: Tue Jun 15 14:26:44 2021 -0700 Port remaining Fortran and CPU kernels in FluxRegister to GPU (#2112) Src/AmrCore/AMReX_FLUXREG_F.H Src/AmrCore/AMReX_FLUXREG_nd.F90 Src/AmrCore/AMReX_FluxRegister.H Src/AmrCore/AMReX_FluxRegister.cpp Src/AmrCore/CMakeLists.txt Src/AmrCore/Make.package Src/Base/AMReX_FBI.H Src/Particle/AMReX_NeighborParticlesGPUImpl.H commit 659ab43420a2c4a918cda2702910771fbbc395d3 Author: Weiqun Zhang Date: Mon Jun 14 18:01:42 2021 -0700 TinyProfiler: Add spaces to print (#2110) Add more spaces so that the output looks nicer when there are multiple layers of profilers. Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_TinyProfiler.cpp commit 8182a3a630e580ed9158bd1da33276380f170d41 Author: Andrew Myers Date: Mon Jun 14 14:33:08 2021 -0700 Reimplement partitionParticlesByDest (#2109) Src/Particle/AMReX_ParticleUtil.H commit d7ff203305e8c8605ca6f92e9212acf717da56de Author: Weiqun Zhang Date: Mon Jun 14 12:10:05 2021 -0700 GpuArray template parameter (#2108) Use unsigned int as the type for size instead of std::size_t so that the forward declaration does not need to include another header. Src/Base/AMReX_Array.H Src/Base/AMReX_BaseFwd.H commit c221a567dbd8cadd474146d9cc36ba5152178fb1 Author: Weiqun Zhang Date: Mon Jun 14 10:38:00 2021 -0700 Fix a bug in in-place ExclusiveSum (#2105) For ExclusiveSum versions using memcpy to get the total sum, we need to copy the last element of the input data to the host before the scan so that it's safe for in-place scan. Also use std::exclusive_scan and inclusive_scan on CPU if available. Src/Base/AMReX_Scan.H commit 8357d530311235e3f76899051ca8d44a51b971c0 Author: Luca Fedeli Date: Mon Jun 14 17:18:20 2021 +0200 add missing class (#2106) Src/Base/AMReX_BaseFwd.H commit fc9bc1e3efdaead0253d0a55dcf1abb7abc06bd7 Author: Weiqun Zhang Date: Sun Jun 13 12:42:38 2021 -0700 Add more warnings to CI and fix warnings (#2104) Also fixed a bug in device version of `Random_int`. .github/workflows/cuda.yml .github/workflows/hip.yml .github/workflows/intel.yml .github/workflows/linux.yml .github/workflows/macos.yml Src/Base/AMReX_BLBackTrace.cpp Src/Base/AMReX_Random.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_Scan.H Src/Particle/AMReX_NeighborParticlesCPUImpl.H Src/Particle/AMReX_NeighborParticlesI.H Src/Particle/AMReX_ParticleTransformation.H Src/Particle/AMReX_WriteBinaryParticleData.H Tests/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tests/MultiBlock/Advection/main.cpp Tests/MultiBlock/IndexType/main.cpp Tests/Particles/AssignMultiLevelDensity/main.cpp Tests/Particles/GhostsAndVirtuals/main.cpp Tests/Particles/NeighborParticles/MDParticleContainer.cpp Tests/Particles/ParallelContext/main.cpp Tests/Particles/ParticleArray/main.cpp Tests/Particles/ParticleIterator/main.cpp Tests/Particles/ParticleMesh/main.cpp Tests/Particles/SparseBins/main.cpp Tests/Particles/TypeDescriptor/main.cpp Tools/CMake/AMReXParallelBackends.cmake Tools/GNUMake/comps/dpcpp.mak Tools/GNUMake/comps/gnu.mak Tools/GNUMake/comps/hip.mak Tools/Plotfile/fextract.cpp commit 2ff4519b6e16c1f0906e637559f62d183158734c Author: Weiqun Zhang Date: Sat Jun 12 08:07:04 2021 -0700 Missing defined in preprocessing (#2102) Src/Base/AMReX_FPC.cpp commit 2a676f8a25e6cd75ed2e3936bd0cd2e30141363f Author: Michael Zingale Date: Fri Jun 11 20:42:24 2021 -0400 add AUTO_BUILD_SOURCES to the dependencies for GNU make (#2101) AUTO_BUILD_SOURCES can be filled by an application code with the source files that are created automatically at build time to ensure they are made before compilation / dependency resolution. Tools/GNUMake/Make.rules commit 64e41a8d314fc28f6d6473095ac5bdcd256c445f Author: Weiqun Zhang Date: Fri Jun 11 17:20:08 2021 -0700 Add a number of forward declaration headers (#2100) Src/Amr/AMReX_AmrFwd.H Src/Amr/CMakeLists.txt Src/Amr/Make.package Src/AmrCore/AMReX_AmrCoreFwd.H Src/AmrCore/CMakeLists.txt Src/AmrCore/Make.package Src/Base/AMReX_Array4.H Src/Base/AMReX_BaseFwd.H Src/Base/CMakeLists.txt Src/Base/Make.package Src/Boundary/AMReX_BoundaryFwd.H Src/Boundary/CMakeLists.txt Src/Boundary/Make.package commit 4cf09ce1a3eb2c06460866af04b0ec420300f1d0 Author: Axel Huebl Date: Wed Jun 9 20:01:14 2021 -0700 Particle Device Members: Inline (#2097) Member functions that are not `operator()` are not inlined by default. I think we should inline those getters, since they are too tiny / too deep in hot loops to make sense to indirect. (The compiler likely figured this out on its own already, at least I saw no immediate difference in a massive WarpX kernel with NVCC and looking at ptxas info.) Src/Particle/AMReX_Particle.H commit 7eaf1ee13726cfb689d4f0bffe3b5d056b09eceb Author: Axel Huebl Date: Wed Jun 9 15:19:21 2021 -0700 CI: Split Linux Workflows (#2096) Split workflows into - linux (CPU) - cuda - intel (DPC++/ICC/ICX) - hip workflows. Individual workflows can be individually restarted and this increases order a bit as well. .github/workflows/cuda.yml .github/workflows/hip.yml .github/workflows/intel.yml .github/workflows/linux.yml commit e6c52ec95486d8d26ec679d9aea561fa6266d844 Author: Axel Huebl Date: Wed Jun 9 09:17:03 2021 -0700 CMake 3.20+ CUDA: Fast Math (#2094) Honor CUDA fast math option `AMReX_CUDA_FASTMATH` (default: ON) in modernized CUDA CMake logic. Tools/CMake/AMReXParallelBackends.cmake commit 56de38905af1fe6b1eff8fe390de50bcc65449c2 Author: Axel Huebl Date: Mon Jun 7 19:20:17 2021 -0700 CMake 3.20+ CUDA: Shell-Escape --Werror (#2093) ## Summary When adding a list of flags to CMake targets, duplicated flags get by default deduplicated. This caused with our new CMake 3.20+ CUDA logic #2012 that this gets passed to the linker: `--Werror ext-lambda-captures-this cross-execution-space-call`. The CMake `SHELL:` prefix is exactly made for this instance, keeping flags that belong together also over transformations like this together: https://cmake.org/cmake/help/latest/command/target_compile_options.html#arguments Since we injected directly into the global `CMAKE_CUDA_FLAGS` flags for earlier CMake versions, this deduplication logic does not apply and thus we have not experienced it earlier. ## Additional background Seen first with CI in AMReX Tutorials (e.g. https://github.com/AMReX-Codes/amrex-tutorials/pull/12). Tools/CMake/AMReXParallelBackends.cmake commit 88b50cf2e427c5358bed05d792f557f5da303028 Author: Wileam Y. Phan <50928756+wyphan@users.noreply.github.com> Date: Mon Jun 7 22:04:23 2021 -0400 Add link to "Building with CMake" page in INSTALL file (#2092) Add a link to "Building with CMake" page in the AMReX documentation web page into the INSTALL file. INSTALL commit 53614b4a5b8bd769918523af10341b22c57a5596 Author: Weiqun Zhang Date: Mon Jun 7 12:18:04 2021 -0700 Remove VecReduce and FabReduce (#2091) They were used for the old atomics based reduction approach. Only PeleC uses VecReduce. The use of VecReduce there can be replaced with Reduce::Sum. Src/Base/AMReX_GpuLaunchFunctsC.H Src/Base/AMReX_GpuLaunchFunctsG.H commit 79678dd3b8c1ecf92c0aaf896c4da1ab7e297465 Author: Weiqun Zhang Date: Mon Jun 7 12:17:23 2021 -0700 Add more versions of ParallelFor(MF) (#2089) Src/Base/AMReX_Algorithm.H Src/Base/AMReX_FBI.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_MFParallelFor.H Src/Base/AMReX_MFParallelForC.H Src/Base/AMReX_MFParallelForG.H Src/Base/CMakeLists.txt Src/Base/Make.package commit 87e01fe44b483adab034072dc000aed67aa1b5a7 Author: Roberto Porcu <53792251+rporcu@users.noreply.github.com> Date: Fri Jun 4 17:53:15 2021 -0400 Fix indexes bug for particles output in GPU version of packIOData (#2087) Src/Particle/AMReX_WriteBinaryParticleData.H commit 9e3fb2f574453dc13e7e1d03a53f0ce41a9ad0e1 Author: Weiqun Zhang Date: Fri Jun 4 11:32:00 2021 -0700 Remove deprecated Reduce functions (#2086) It includes Min, Max, and MinMax with the signature like `T Sum (N n, U const* v, T init_val, BOP bop)`. The other versions are still available. Src/Base/AMReX_Reduce.H commit eb2fb8eb11d4df88d5b6c04a68f2971a6c3b97f0 Author: Weiqun Zhang Date: Thu Jun 3 21:31:05 2021 -0700 Fix some macro issues in CMake Fortran build (#2084) Include AMReX_Config.H in Fortran files that need preprocessing. Do not pass those definitions already defined in AMReX_Config.H to the Fortran compiler in the cmake build. This fixes the warnings about redefined macros. Src/AmrCore/AMReX_INTERP_2D.F90 Src/AmrCore/AMReX_INTERP_3D.F90 Src/Base/AMReX_FILCC_1D.F90 Src/Base/AMReX_FILCC_2D.F90 Src/Base/AMReX_FILCC_3D.F90 Src/Base/AMReX_acc_mod.F90 Src/Base/AMReX_filcc_mod.F90 Src/Base/AMReX_fort_mod.F90 Src/Base/AMReX_omp_mod.F90 Src/Boundary/AMReX_LO_UTIL.F90 Src/EB/AMReX_ebcellflag_mod.F90 Src/F_Interfaces/AmrCore/AMReX_fillpatch_mod.F90 Src/F_Interfaces/Base/AMReX_fab_mod.F90 Src/F_Interfaces/Base/AMReX_parallel_mod.F90 Src/F_Interfaces/Particle/AMReX_particlecontainer_mod.F90 Src/LinearSolvers/MLMG/AMReX_MLLinOp_nd.F90 Tests/FortranInterface/Advection_F/Source/compute_dt_mod.F90 Tests/FortranInterface/Advection_F/Source/evolve_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/compute_dt_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/evolve_mod.F90 Tools/CMake/AMReXSetDefines.cmake commit 5fd8671f9a9e7b319b03faae03c1fdfa371626f5 Author: Weiqun Zhang Date: Thu Jun 3 10:06:26 2021 -0700 Disable kernel fusing if gpu rdc is off. (#2083) Because the kernel fusing depends on rdc. Src/Base/AMReX_GpuFuse.H Src/Base/AMReX_GpuFuse.cpp Src/Base/AMReX_GpuLaunch.H Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_GpuLaunchMacrosG.H Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReX_Config.H.in Tools/GNUMake/Make.defs commit 5fca8d0268c461720d9665f0aef6a4d25bd11381 Author: Weiqun Zhang Date: Wed Jun 2 21:23:35 2021 -0700 ParallelFor for MultiFab/FabArray (#2073) Add an experimental feature that can be used to launch a single kernel to work on MultiFabs or FabArrays. An example is shown below, where `mf` is a MultiFab. ``` auto ma = mf.arrays(); amrex::ParallelFor(mf, [=] AMREX_GPU_DEVICE (int box_no, int i, int j, int k) { ma[box_no](i,j,k) = 1.0; }); Gpu::synchronize(); ``` It's still unclear how we handle tiling and OpenMP and how it performs on CPU. However, there is strong evidence that this improves performance for GPU runs when there are small boxes. The purpose of this PR is to get the feature into the AMReX repo first and do further tuning later. Since the new function is not used at all, it should not affect any codes. Src/Base/AMReX_CArena.cpp Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_MFIter.H Src/Base/AMReX_MFParallelForC.H Src/Base/AMReX_MFParallelForG.H Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_TypeTraits.H Src/Base/CMakeLists.txt Src/Base/Make.package Src/EB/AMReX_MultiCutFab.H commit 7404b8a26a60e30cb6be4caf991522676c4bc36c Author: Andrew Myers Date: Wed Jun 2 20:34:20 2021 -0700 ParticleContainer-level reduction functions that work on tuples of data at once. (#2082) Src/Particle/AMReX_ParticleReduce.H Tests/Particles/ParticleReduce/main.cpp commit ea9345194adaf8a202a1b3a48d438697d008bdad Author: Axel Huebl Date: Wed Jun 2 20:15:29 2021 -0700 AMReX_PROBINT: Depends on AMReX_AMRLEVEL (#2081) CMake: do not enable if `AMReX_AMRLEVEL` is off anyway. Tools/CMake/AMReXOptions.cmake commit 3f70dd18d624d90161d323e218ccb398f86fecea Author: Weiqun Zhang Date: Wed Jun 2 20:14:41 2021 -0700 Clean up device random engine states (#2080) Src/Base/AMReX_Random.cpp Src/Base/AMReX_RandomEngine.H commit 241e4ef89f36363c2b00bf99141527db420cea4a Author: Weiqun Zhang Date: Wed Jun 2 20:14:12 2021 -0700 Arena aliases (#2078) Make The_Device_Arena and The_Managed_Arena an alias to The_Arena unless the memory allocation type in incompatible. This should help avoid unnecessary memory fragmentation. Src/Base/AMReX_Arena.cpp commit d20237636ccec63dcfb8540cfa681e0ebd893a40 Author: Axel Huebl Date: Wed Jun 2 16:34:29 2021 -0700 Docs: RDC Option (#2079) ## Summary Add a bit more detailed text to the AMReX_GPU_RDC option. This should help people searching for the option and explain the acronym. ## Additional background Did not yet find a way to slip in "separable compilation" as a term as well... Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/CMake/AMReXOptions.cmake commit f7fd082bc8ab099d42784f69d6fe49dea3d388a7 Author: Axel Huebl Date: Wed Jun 2 13:39:30 2021 -0700 CMake CUDA: Filter out SM Before Pascal (#2077) Filter out unsupported CUDA architectures in CMake, for both legacy and CMake 3.20+ logic. Tools/CMake/AMReXUtils.cmake commit 2acc1caacf687c92e3cb07a54f7e7ba1f4251888 Author: Andrew Myers Date: Wed Jun 2 13:24:22 2021 -0700 fix bug introduced in PR 2063 (#2076) Src/Particle/AMReX_WriteBinaryParticleData.H commit 439663145ebf0d9f5e66caad738b2bd968ed9331 Author: Andrew Myers Date: Wed Jun 2 13:02:54 2021 -0700 fix signed / unsigned comparison warning (#2075) Src/Particle/AMReX_ParticleCommunication.cpp commit c099e342334c01c8fa2055b2e1e806c62bdf3035 Author: Axel Huebl Date: Wed Jun 2 10:36:22 2021 -0700 CMake 3.20+ CUDA: Policy & Include (#2074) ## Summary Do not include the deprecated CUDA setup with CMake 3.20+ and support new policy for CUDA_ARCHITECTURES. Also fix errors of the kind: ``` ERROR: set_target_properties called with incorrect number of arguments. Value in AMREX_CUDA_ARCHS is 53;60;61;70;75;80;86;86 ``` by quoting the semicolon-separated list. ## Additional background Those are regressions to #2012 that appear with our modernized logic in CMake 3.20+ with CUDA. I saw those regressions with [HiPACE++](https://github.com/Hi-PACE/hipace). CMakeLists.txt Tools/CMake/AMReXParallelBackends.cmake Tools/CMake/AMReXTargetHelpers.cmake commit e11fa66c9755e2909280930c387020af2849c1cf Author: Weiqun Zhang Date: Tue Jun 1 15:47:46 2021 -0700 GPU RDC Support (#2061) Disable GPU RDC by default. The user can enable it by `USE_GPU_RDC=TRUE` in gnu make and `-DAMReX_GPU_RDC=ON` in cmake. Remove CUDA specific random number functions that rely on rdc. Co-authored-by: Axel Huebl Docs/sphinx_documentation/source/BuildingAMReX.rst Src/Base/AMReX.H Src/Base/AMReX.cpp Src/Base/AMReX_Arena.H Src/Base/AMReX_Random.H Src/Base/AMReX_Random.cpp Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXTargetHelpers.cmake Tools/GNUMake/Make.defs Tools/GNUMake/comps/hip.mak Tools/GNUMake/comps/nvcc.mak Tools/Plotfile/AMReX_PPMUtil.cpp commit 70922cb7cb79bc06a8843f932367c72cf7071ef7 Author: Marc Day Date: Tue Jun 1 16:45:03 2021 -0600 Generalize fextract allowing to specify coords perpendicular to slicedir (#2035) Generalize fextract. Allow spec as coordinate + dir, so that 1D slice can be positioned as desired, within constraints of backward compatibility. Tools/Plotfile/fextract.cpp commit 702af37a91c60bf3916306884ef78319beb5b527 Author: Cody Balos Date: Tue Jun 1 13:11:23 2021 -0700 add interface between sundials SUNMemory and the Arena (#1800) ## Summary As @asalmgren @jmsexton03 @mic84 @ajnonaka @dwillcox @gardner48 @cswoodward and I discussed... this PR adds an interface between the AMReX Arena and the SUNDIALS SUNMemory API. This is currently used in Pele and will be used in Nyx also. In general, any AMReX based app that also uses SUNDIALS (and its provided data structures) will likely find this useful when targeting GPUs. CC @drummerdoc ## Additional background In SUNDIALS 5.4.0, the SUNMemory API was introduced to allow users to substitute their own memory management underneath SUNDIALS data structures. In Pele, using the AMReX arena underneath the SUNDIALS data structures results in as much as a 15% speedup. I have tried to structure the code to follow AMReX conventions. Since we also will be bringing in an interface between the MultiFab and the SUNDIALS N_Vector, I have also tried to structure the code so that it will be easy to add that interface in as well. Co-authored-by: Weiqun Zhang Co-authored-by: David Gardner Co-authored-by: Michele Rosso Co-authored-by: Jean M. Sexton Docs/sphinx_documentation/source/BuildingAMReX.rst Docs/sphinx_documentation/source/GPU.rst GNUmakefile.in Src/Base/AMReX.cpp Src/CMakeLists.txt Src/Extern/SUNDIALS/AMReX_SUNMemory.H Src/Extern/SUNDIALS/AMReX_SUNMemory.cpp Src/Extern/SUNDIALS/AMReX_Sundials.H Src/Extern/SUNDIALS/CMakeLists.txt Src/Extern/SUNDIALS/Make.package Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReXThirdPartyLibraries.cmake Tools/CMake/AMReX_Config.H.in Tools/GNUMake/Make.defs Tools/GNUMake/packages/Make.sundials commit ef1c484bc467486be3cdff6c90884b0352c39ab5 Author: Andrew Myers Date: Tue Jun 1 12:27:43 2021 -0700 Do not rely on managed memory in ParticleContainer::WriteAsciiFile (#2070) This function is mainly used for small runs / debugging and does not provide filters, hence it should be fine just to copy all the particles to pinned host memory before writing them to disk. Src/Particle/AMReX_ParticleIO.H commit ecffb6a400180f2b2497e66f7c655fafd6bd6eb4 Author: Axel Huebl Date: Tue Jun 1 12:03:13 2021 -0700 HIP ROCTX: USE_RPATH Control (#2064) Just rely on USE_RPATH instead for GNUmake. Follow-up to #2057 Tools/GNUMake/comps/hip.mak commit e7a725644a3eb72dba5c9cd74d35eff4d96af8a8 Author: Andrew Myers Date: Tue Jun 1 12:02:08 2021 -0700 Particle plotfile/checkpoint without managed memory. (#2063) Src/Particle/AMReX_NeighborList.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleTile.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_WriteBinaryParticleData.H commit b5826c021316179174228e56ad74736c8823caba Author: Weiqun Zhang Date: Tue Jun 1 09:50:40 2021 -0700 Support preprocessing of AMREX_SPACEDIM in input files. (#2049) Src/Base/AMReX_ParmParse.H Src/Base/AMReX_ParmParse.cpp commit 1aa8d382af3195e96eea4625dc452c832dfedc53 Author: Weiqun Zhang Date: Tue Jun 1 08:06:13 2021 -0700 Update CHANGES for 21.06 (#2072) CHANGES commit 55ad11dae86fca6737e6e7b8c1656100e5a3bead Author: Axel Huebl Date: Fri May 28 13:03:39 2021 -0700 CI: Cover Fortran (#2067) * CI: Cover Fortran It looks like at some point we changed the `AMReX_Fortran` default from ON to OFF and forgot to adjust CI. This re-enables Fortran compile tests. * CMake: Fix Install Smoke Test (Fortran) Avoid erroring out if Fortran is enabled. .github/workflows/linux.yml Tests/CMakeTestInstall/CMakeLists.txt Tools/CMake/AMReXInstallHelpers.cmake commit 1492a91642e47515b688e58047232daa148196e0 Author: Mark Meredith Date: Fri May 28 00:59:42 2021 -0400 Set HIP_LIBRARIES before using (#2065) Tools/CMake/AMReXParallelBackends.cmake commit 52847883765f041e908350c4d2c03799f6924b1c Author: Axel Huebl Date: Thu May 27 18:09:02 2021 -0700 CMake: HIP Compiler - don't fail (#2062) Potentially, users might use `CC` (Cray Prg Envs) and later `clang++` (underlying compiler) as compilers. Only warn if not met. ROCm/Cray environments are currently being developed and we want to keep things flexible for codes trying to use the compiler wrapper instead of hpcc. Tools/CMake/AMReXParallelBackends.cmake commit 2c2c6992b4169613ba0c956a2a2bd120552f446a Author: jmsexton03 Date: Thu May 27 15:38:31 2021 -0700 Add roctx ranges to TinyProfiler (#2057) ## Summary Adds roctx markup similarly to existing nvtx markup. ## Additional background New features are controlled by added compile flag USE_ROCTX. This assumes the location of roctracer/libroctracer64 and roctracer/libroctx64 are similar to other ROCm library installations such as rocrand. Example run to generate results.json ``` export HIP_PATH=/path/to/rocm/root cd amrex/Tests/GPU/CNS/Exec/Sod make USE_HIP=TRUE TINY_PROFILE=TRUE USE_ROCTX=TRUE USE_MPI=FALSE NO_CONFIG_CHECKING=TRUE srun -n 1 ${HIP_PATH}/bin/rocprof --hsa-trace --stats --timestamp on --roctx-trace ./CNS3d.*.TPROF.HIP.ex inputs ``` This results.json file can be viewed with a browser as described in the documentation using chrome://tracing/ https://github.com/ROCm-Developer-Tools/rocprofiler/blob/amd-master/doc/rocprof.md#43rd-party-visualization-tools Co-authored-by: Axel Huebl Co-authored-by: Weiqun Zhang .github/workflows/dependencies/dependencies_hip.sh Docs/sphinx_documentation/source/BuildingAMReX.rst Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_TinyProfiler.cpp Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXParallelBackends.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReX_Config.H.in Tools/GNUMake/comps/hip.mak commit 01361ca148d01c6416e6e87ba672c83b694f3385 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Thu May 27 14:46:35 2021 -0700 Clean up some unused variables. (#2060) Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Tests/MultiBlock/IndexType/main.cpp Tests/Particles/GhostsAndVirtuals/main.cpp commit 7dcbf3946a95a96b05f6c9ca6c2ee4d7fb5e8dca Author: Mark Meredith Date: Thu May 27 17:22:44 2021 -0400 Link with HIP_LIBRARIES (#2055) * Link with HIP_LIBRARIES * Try LINK_LANGUAGE GenExpr * Only link with HIP_LIBRARIES for C++ only config * fix * Add HIP_LIBRARIES for all non-Fortran * CI: HIP no-Fortran Build * ROCm spelling Co-authored-by: Axel Huebl .github/workflows/linux.yml Tools/CMake/AMReXParallelBackends.cmake commit 0afebba6ae751a2b58fe3e1f4e8c05e2b9cf2025 Author: Lucas Esclapez <13371051+esclapez@users.noreply.github.com> Date: Thu May 27 12:46:15 2021 -0700 Extrapolator with nGrow > 1. (#2054) Extend the AMReX_Extrapolator in order to handle more than 1 layer of ghost cells. This is done by repeating the original algorithm several times while increasing the box size by 1 at every iteration. Src/Amr/AMReX_Extrapolater.H Src/Amr/AMReX_Extrapolater.cpp Src/Amr/AMReX_extrapolater_2D_K.H Src/Amr/AMReX_extrapolater_3D_K.H commit db8e47927065fcf283f6aae631f91a07b222876b Author: kngott <30483578+kngott@users.noreply.github.com> Date: Wed May 26 12:40:32 2021 -0700 Perlmutter Make: CUDA_PATH (#2059) * Perlmutter: find CUDA_ROOT or CUDA_PATH. * Cover all bases. Tools/GNUMake/sites/Make.nersc commit affbd9fe2ca8fecab786c465911067ca05a2c225 Author: drangara <69211175+drangara@users.noreply.github.com> Date: Wed May 26 15:29:53 2021 -0400 Fix to correct CUDA flag and add missing libraries in hypre cmake (#2056) * Fix to correct CUDA flag and add missing libraries in hypre cmake * use CUDAToolkit instead of find_library Tools/CMake/AMReXThirdPartyLibraries.cmake commit 4b29e5d85c2d0eb09b21f50a05a7ef590919d18b Author: kngott <30483578+kngott@users.noreply.github.com> Date: Wed May 26 10:12:30 2021 -0700 Perlmutter make (#2058) * Perlmutter make first draft. Allow Pele to start testing. * Keep mpi_nvidia. Tools/GNUMake/sites/Make.nersc commit 33d8bbb1b15009c68bc71ee72c9b067b730148d5 Author: Weiqun Zhang Date: Tue May 25 15:16:38 2021 -0700 Fix compilation of the TensorOverset test (#2053) Tests/LinearSolvers/TensorOverset/MyTest.cpp commit dedbdce8b963167b5ae8a64ca82cf9c8ce6f5d86 Author: Weiqun Zhang Date: Tue May 25 15:16:16 2021 -0700 Remove IndexSequence (#2052) Since we use C++14 now, our own implementation of index sequence can be removed. Src/Base/AMReX_IndexSequence.H Src/Base/AMReX_Tuple.H Src/Base/CMakeLists.txt Src/Base/Make.package Src/EB/AMReX_EB2_IF_Intersection.H Src/EB/AMReX_EB2_IF_Union.H Src/Particle/AMReX_ParticleArray.H commit 12b8518603dd480e01119aa445410d8684858472 Author: Weiqun Zhang Date: Tue May 25 11:49:43 2021 -0700 Use Cray wrappers to get mpi flags on spock (#2051) Tools/GNUMake/sites/Make.olcf commit 0589d3a76098d90e0f7d364234233643cb212699 Author: Mark Meredith Date: Tue May 25 14:30:38 2021 -0400 CMake link with roc::rocprim for HIP (#2050) * CMake link with roc::rocprim for HIP * link with HIP_LIBRARIES * Update Tools/CMake/AMReXParallelBackends.cmake Co-authored-by: Axel Huebl Co-authored-by: Axel Huebl Tools/CMake/AMReXParallelBackends.cmake commit fd5163bb48fdc6b802cf3e9a8d2c861954475e8c Author: Andrew Myers Date: Tue May 25 09:24:32 2021 -0700 Fix PODVector to work without managed memory. (#2048) In a number of places in `amrex::PODVector`, we were always running operations on the host, assuming the presence of managed memory. This fixes these to run either on the host or on the device, depending on the execution policy of the `PODVector`'s `Allocator` (meaning, for managed and device vectors we run on the device, for host and pinned we run on the host). Additionally, this improves the existing test of `PODVector` by passing it `amrex::the_arena_is_managed=0`, which is more sensitive to errors. This fixes an issue reported by @jmsexton03 regarding Nyx and the InitFromBinaryFile capabilities. Src/Base/AMReX_PODVector.H Tests/GPU/Vector/inputs Tests/GPU/Vector/main.cpp commit 11bdf9e45e39070d5072562db633a1da45d36bd6 Author: Andrew Myers Date: Tue May 25 09:13:36 2021 -0700 When building neighbor masks, perform setVal on device so it will work without managed memory. (#2046) This fixes an issue reported by Chris Lishka from Intel re: mfix. Src/Particle/AMReX_NeighborParticlesGPUImpl.H commit e6654533565fad924793bb3515dd1ee086d1bccb Author: kngott <30483578+kngott@users.noreply.github.com> Date: Tue May 25 09:11:52 2021 -0700 Async Output Docs (#2041) Docs/sphinx_documentation/source/IO.rst commit af5a869484df0ee842afcbf231473d1c472499e5 Author: Weiqun Zhang Date: Mon May 24 17:22:51 2021 -0700 Option to set amd gpu targets (#2038) In GNU make, one can set amd gpu targets with make argument AMD_ARCH or environment variable AMREX_AMD_ARCH. Also set MPI wrapper's underlying compiler to hipcc. This is needed if the machine is unknown. For machines known to the gnu make build system, we could either use mpicxx or hipcc -Ipath/to/mpi/include. Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/GNUMake/Make.defs Tools/GNUMake/comps/hip.mak Tools/GNUMake/sites/Make.frontier-coe Tools/GNUMake/sites/Make.olcf commit e28bbc2d08e79ccb9103d38a9df30aac9f335b16 Author: Weiqun Zhang Date: Mon May 24 17:19:55 2021 -0700 Non-GPU build on spock (#2047) Do not link to gpu aware mpi if it's not a GPU build. Tools/CMake/AMReXUtils.cmake Tools/GNUMake/sites/Make.olcf commit 727829666aa490873a747f3d00c2b3c7ddc91508 Author: Axel Huebl Date: Mon May 24 12:34:53 2021 -0700 Work-Around: oneAPI 2021.3.0 SYCL with oneDPL (#2025) Add recommended work-around flags to oneAPI 2021.3.0 SYCL compilers. https://software.intel.com/content/www/us/en/develop/articles/intel-oneapi-dpcpp-library-release-notes.html#inpage-nav-2-3 ``` The use of oneDPL together with the GNU C++ standard library (libstdc++) version 9 or 10 may lead to compilation errors (caused by oneTBB API changes). To overcome these issues, include oneDPL header files before the standard C++ header files, or disable parallel algorithms support in the standard library. For more information, please see Intel® oneAPI Threading Building Blocks (oneTBB) Release Notes. ``` ``` An application using Parallel STL algorithms in libstdc++ versions 9 and 10 may fail to compile due to incompatible interface changes between earlier versions of Threading Building Blocks (TBB) and oneAPI Threading Building Blocks (oneTBB). Disable support for Parallel STL algorithms by defining PSTL_USE_PARALLEL_POLICIES (in libstdc++ 9) or _GLIBCXX_USE_TBB_PAR_BACKEND (in libstdc++ 10) macro to zero before inclusion of the first standard header file in each translation unit. ``` Docs/sphinx_documentation/source/GPU.rst Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSYCL.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReX_Config.H.in commit 646d5f63445ff46e4df23cbdb1fbe662ca1708fb Author: Axel Huebl Date: Mon May 24 11:20:34 2021 -0700 ParamParse: Find Entries under Prefix (#2043) Extend the "unused params" checks to find generally parameters under a given inputs prefix. Demonstrator & test in: https://github.com/AMReX-Codes/amrex-tutorials/pull/7 Src/Base/AMReX_ParmParse.H Src/Base/AMReX_ParmParse.cpp commit 8935b4ca1e8b8ab2e69dc8f1f85c84a276e8efe0 Author: Max Katz Date: Mon May 24 09:14:23 2021 -0700 Add relative gradient to AMRErrorTag (#2044) Src/AmrCore/AMReX_ErrorList.H Src/AmrCore/AMReX_ErrorList.cpp commit 25b08dc3a46b9b2b96f8e004cb15e1bf67e94bef Author: Weiqun Zhang Date: Fri May 21 10:23:39 2021 -0700 Workaround argument mismatch in MPI reduce for gfortran >= 10 (#2042) In #2040, we added -fallow-argument-mismatch as a workaround. This commit reverts the change and implements an alternative approach that does not require that compiler compiler. We just need to make sure that there are no argument mismatches in the same file. Src/F_Interfaces/Base/AMReX_fi_mpi_mod.F90 Src/F_Interfaces/Base/AMReX_mpi_reduce_int.F90 Src/F_Interfaces/Base/AMReX_mpi_reduce_real.F90 Src/F_Interfaces/Base/AMReX_parallel_mod.F90 Src/F_Interfaces/Base/Make.package Src/F_Interfaces/CMakeLists.txt Tools/GNUMake/comps/gnu.mak Tools/GNUMake/comps/hip.mak Tools/GNUMake/comps/llvm.mak commit 2255a3c7196ca7ac9225f58edb82c150a2cc0ec9 Author: Andrew Myers Date: Thu May 20 16:17:37 2021 -0700 Wrappers for builtin clz extensions and tests thereof. (#2036) Src/Base/AMReX_Algorithm.H Src/Base/AMReX_Extension.H Tests/CLZ/CMakeLists.txt Tests/CLZ/GNUmakefile Tests/CLZ/Make.package Tests/CLZ/main.cpp Tests/CMakeLists.txt commit 90977cbfe14d79fa391fe4607c5b59513b9f1413 Author: Weiqun Zhang Date: Thu May 20 12:54:13 2021 -0700 Workaround for gfortran 10 (#2040) Starting from gfortran 10, mismatches between actual and dummy argument lists in a single file have been rejected with an error. This causes issues for MPI calls. This commit adds -fallow-argument-mismatch as a workaround. Tools/GNUMake/comps/gnu.mak Tools/GNUMake/comps/hip.mak Tools/GNUMake/comps/llvm.mak commit 2bbfef42fcd4b5d4386a6145f48eddab7692f907 Author: Axel Huebl Date: Thu May 20 10:27:06 2021 -0700 HIP: AMREX_AMD_ARCH Env Variable (#2034) Adding an AMREX_AMD_ARCH env variable to control the AMD GPU arch from user environments & profiles. Docs/sphinx_documentation/source/GPU.rst Tools/CMake/AMReXOptions.cmake commit 16714342b3c08a69438313a9007f93d8c5b7211b Author: Weiqun Zhang Date: Tue May 18 09:50:30 2021 -0700 Add spock support to GNU make (#2028) Tools/GNUMake/Make.machines Tools/GNUMake/sites/Make.olcf commit a1faf316ff6556aabf76fd6d605bf4a59d6c0f59 Author: Weiqun Zhang Date: Tue May 18 08:32:04 2021 -0700 nvcc --display-error-number --diag-error errNum (#2033) Since 11.2, nvcc can display a diagnostic number for any warning message and it can turn specified warnings into errors These two new flags are added to the build system, and it's now an error to write a __device__ variable in a host function. Co-authored-by: Michele Rosso Tools/CMake/AMReXParallelBackends.cmake Tools/CMake/AMReX_SetupCUDA.cmake Tools/GNUMake/comps/nvcc.mak commit aff93b8a26348bb5b42d6af9c3a27ad44e9389f9 Author: Andrew Myers Date: Tue May 18 08:21:08 2021 -0700 Build neighbor masks without managed memory (#2030) Construct these vectors on the host then copy to the device for platforms without managed memory. Src/Particle/AMReX_NeighborParticlesGPUImpl.H commit 79f79aae9f5cc21c09b0353bc0612f41daed4bad Author: Weiqun Zhang Date: Mon May 17 19:13:16 2021 -0700 Option to use oneDPL (#2027) If both USE_ONEDPL and USE_DPCPP are TRUE in GNU make, use oneDPL for the pointer based inclusive and exclusive sum. Also updated the memory fence calls and fixed the compilation for the scan of long for dpcpp. Src/Base/AMReX_FBI.H Src/Base/AMReX_Scan.H Tools/GNUMake/Make.defs Tools/GNUMake/comps/dpcpp.mak commit c6c493bb458515e3ba4b856af097562dcb49a19b Author: mic84 Date: Mon May 17 18:28:54 2021 -0700 CMake: add support for CMake >= 3.20 (#2012) * CMake: group CUDA options in a standalone file * CMake: continue enabling new CUDA support * CMake: make set_mininum_cxx_compiler_version work for a generic language * CMake: rename AMReX_Utils.cmake -> AMReXUtils.cmake * CMake: fix bug * CMake: get rid of NVCC_VERSION_MAJOR and NVCC_VERSION_MINOR * CMake: some upgrades * CMake: upgrade CUDA support * CMake: fix bug * CMake: forgot to guard a set_target_property * CMake: handle CMAKE_CUDA_ARCHITECTURES * CMake: add CUDA flags * CMake: bump up minimum GCC version to 5.1. * CMake: add print for debugging * CMake: add support for PLATFORM_ID genex * Revert "CMake: add print for debugging" This reverts commit 146f645ebae1ac5525e1f5f11cbd167e479dd00f. * CMake: evaluate SHELL genex * CMake: fix printing out of CUDA flags * CMake: fix bug * Update Doc * Update Docs/sphinx_documentation/source/GPU.rst Co-authored-by: Axel Huebl Co-authored-by: Axel Huebl CMakeLists.txt Docs/sphinx_documentation/source/GPU.rst Src/CMakeLists.txt Tools/CMake/AMReXCUDAOptions.cmake Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXFlagsTargets.cmake Tools/CMake/AMReXGenexHelpers.cmake Tools/CMake/AMReXParallelBackends.cmake Tools/CMake/AMReXTargetHelpers.cmake Tools/CMake/AMReXUtils.cmake Tools/CMake/AMReX_Config.cmake Tools/CMake/AMReX_SetupCUDA.cmake Tools/CMake/AMReX_Utils.cmake commit 9cf1ca565447d5776bd7e7fadd01fdf2e114d53e Author: Weiqun Zhang Date: Mon May 17 17:46:06 2021 -0700 Turn on RDC support for HIP (#2029) This will allow `extern` device variables. Tools/GNUMake/comps/hip.mak commit 6f1085a23de1f7ee1a5f92dd173e984c59839511 Author: Max Katz Date: Sun May 16 11:05:05 2021 -0700 Device profilerStart and profilerStop APIs (#2026) Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp commit c6a8fa243e8449938b8b4b5b16a9e95df9487d48 Author: Emmanuel Motheau Date: Sun May 16 11:04:20 2021 -0700 Workaround to fix the AugmentPlotfile tool (#2023) Tools/C_util/AugmentPlotfile/AugmentPlotfile.cpp commit 442c6302b76ea0ce7d0018f69ae957686d4c4add Author: Weiqun Zhang Date: Fri May 14 18:46:38 2021 -0700 Use rocm 4.1.1 for hip ci (#2024) .github/workflows/dependencies/dependencies_hip.sh .github/workflows/linux.yml commit 6795c1f45833f1784b24d2277664d670f8c7171c Author: Max Katz Date: Fri May 14 17:41:34 2021 -0700 Make nvcc play nicely with CC wrapper (#2019) Tools/GNUMake/Make.defs Tools/GNUMake/comps/cray.mak commit ab84978412f2a00aabf966cbc54b87da39f9ab60 Author: Max Katz Date: Fri May 14 17:34:08 2021 -0700 Add Perlmutter to list of NERSC hosts (#2021) Tools/GNUMake/Make.machines Tools/GNUMake/sites/Make.nersc commit 65226552590664d61d32b2d3d8e0de7e691d7e62 Author: Max Katz Date: Fri May 14 17:32:59 2021 -0700 Cray ftn does not accept isystem (#2020) Tools/GNUMake/Make.defs commit d0210641e10dd257a5f2942e992e3c9ab5a4fcfb Author: Max Katz Date: Fri May 14 17:31:38 2021 -0700 Clang Cray can also show up with explicit Clang identifier (#2018) Tools/GNUMake/comps/cray.mak commit 2756788d7b06c3cc7ad5c471ee271edcfa484845 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Fri May 14 12:20:10 2021 -0700 Label function HOST_DEVICE. (#2022) Src/Base/AMReX_RealBox.H commit f7d87b8be6b9bc9373aa17e8b2805256afcd0c55 Author: Max Katz Date: Fri May 14 08:37:11 2021 -0700 Remove restrictions on nvcc host compiler (#2017) These restrictions were originally enforced back in the days of CUDA Fortran, there is no need to require this now. Tools/GNUMake/Make.defs commit 6c9a3f73f3034b5e2b299b6d0a6a7a4333dc1875 Author: Weiqun Zhang Date: Wed May 12 21:58:45 2021 -0700 Remove minBlocksPerMultiprocessor from launch bounds (#2015) After reading CUDA Programming Guide, I think we should remove minBlocksPerMultiprocessor from launch bounds, so that it's closer to what we had before. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#launch-bounds Src/Base/AMReX_GpuLaunchGlobal.H commit 19a2152a097dda52df64e2e42ba60e267fda74be Author: Weiqun Zhang Date: Wed May 12 20:53:02 2021 -0700 Loosen a HIP regresstion test's tolerance (#2014) Tools/RegressionTesting/AMReX-hip-tests.ini commit ad3eea065f3387fd374d9480a31419ba3bbb72bc Author: Weiqun Zhang Date: Wed May 12 20:49:06 2021 -0700 Launch bounds (#2013) Set launch bounds to AMREX_GPU_MAX_THREADS (256 by default) for all kernel launches. This helps resolve some HIP compiler issues we have had recently. Src/Base/AMReX_GpuLaunch.H Src/Base/AMReX_GpuReduce.H commit 50847f55331c15f10e38f4e710c0fef292737fe5 Author: Weiqun Zhang Date: Wed May 12 07:45:20 2021 -0700 Disable HIP CI (#2011) ROCm 4.2 has a bug that causes the ci to fail. .github/workflows/linux.yml commit 73dc2422bb07c7346c29c01d1bbfa5a83352ceae Author: asalmgren Date: Tue May 11 20:45:04 2021 -0700 Test on vfrac rather than isRegular when doing the slope limiting -- we were doing this in one (#2008) place but not another place which made the behavior inconsistent. Src/EB/AMReX_EB_slopes_K.H commit 20be66e90b6685865f534ab83b1b8850ad848bcb Author: Weiqun Zhang Date: Tue May 11 16:44:04 2021 -0700 Arena initial size (#2007) Add ParmParse parameters to allow the users to change the initial size of various Arenas. In particular this allows the users to not allocate any managed memory if they set `amrex.the_arena_is_managed=0 amrex.the_managed_arena_init_size=0`. Src/Base/AMReX_Arena.cpp commit 8a650f69446681320504fe5156591dc547bcb0c9 Author: Weiqun Zhang Date: Tue May 11 16:34:48 2021 -0700 Use CPU memory in memory pools for OpenMP threads (#2006) The memory pools are primarily used by Fortran functions as scratch space inside OpenMP parallel regions. There are no reasons to allocate them in managed device memory. Src/Base/AMReX_MemPool.cpp commit bf524ee95ff4a068ddf0a27599d4ce652e25b3cb Author: Weiqun Zhang Date: Tue May 11 13:02:03 2021 -0700 No casting to unsigned long long needed in iMultiFab::sum (#2005) Because we no longer use atomics in reduction, we do not need to cast to unsigned long long in iMultiFab::sum anymore. Have to install tgtermes.sty to build the docs. .github/workflows/dependencies/documentation.sh Src/Base/AMReX_iMultiFab.cpp commit fb0c16e34b9340e68ee63013e64570240e436815 Author: Max Katz Date: Sat May 8 16:12:26 2021 -0700 Avoid call to amrex::max with only one argument in 1D (#2004) Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp commit be62957c227508cda828bd4d4b7ea2ab9d3a2b87 Author: Weiqun Zhang Date: Fri May 7 14:03:21 2021 -0700 Cray complier type detection (#2001) There are two types of Cray compilers, the classical and clang based. The version number is not a reliable way to determine the type. For example, cce/9 on Cori has both versions. Cray compilers for aarch64 on Ookami are classical. Thus we switch to grep "LLVM" in `CC --version`. This works on both Cori and Ookami. Tools/GNUMake/comps/cray.mak commit 3c3bb354d7fb8c9d4b1072d5aabab74b7cc34117 Author: Weiqun Zhang Date: Fri May 7 13:25:22 2021 -0700 Arm Clang (#2002) Add support for Arm Clang in GNU make. Tools/GNUMake/Make.defs Tools/GNUMake/comps/armclang.mak Tools/libamrex/configure.py Tools/libamrex/mkconfig.py commit 9496af4b7bc2e768608c026b06e3afe6f55ce01c Author: Andrew Myers Date: Fri May 7 10:56:30 2021 -0700 Move Morton code utility functions somewhere they can be reused. (#1999) Also ports them to GPU. Src/Base/AMReX_DistributionMapping.cpp Src/Base/AMReX_Morton.H Src/Base/CMakeLists.txt Src/Base/Make.package commit ce04df6c1cb870860672c218f3e3b641f1ec24ec Author: Andrew Myers Date: Wed May 5 17:05:05 2021 -0700 Templated implementation of ostream operator for both Dim3 and XDim3 (#1997) Src/Base/AMReX_Dim3.H Src/Base/AMReX_Dim3.cpp Src/Base/CMakeLists.txt Src/Base/Make.package commit 8d7d25ff6ae426f31cd609a97d66bc8608fc03d1 Author: Weiqun Zhang Date: Wed May 5 16:57:15 2021 -0700 Disable HIP device assertion (#1998) It makes the compliation very slow and it does not currently work. Src/Base/AMReX.H Src/Base/AMReX_GpuQualifiers.H commit a2a23d75fafb05f7f2a974d0fe7bf5ff584a2823 Author: Andrew Myers Date: Wed May 5 14:59:01 2021 -0700 Generalize elemwiseMin and elemwiseMax. (#1996) This allows these functions to be called on `XDim3` objects, as well as any other object that has `x`, `y`, and `z` data members, instead of just `Dim3`. Additionally, it makes them variadic so they work on any number of objects, instead of just two. Src/Base/AMReX_Algorithm.H commit c1375ccb916f35caca50cc52c72535025e179124 Author: Weiqun Zhang Date: Wed May 5 14:29:04 2021 -0700 Add new parameter to BoxArray::contains(BoxArray) (#1995) In regrid, we need to figure out how many cells we need to grow the current coarse level grids in order for the fine grids to be properly nested. We used BoxArray::contains(BoxArray) in a loop and grew the coarse BoxArray in the loop. In this commit, we add a new parameter to BoxArray::contains(BoxArray) to specify the number of grow cells without having to actually grow the BoxArray. This is better than the previous version because growing BoxArray would invalidate the hash data in BoxArray. We also modify the implementation of BoxArray::contains(Box) in this commit. Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp commit b8dad4457ae795d39bfc4d9e7c3c1927102873df Author: kngott <30483578+kngott@users.noreply.github.com> Date: Tue May 4 15:50:50 2021 -0700 Remove Warnings in DEBUG in AssignParticle Test (#1993) Tests/Particles/AssignDensity/main.cpp commit c6702b5772d40bd7d34aeb65fa445510e8bce01e Author: Weiqun Zhang Date: Tue May 4 15:49:22 2021 -0700 Optimizing away temporary BoxArrays in AmrMesh::MakeNewGrids (#1994) Previously, we stored the information needed for proper nesting in BoxList. Then when they were used, they were repeatedly converted to BoxArray for hash based fast intersection. In this commit, we store them directly in BoxArray, thus removing those hidden temporaries. Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp Src/AmrCore/AMReX_AmrMesh.cpp Src/AmrCore/AMReX_Cluster.H Src/AmrCore/AMReX_Cluster.cpp Src/AmrCore/AMReX_TagBox.H Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX_BoxDomain.H Src/Base/AMReX_BoxDomain.cpp commit 57e8af38fa513674ad9097f6bc6858d8eb669015 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Tue May 4 14:43:40 2021 -0700 Non-blocking SumBoundary (#1991) Non-blocking SumBoundary, using the stored MF pointer in PCData. Src/Base/AMReX_MultiFab.H Src/Base/AMReX_MultiFab.cpp commit bc335117a000dd5df511d2dc298ce041e09bafee Author: Weiqun Zhang Date: Tue May 4 12:40:10 2021 -0700 Remove an expensive assertion in BoxArray::contains (#1992) The assertion gets extremely expensive for large runs with long skinny patches when it's used by ClusterList::intersect. Src/Base/AMReX_BoxArray.cpp commit 50280bc83decd9c04df80697db3d95ca2d70b947 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Tue May 4 12:29:26 2021 -0700 Docs: Add non-blocking comms. (#1988) Very brief discussion of non-blocking comms for the documentation (currently, I can't find any reference). Docs/sphinx_documentation/source/Basics.rst commit 741fdd27e355d2c5992facd79d41fc283c4ea777 Author: Zach Jibben Date: Mon May 3 22:12:21 2021 -0600 Define BL_USE_MPI compilation flag when MPI and Fortran are enabled (#1990) Resolves #1989 Tools/CMake/AMReXSetDefines.cmake commit b451e90934eb0a8f0d10878e4e087b3aa933d78e Author: kngott <30483578+kngott@users.noreply.github.com> Date: Mon May 3 16:10:47 2021 -0700 FBData object (#1983) Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayCommI.H commit b27fc508b608eb6f78a9d9bcfe11f1ef0a226461 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Mon May 3 15:41:18 2021 -0700 Deprecate FabArray::copy() (#1985) This deprecates the legacy `FabArray::copy()` in favor of the better named `FabArray::ParallelCopy()` functions. This should help alleviate the problem of confusion between `FabArray::copy()` and `MultiFab::Copy()`, which is a local copy. Also removes "copy" calls from the rest of the repo. Src/Amr/AMReX_AuxBoundaryData.cpp Src/AmrCore/AMReX_FillPatchUtil.cpp Src/AmrCore/AMReX_FillPatchUtil_I.H Src/Base/AMReX_FACopyDescriptor.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_MultiFabUtilI.H Src/Base/AMReX_VisMF.cpp Src/Boundary/AMReX_FabSet.cpp Src/EB/AMReX_EBMultiFabUtil.cpp Src/Extern/ProfParser/AMReX_BLProfUtilities.cpp Src/Extern/amrdata/AMReX_DataServices.cpp Src/Particle/AMReX_ParticleMesh.H Src/Particle/AMReX_TracerParticles.cpp commit ff82946b64fa3dcd4c01df52e564c83ecd752089 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Mon May 3 15:29:33 2021 -0700 PCData object for non-blocking ParallelCopy (#1973) Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayCommI.H commit 783cbf183adac05004bf1b048942888243e59032 Author: Weiqun Zhang Date: Mon May 3 10:39:06 2021 -0700 Fix proper nesting issue for large error buffer (#1987) Reimplement the code that ensures proper nesting by projecting the fine grids onto the coarse. The previous implementation has a bug that causes it to fail when the error buffer is very large. In the very beginning of BoxLib, the enforcement of proper nesting was actually done in a similar way as the approach in this commit (i.e., calling TagBoxArray::setVal(BoxArray,TagBox::CLEAR) with projected down BoxArray). However, it had the issue of buffering the buffer cells, creating unnecessarily big refinement patches. That issue was fixed in a block of very complicated code that was hard to reason, also in the very beginning of BoxLib days. Since it worked, the code had not been touched at all. However, it was reported recently it failed when a very large error buffet was used. In this commit, we switched back to the original setVal approach and it is performed after the tagged cells are buffered already, thus avoiding the issue of buffering the buffer cells. Furthermore, it is performed after the tags are coarsened, thus performing less work. This also adds a check to make sure that blocking factors do not vary too much between levels. Otherwise regrid will break. Src/AmrCore/AMReX_AmrMesh.cpp Src/AmrCore/AMReX_TagBox.cpp commit 2b397a9ba0c99376177ec33138e8ed3de707cd8a Author: Houjun Tang Date: Mon May 3 10:36:17 2021 -0700 Update HDF5 I/O to use new HDF5 Async VOL APIs (#1948) Src/Base/AMReX_PlotFileUtil.cpp Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_ParticleUtil.cpp Src/Particle/AMReX_Particles.H Tests/HDF5Benchmark/GNUmakefile Tests/HDF5Benchmark/main.cpp commit c670d955e90912515389f4715321abe67c4cf206 Author: Mark Meredith Date: Mon May 3 11:59:13 2021 -0400 Remove unused variables (fix warnings) (#1976) Found when building MFiX Src/EB/AMReX_EB_slopes_K.H commit cff96a9f23638819cff688071ff2588d8cfadd63 Author: Weiqun Zhang Date: Sun May 2 10:26:43 2021 -0700 Update CHANGES for 21.05 (#1986) CHANGES commit 8e8b3d75376a081c02c1735fe8d1520a12bfb9f3 Author: PaulMullowney <60452402+PaulMullowney@users.noreply.github.com> Date: Fri Apr 30 10:19:42 2021 -0600 Hypre Default Params (#1984) This PR reverts the default values for bamg_relax_type back to 6 (Symmetric Gauss Seidel). This appears to be more robust than 11 (2 stage Gauss Seidel). Src/Extern/HYPRE/AMReX_HypreIJIface.cpp commit fe2830d4dd426c96cc0454dfb337e08bd489c7e9 Author: PaulMullowney <60452402+PaulMullowney@users.noreply.github.com> Date: Thu Apr 29 16:48:56 2021 -0600 Fix to this parameter (#1982) Src/Extern/HYPRE/AMReX_HypreIJIface.cpp commit bfca28a1f40a9774790d09c3247b39675986aaae Author: PaulMullowney <60452402+PaulMullowney@users.noreply.github.com> Date: Thu Apr 29 10:12:41 2021 -0600 modifying some hypre params and changing defaults. (#1981) Src/Extern/HYPRE/AMReX_HypreIJIface.cpp commit 47d580e2943dcc90998b8852ead53baaf4dd9a76 Author: cdaley Date: Wed Apr 28 19:57:11 2021 -0700 Support OpenMP offload with the NVIDIA compiler (#1980) These changes enable the ElectromagneticPIC OpenMP offload application to succeed (Particles/ElectromagneticPIC/Exec/OpenMP). It requires NVIDIA HPC SDK 21.3 or higher. Earlier versions of NVIDIA HPC SDK fail for one of two reasons: no interoperability between CUDA and OpenMP offload, or failure to support Fortran automatic arrays in OpenMP offload regions. I tested on Cori-GPU with nvhpc/21.3 and cuda/11.1.1. Co-authored-by: Christopher Daley Tools/GNUMake/Make.defs Tools/GNUMake/comps/nvhpc.mak Tools/libamrex/mkconfig.py commit a61b40464f3f16eff420d19d99cff0fa010eb2b6 Author: Weiqun Zhang Date: Tue Apr 27 11:32:55 2021 -0700 Use full path to AMReX_Config.H in gnu make target (#1979) Tools/GNUMake/Make.rules commit 30879b025d222b383aa22540598dc1891ce73478 Author: Weiqun Zhang Date: Mon Apr 26 16:43:20 2021 -0700 Fix GNU make for nvcc 11.0 (#1977) This fixes a bug introduced in #1967. It turns out nvcc 11.0 does not support -MP. Tools/GNUMake/comps/nvcc.mak commit 0340e766b71005c9675d6e9b759c66bb5c705297 Author: Weiqun Zhang Date: Mon Apr 26 13:20:16 2021 -0700 Add CI for 2D Single Precision EB HIP build (#1974) Also fix some warnings. .github/workflows/linux.yml Src/Base/AMReX_GpuLaunchGlobal.H Src/Base/AMReX_Reduce.H Src/EB/AMReX_EB2_2D_C.cpp commit 521ff492834377cc5ee91d6eb72db2471ccde3b7 Author: Weiqun Zhang Date: Mon Apr 26 13:00:09 2021 -0700 Fix dependency of f90.depends (#1975) Tools/GNUMake/Make.rules commit b1709947c0a354dc5dbefeb3421497478ef2333c Author: Weiqun Zhang Date: Mon Apr 26 09:07:35 2021 -0700 GNU Make: -MMD -MP (#1967) Use -MMD -MP instead of -M to generate dependency unless it is not supported by the compiler. In the new approach, the dependencies are generated during compilation without a separate step. Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules Tools/GNUMake/comps/cray.mak Tools/GNUMake/comps/hip.mak Tools/GNUMake/comps/llvm.mak Tools/GNUMake/comps/nvcc.mak Tools/GNUMake/comps/pgi.mak commit e6833a86ebf35861017cfd5f16ca66ac9dc8b7ae Author: Luca Fedeli Date: Mon Apr 26 17:13:19 2021 +0200 Fix missing Real() in AMReX_EB2_2D_C.cpp (#1972) This PR fixes few missing `Real()` in `AMReX_EB2_2D_C.cpp`, which currently prevent the compilation of a function in single precision. Src/EB/AMReX_EB2_2D_C.cpp commit 32597f78f68fc8b948ceb10a22c0d01ab2b513ec Author: Weiqun Zhang Date: Mon Apr 26 08:11:58 2021 -0700 Undeprecate ReduceData::value() (#1970) ReduceData::value() with no argument was recently marked as a depracated function because it involves calling a std::function. However, the call is never made inside a loop like MFIter. So there is no practical difference in performance between ReduceData::vaule and ReduceData::value(ReduceOps&). It is hereby undeprecated to avoid warning messages for a lot of applications. Src/Base/AMReX_Reduce.H commit 5eedbae2d805d86014788427bee39e6cccb2155c Author: Shreyas Ananthan Date: Sun Apr 25 13:12:48 2021 -0600 Workaround compile error on Apple M1 (ARM64) (#1969) Fixes #1968 Src/Base/AMReX.cpp commit 41d1ea1873494d5e59c6f7da16d73b634e2a9362 Author: asalmgren Date: Fri Apr 23 10:54:14 2021 -0700 Changes to EB slopes (#1965) * 1) Change test on flag.isRegular() to (vfrac == 1) -- this only matters when we have cells with vfrac == 1 but with an EB face -- this change will allow us to use the non-EB stencil when parallel to an EB face if all the vfrac's we need are = 1, and is also consistent with how we treat slopes tangential to domain boundaries 2) pass max_order in so we can default to 4th order if max_order == 4 and there are enough cells with vfrac == 1 3) Modify the Barth-Jespersen limiting so that, for example, it doesn't use cells at i-1 when computing a slope on the i+1/2 face. NOTE: this contains a change to the interfaces to the slope routines so all code that calls these will need to be modified to pass in vfrac and max_order * Remove trailing white spaces * remove more trailing white spaces * the last of the trailing white spaces ... * Compile in 2d Src/EB/AMReX_EB_slopes_K.H commit c0570ec8d60f1a6227b5835f9105801a40a81eab Author: Axel Huebl Date: Fri Apr 23 10:24:16 2021 -0700 Docs: CMake Multiple Archs (#1964) Answer a user question with a new doc entry. Docs/sphinx_documentation/source/GPU.rst commit e85b665bcb032f4064fb9845a925c6d3d8c8aa03 Author: Weiqun Zhang Date: Fri Apr 23 09:06:37 2021 -0700 Split Large Files (#1966) Split some large files in MLMG into smaller files to improve compile time. Src/LinearSolvers/CMakeLists.txt Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_F.cpp Src/LinearSolvers/MLMG/AMReX_MLEBTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLEBTensorOp_bc.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_eb.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_hypre.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_misc.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sten.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian_sync.cpp Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLTensorOp_grad.cpp Src/LinearSolvers/MLMG/Make.package commit 84cc7114097b548dce3b704b6eb9681b425f1e94 Author: Weiqun Zhang Date: Thu Apr 22 17:56:10 2021 -0700 Build Time Log (#1963) In GNU Make, if one uses `LOG_BUILD_TIME=TRUE`, a log of build time will be available in the end. Tools/C_scripts/gatherbuildtime.py Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules commit 6e5a03f8f8346e0f4840d1b5670f9afa60a0fc4a Author: Weiqun Zhang Date: Thu Apr 22 10:19:42 2021 -0700 Using rocPRIM in HIP Version of Scan (#1962) Reimplement HIP version of scan using rocprim. This uses a few things in rocprim::detail. Hopefully it will not break in a future release of HIP. Also some minor tweaks to the CUDA version of Scan. Src/Base/AMReX_GpuLaunchGlobal.H Src/Base/AMReX_Scan.H commit a0790de872d6b18b563712b9f1b18323b8d00072 Author: Weiqun Zhang Date: Tue Apr 20 16:21:37 2021 -0700 Reimplementation of ReduceOps (#1955) The new approach does reduction in two passes. Unlike the old approach, this does not require atomics, and has much better performance for types that do not have hardware support for atomics. Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX_BaseFab.H Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_Tuple.H Src/Base/AMReX_iMultiFab.cpp Src/EB/AMReX_EB2_GeometryShop.H Src/EB/AMReX_EB2_Level.cpp Src/EB/AMReX_EBCellFlag.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleInit.H Src/Particle/AMReX_ParticleLocator.H Src/Particle/AMReX_ParticleReduce.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_WriteBinaryParticleData.H Tests/Particles/Intersection/main.cpp commit 1796dd911bae1ae0b58ae92e4e023bdb00a0f44a Author: mic84 Date: Mon Apr 19 17:01:36 2021 -0700 CMake: implement logic to support compiler ID IntelLLVM (new in CMake 3.20) (#1959) CMake 3.20+ has its own compiler identification for the new oneAPI LLVM-ICX/DPCXX implementation. Tools/CMake/AMReXFlagsTargets.cmake Tools/CMake/AMReXGenerateConfigHeader.cmake Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSYCL.cmake commit 0a50fded0df7539b5e8e32b2a88966d8d671e264 Author: Weiqun Zhang Date: Mon Apr 19 14:19:53 2021 -0700 DPCPP: fix atomicAdd for float (#1958) We recently removed the template specialization for atomicAdd of float in the DPC++ backend. That was a mistake because SYCL does not support atomicAdd for float, and we have to use atomicCAS to implement it. Src/Base/AMReX_GpuAtomic.H Tests/Amr/Advection_AmrCore/Source/Src_K/slope_K.H commit c409f9e1ffc870fd4926190ffceffc7d0d5ad409 Author: Weiqun Zhang Date: Mon Apr 19 13:30:05 2021 -0700 CPU version of TagBox::buffer (#1957) Add a CPU version of TagBox::buffer. @MSABuschman reported in #1951 that TagBox::buffer has been very slow since commit #1258 if the error buffer size is large. The function was rewritten in #1258 to do the work on GPU. In this PR, the old version is reintroduced for CPU. Note that the current implementation is expected to have poor performance on GPU if it has a very large error buffer. It's still not clear how we should implement this function for GPU if a large error buffer is used. .github/workflows/linux.yml Src/AmrCore/AMReX_TagBox.H Src/AmrCore/AMReX_TagBox.cpp commit 4320729fe46992851b3f222d28833109cff0234e Author: jmsexton03 Date: Sun Apr 18 11:21:27 2021 -0700 Fix out of bound error in AsyncWrite (#1753) Fix out of bound error when the local number of fabs is zero. Co-authored-by: Weiqun Zhang Src/Base/AMReX_VisMF.cpp commit b30cae951ba2c7c004f06e3d7d382c7e79793887 Author: Weiqun Zhang Date: Sun Apr 18 08:15:16 2021 -0700 Fab data I/O (#1956) When Fab data are in GPU device memory (including managed memory), use pinned memory as buffer to avoid accessing the data on the host. Previously, MultiFab, plotfile and checkpoint writers except for the async versions relied on that the memory was managed and therefore could be used directly in I/O. Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_VisMF.cpp commit f21f15d8f7a3098d9fa13ac6b316ca518f464ac5 Author: Max Katz Date: Fri Apr 16 17:54:37 2021 -0700 Allow user to opt out of renaming MTMPI suffix (#1954) If MPI_THREAD_MULTIPLE is going to be on by default in an application, might as well not need to rename the suffix, so this gives the application the ability to just keep the suffix MPI. Tools/GNUMake/Make.defs commit 3812eadb83a667a615630a2ff8d92def964c3e58 Author: drangara <69211175+drangara@users.noreply.github.com> Date: Fri Apr 16 19:57:47 2021 -0400 Add triangle wave function to least squares test (#1947) * Add a triangle wave test for Least Squares centroids, fix some bugs in test setup * update doc * fix formatting * fix formatting Tests/LinearSolvers/LeastSquares/Make.package Tests/LinearSolvers/LeastSquares/MyTest.H Tests/LinearSolvers/LeastSquares/MyTest.cpp Tests/LinearSolvers/LeastSquares/README.md Tests/LinearSolvers/LeastSquares/initData.cpp Tests/LinearSolvers/LeastSquares/initTriangleWaveData.cpp Tests/LinearSolvers/LeastSquares/initTriangleWaveDataFor2D.cpp Tests/LinearSolvers/LeastSquares/inputs.2d.trianglewave commit eaad7627ff925f8ae9b290a7d6a2d5f2fba54239 Author: asalmgren Date: Fri Apr 16 15:52:15 2021 -0700 level_0_gome --> level_0_geom (#1953) Src/AmrCore/AMReX_AmrCore.cpp commit 029a8b0608beab17f440718b5b147dc5dd24b67c Author: Weiqun Zhang Date: Fri Apr 16 13:51:19 2021 -0700 Port FluxRegister::ClearInternalBorders to GPU (#1952) Src/AmrCore/AMReX_FluxRegister.cpp commit 42da8fb5ac8df41566a045bb3332df6ad24126d8 Author: Weiqun Zhang Date: Thu Apr 15 19:19:25 2021 -0700 Fix the return of MLPoisson::makeNLinOp (#1950) Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp commit 66f99da25c6a305fae47f0b952ebe670af867b19 Author: Andrew Myers Date: Thu Apr 15 18:07:28 2021 -0700 Skip temporary in ParticleToMesh when it is not needed. (#1949) Src/AmrCore/AMReX_AmrParticles.H Src/Particle/AMReX_ParticleMesh.H commit d2e37b06bed136e001ae62718ba5754720a01f0d Author: Andrew Myers Date: Thu Apr 15 11:32:42 2021 -0700 Multilevel version of ParticleToMesh (#1945) This uses the same algorithm as the AssignDensity method in `AmrCore/AMReX_AmrParticles.H` to handle the coarse / fine boundaries. However, it allows a user-specified lambda function to define the interpolation operation, as in the single-level `ParticleToMesh` routine. Src/AmrCore/AMReX_AmrParticles.H Src/Particle/AMReX_ParticleMesh.H Src/Particle/AMReX_ParticleUtil.H Tests/Particles/ParticleMesh/main.cpp Tests/Particles/ParticleMeshMultiLevel/CMakeLists.txt Tests/Particles/ParticleMeshMultiLevel/GNUmakefile Tests/Particles/ParticleMeshMultiLevel/Make.package Tests/Particles/ParticleMeshMultiLevel/inputs Tests/Particles/ParticleMeshMultiLevel/main.cpp commit b1d05dec0da3b6d0074af8850163837c53488ebf Author: Erik Schnetter Date: Thu Apr 15 13:32:54 2021 -0400 Correct typo in comment (#1946) Tools/CMake/AMReXOptions.cmake commit 914459f443f5ed54b5e33551802d10521042af9b Author: Don E. Willcox Date: Wed Apr 14 16:13:44 2021 -0700 fix greater->less in docs for abort_on_out_of_gpu_memory (#1944) Docs/sphinx_documentation/source/GPU.rst commit 87ee8f03c3b38c645312331fa75c92b6a932a87b Author: Weiqun Zhang Date: Tue Apr 13 16:45:37 2021 -0700 Use CUB for block reduction (#1943) Also removed some reduction functions that have been deprecated long ago. Src/Base/AMReX_GpuFuse.cpp Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_Reduce.H commit ba378870a10ad475c647cad3b8650151fc896580 Author: Weiqun Zhang Date: Tue Apr 13 08:24:57 2021 -0700 Reimplementation of PrefixSum with cub (#1942) For CUDA >= 11, use cub to reimplement PrefixSum that can take two lambda functions. Both cub and thrust only provide scan functions that take iterators. Src/Base/AMReX_Reduce.H Src/Base/AMReX_Scan.H commit 58c3ba169a2b8fc311395a959724ea63dc37aa3c Author: Weiqun Zhang Date: Sun Apr 11 17:03:22 2021 -0700 Avoid AMREX_NO_UNIQUE_ADDRESS (#1941) * Avoid AMREX_NO_UNIQUE_ADDRESS because it causes gcc 9.3 to crash. * Fix AMREX_NO_UNIQUE_ADDRESS and AMREX_NODISCARD macros. * Adjust compiler flag in CI to allow for C++17 extension when compiling with C++14. .github/workflows/linux.yml Src/Base/AMReX_Extension.H Src/Base/AMReX_NonLocalBC.H commit 18c5e96e04c982ef886d8aa9f6e65ebee1564f35 Author: Weiqun Zhang Date: Thu Apr 8 12:58:09 2021 -0700 Rename variable to fix a shadow warning (#1939) Src/Particle/AMReX_ParticleUtil.H commit 51ba2f7ebbc2dc759845fdab073d353ab175507f Author: Weiqun Zhang Date: Thu Apr 8 11:55:51 2021 -0700 Workaround a compiler bug(?) reported in #1937 (#1938) Src/Base/AMReX_Arena.cpp commit 7c259bab6cc583a92a8d7f6b640af6618e6206f3 Author: Weiqun Zhang Date: Wed Apr 7 17:52:04 2021 -0700 Cell-Centered Overset Solver Coarsening (#1915) Currently for the cell-centered overset multigrid solver, coarsening stops when a coarsened cell would contain both regular and masked-out cells. Here, we further coarsen the grids by ignoring the overset mask and giving the masked-out cells a huge value of alpha so that their values are essentially frozen. Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson.H Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp commit db9303290d4912f15230c792f4e1c91bce0fb0ce Author: Weiqun Zhang Date: Wed Apr 7 15:38:40 2021 -0700 Hypre Nodal Setup (#1906) * Hypre Nodal Setup * Allow CoarseningStrategy::Sigma in Hypre nodal solver. * Port Hypre nodal setup to GPU. * For EB nodal solver, label a node ad Dirichlet if its diagonal component of the stencil is zero. * remove EB stuff in hypre nodal setup because it's no longer needed due to the change in dirichlet mask Src/Base/AMReX_FBI.H Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuLaunchFunctsC.H Src/Base/AMReX_Scan.H Src/EB/AMReX_EB2_Level.cpp Src/Extern/HYPRE/AMReX_HypreNodeLap.H Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.cpp commit 8f9589682e56d7f443e42de032b027bd7b1944fa Author: mic84 Date: Wed Apr 7 14:15:50 2021 -0700 CMake: add support for BOUND_CHECK (#1934) Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReX_Config.H.in commit d5e4a3882b263c858221de1130f73e04825a2e1e Author: Luca Fedeli Date: Wed Apr 7 16:54:28 2021 +0200 Fix other missing Real() (#1933) This PR fixes few missing `Real()` in `AMReX_EB2_3D_C.cpp`, which currently prevent the compilation of some methods in single precision. Src/EB/AMReX_EB2_3D_C.cpp commit 3f3a9174286aa06d7826367b3d07b29c339d428a Author: Luca Fedeli Date: Wed Apr 7 04:13:13 2021 +0200 fix missing Real (#1932) Src/EB/AMReX_EB2_3D_C.H commit 4123e87ce3779855a638663008de8dcec5ccb99f Author: Andrew Myers Date: Tue Apr 6 12:08:23 2021 -0400 Alias HostVector to PinnedVector and update documentation. (#1931) Src/Base/AMReX_GpuContainers.H commit 4cb1c5b79906fe566d1b80777315cf4817924b1a Author: Weiqun Zhang Date: Mon Apr 5 16:56:23 2021 -0700 Make CNS work on CPU too (#1930) Src/Base/AMReX_BaseFab.H Src/Base/AMReX_FabConv.cpp Src/Base/AMReX_IArrayBox.cpp Tests/GPU/CNS/Exec/RT/cns_prob.cpp Tests/GPU/CNS/Exec/Sod/cns_prob.cpp Tests/GPU/CNS/Source/CNS.cpp commit 7c37a75e23eabc317c7a969e21153183db616fd1 Author: Axel Huebl Date: Mon Apr 5 16:15:02 2021 -0700 std::isfinite/inf for SYCL (#1929) Wrappity wrap the non-constexpr functions: https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/C-CXX-StandardLibrary/C-CXX-StandardLibrary.rst Note that `std::isnan` works by accident/virtue of being constexpr. Src/Base/AMReX_Math.H commit 1ee06e64345a52259315181ac13a821a4ac64f09 Author: Weiqun Zhang Date: Mon Apr 5 14:10:12 2021 -0700 Update RNG for latest oneAPI (#1928) MKL now has device support for uniform distribution of unsigned int and Poisson distribution. Src/Base/AMReX_Random.H commit deddea69d9f7e2b5cac2727cadfe77e0806c194b Author: Weiqun Zhang Date: Mon Apr 5 10:13:30 2021 -0700 Use rocprim for scan (#1925) .github/workflows/dependencies/dependencies_hip.sh .github/workflows/linux.yml Src/Base/AMReX_Scan.H commit 06cec5dd8ede26ea3c0646368bdbdb32f35842ae Author: Weiqun Zhang Date: Mon Apr 5 09:31:25 2021 -0700 Use pinned memory for data transfer between host and device (#1923) The amount of data to be transferred in TagBox and STLtools is likely to be big. So pinned memory is used instead of pageable. AsyncArray is sometimes used to transfer a large amount of data. So pinned memory is used. Src/AmrCore/AMReX_AmrMesh.cpp Src/AmrCore/AMReX_TagBox.H Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuAsyncArray.cpp Src/EB/AMReX_EB_STL_utils.H commit d3a24fe0519d8eb7cfd7043be2907e08ee534352 Author: Weiqun Zhang Date: Mon Apr 5 09:16:28 2021 -0700 Use cub::DeviceScan for CUDA >= 11 (#1924) Src/Base/AMReX_Scan.H Tools/GNUMake/Make.defs Tools/GNUMake/comps/hip.mak Tools/GNUMake/comps/nvcc.mak commit ddfbf3de3bdf5ced8f42d0828ed52d95fcea22ec Author: Weiqun Zhang Date: Mon Apr 5 09:16:07 2021 -0700 Fix compilation broken in #1920 (#1926) Tests/LinearSolvers/CellEB/MyTest.cpp Tests/LinearSolvers/CellEB2/MyTest.cpp Tests/LinearSolvers/EBConvergenceTest/MyTest.cpp Tests/LinearSolvers/EBflux_grad/MyTest.cpp Tests/LinearSolvers/LeastSquares/initData.cpp Tests/LinearSolvers/NodeEB/MyTest.cpp commit 5ec3d3799ff238208ae41949ce506d3491b37208 Author: Weiqun Zhang Date: Fri Apr 2 19:03:42 2021 -0700 Move Tests/CNS to Tests/EB/CNS (#1922) Tests/EB/CNS/CMakeLists.txt Tests/EB/CNS/Exec/Combustor/GNUmakefile Tests/EB/CNS/Exec/Combustor/Make.package Tests/EB/CNS/Exec/Combustor/bc_fill_nd.F90 Tests/EB/CNS/Exec/Combustor/bc_fill_nd.F90_jbb Tests/EB/CNS/Exec/Combustor/cns_prob.F90 Tests/EB/CNS/Exec/Combustor/cns_prob.F90_jbb Tests/EB/CNS/Exec/Combustor/inputs Tests/EB/CNS/Exec/Combustor/inputs.regt Tests/EB/CNS/Exec/Make.CNS Tests/EB/CNS/Exec/Pulse/GNUmakefile Tests/EB/CNS/Exec/Pulse/Make.package Tests/EB/CNS/Exec/Pulse/cns_prob.F90 Tests/EB/CNS/Exec/Pulse/inputs Tests/EB/CNS/Exec/Pulse/inputs.regt Tests/EB/CNS/Exec/ShockRef/GNUmakefile Tests/EB/CNS/Exec/ShockRef/Make.package Tests/EB/CNS/Exec/ShockRef/cns_prob.F90 Tests/EB/CNS/Exec/ShockRef/inputs Tests/EB/CNS/Exec/ShockRef/inputs.amr Tests/EB/CNS/Exec/ShockRef/inputs.regt Tests/EB/CNS/Exec/Sod/GNUmakefile Tests/EB/CNS/Exec/Sod/Make.package Tests/EB/CNS/Exec/Sod/cns_prob.F90 Tests/EB/CNS/Exec/Sod/inputs Tests/EB/CNS/Source/CNS.H Tests/EB/CNS/Source/CNS.cpp Tests/EB/CNS/Source/CNSBld.cpp Tests/EB/CNS/Source/CNS_F.H Tests/EB/CNS/Source/CNS_advance.cpp Tests/EB/CNS/Source/CNS_init_eb2.cpp Tests/EB/CNS/Source/CNS_io.cpp Tests/EB/CNS/Source/CNS_setup.cpp Tests/EB/CNS/Source/Make.package Tests/EB/CNS/Source/diffusion/Make.package Tests/EB/CNS/Source/diffusion/cns_diff_mod.F90 Tests/EB/CNS/Source/diffusion/cns_eb_diff_mod.F90 Tests/EB/CNS/Source/diffusion/cns_eb_diff_wall.F90 Tests/EB/CNS/Source/diffusion/diff_coef_mod.F90 Tests/EB/CNS/Source/fortran/CNS_derive.F90 Tests/EB/CNS/Source/fortran/CNS_divop.F90 Tests/EB/CNS/Source/fortran/CNS_dudt.F90 Tests/EB/CNS/Source/fortran/CNS_f.F90 Tests/EB/CNS/Source/fortran/CNS_nd.F90 Tests/EB/CNS/Source/fortran/CNS_physics.F90 Tests/EB/CNS/Source/fortran/CNS_tagging.F90 Tests/EB/CNS/Source/fortran/Make.package Tests/EB/CNS/Source/fortran/bc_fill_nd.F90 Tests/EB/CNS/Source/hydro/Hyp_gamma_MOL.F90 Tests/EB/CNS/Source/hydro/Hyp_gamma_MOL_EB.F90 Tests/EB/CNS/Source/hydro/Make.package Tests/EB/CNS/Source/hydro/analriem3d.F90 Tests/EB/CNS/Source/hydro/cns_eb_hyp_wall.F90 Tests/EB/CNS/Source/hydro/slope_mol_3d_gamma.F90 Tests/EB/CNS/Source/hydro/slope_mol_3d_gamma_EB.F90 Tests/EB/CNS/Source/main.cpp commit 42063585bb299c5dd220aa5b35be9e3f05594da5 Author: Weiqun Zhang Date: Fri Apr 2 18:07:50 2021 -0700 No need for AMREX_IS_TRIVIALLY_COPYABLE && AMREX_IS_TRIVIALLY_DEFAULT_CONSTRUCTIBLE (#1921) They are not needed because gcc 4 is no longer supported. Src/Base/AMReX_BaseFab.H Src/Base/AMReX_CudaGraph.H Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuBuffer.H Src/Base/AMReX_GpuContainers.H Src/Base/AMReX_GpuFuse.H Src/Base/AMReX_GpuMemory.H Src/Base/AMReX_PODVector.H Src/Base/AMReX_ParallelDescriptor.cpp Src/Base/AMReX_Reduce.H Src/Base/AMReX_TypeTraits.H commit 77fb0251e3ad99af069d5d0bcaa80128cc50062b Author: Weiqun Zhang Date: Fri Apr 2 10:30:42 2021 -0700 Use std::make_unique instead of reset since we are using C++14 (#1920) Docs/sphinx_documentation/source/IO.rst Docs/sphinx_documentation/source/LinearSolvers.rst Src/Amr/AMReX_Amr.cpp Src/Amr/AMReX_AmrLevel.cpp Src/Amr/AMReX_StateData.cpp Src/Amr/AMReX_StateDescriptor.cpp Src/AmrCore/AMReX_AmrCore.cpp Src/AmrCore/AMReX_AmrParticles.H Src/AmrCore/AMReX_ErrorList.cpp Src/AmrCore/AMReX_FillPatchUtil_I.H Src/Base/AMReX_AsyncOut.cpp Src/Base/AMReX_BackgroundThread.cpp Src/Base/AMReX_BoxArray.cpp Src/Base/AMReX_CuptiTrace.cpp Src/Base/AMReX_DistributionMapping.cpp Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_ForkJoin.cpp Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuFuse.cpp Src/Base/AMReX_MFIter.cpp Src/Base/AMReX_Machine.cpp Src/Base/AMReX_MemPool.cpp Src/Base/AMReX_MemProfiler.cpp Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_ParallelContext.cpp Src/Base/AMReX_PlotFileDataImpl.cpp Src/Base/AMReX_PlotFileUtil.cpp Src/Base/AMReX_iMultiFab.cpp Src/EB/AMReX_EB2_IndexSpaceI.H Src/EB/AMReX_EBFabFactory.cpp Src/Extern/HYPRE/AMReX_Hypre.cpp Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/Extern/PETSc/AMReX_PETSc.cpp Src/Extern/amrdata/AMReX_AmrData.cpp Src/F_Interfaces/Octree/AMReX_octree_fi.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Src/LinearSolvers/Projections/AMReX_MacProjector.cpp Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp Src/Particle/AMReX_NeighborParticlesGPUImpl.H Src/Particle/AMReX_NeighborParticlesI.H Src/Particle/AMReX_ParticleContainerBase.cpp Src/Particle/AMReX_TracerParticles.cpp Tests/Amr/Advection_AmrCore/Source/AmrCoreAdv.cpp Tests/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tests/FillBoundaryComparison/main.cpp Tests/GPU/CNS/Source/CNS.cpp Tests/GPU/CNS/Source/CNS_io.cpp Tests/HDF5Benchmark/main.cpp Tests/LinearSolvers/CellEB/MyTest.cpp Tests/LinearSolvers/CellEB2/MyTest.cpp Tests/LinearSolvers/CellOverset/MyTest.cpp Tests/LinearSolvers/EBConvergenceTest/MyTest.cpp Tests/LinearSolvers/EBflux_grad/MyTest.cpp Tests/LinearSolvers/LeastSquares/initData.cpp Tests/LinearSolvers/MultiComponent/MCNodalLinOp.cpp Tests/LinearSolvers/NodeEB/MyTest.cpp Tests/LinearSolvers/TensorOverset/MyTest.cpp Tests/Particles/AssignMultiLevelDensity/main.cpp Tests/Particles/AsyncIO/main.cpp commit 57d41ef9cc3f0db4b5d644d0169311b4f9f68a92 Author: Weiqun Zhang Date: Thu Apr 1 08:37:00 2021 -0700 Update CHANGES for 21.04 (#1919) CHANGES commit 622fbf998d44b77f44d1c682b9b33e018402051f Author: Andrew Myers Date: Thu Apr 1 11:04:12 2021 -0400 Docs typo fix. (#1918) Docs/sphinx_documentation/source/LinearSolvers_Chapter.rst commit ecf7e9812760feff4cdb2c53deb3402b68383fc8 Author: asalmgren Date: Wed Mar 31 14:02:27 2021 -0700 Allow us to set the "normalization threshold" variable at run-time (#1917) through the NodalProjector class Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp commit b8559fe999715afd3056cb94371bf2399a461863 Author: Weiqun Zhang Date: Tue Mar 30 17:01:58 2021 -0700 Fix a bug in BaseFab::resize. (#1916) Previously if `BaseFab::resize`'s optional `Arena*` argument is `nullptr` (which is the default) while `BaseFab`'s `Arena*` member is a different `Arena`, the `nullptr` will be interpreted as the default `Arena`, `The_Arena`, ignoring the `BaseFab`'s `Arena` has already been set. This causes the following code to allocate data using `The_Arena`, not `The_Cpu_Arena`. BaseFab fab(The_Cpu_Arena()); fab.resize(box); This PR fixes the bug. Src/Base/AMReX_BaseFab.H commit 81982da08fb25cd266b3666d358d1cbf09bb4de5 Author: asalmgren Date: Tue Mar 30 12:47:49 2021 -0700 Change the default value of m_normalization_threshold from 1e-10 to 1e-8 (#1914) This is motivated by an EB nodal BiCG bottom solver failing to converge with 1e-10 while converging with 1e-8. Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H commit 92945ad3a3560031c43fe7f02b9cc252f8330708 Author: Andrew Myers Date: Tue Mar 30 12:34:53 2021 -0400 Dont zero out input multifab in amrex::ParticleToMesh (#1910) Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleMesh.H commit 78eae52af97d4c6b993feec9bf76f21b9ef7288a Author: asalmgren Date: Mon Mar 29 17:00:17 2021 -0700 Revert the change which zeroed out the center coefficient if it has the wrong (#1911) sign. This breaks more cases than it fixes. Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H commit 227e378901d904040340ad38971db5142f008d15 Author: vzendejasl <59891999+vzendejasl@users.noreply.github.com> Date: Mon Mar 29 13:56:25 2021 -0700 Slopes test (#1859) Slope computation on cell centroids using the Least Square method. These tests are based from the LeastSquares Test (#1707) but instead of calculating scalar values on face centroids they are calculated on cell centroids using a linear function rather than a second order polynomial (#1707). Tests/Slopes/GNUmakefile Tests/Slopes/Make.package Tests/Slopes/MyEB.H Tests/Slopes/MyTest.H Tests/Slopes/MyTest.cpp Tests/Slopes/README.md Tests/Slopes/initData.cpp Tests/Slopes/initEB.cpp Tests/Slopes/initLinearData.cpp Tests/Slopes/initLinearDataFor2D.cpp Tests/Slopes/initLinearDataFor3D.cpp Tests/Slopes/inputs Tests/Slopes/inputs.2d.askew Tests/Slopes/inputs.2d.base Tests/Slopes/inputs.2d.fullyrotated Tests/Slopes/inputs.3d.linear.aligned.xy-x Tests/Slopes/inputs.3d.linear.aligned.xy-y Tests/Slopes/inputs.3d.linear.aligned.xz-x Tests/Slopes/inputs.3d.linear.aligned.xz-z Tests/Slopes/inputs.3d.linear.aligned.yz-y Tests/Slopes/inputs.3d.linear.aligned.yz-z Tests/Slopes/inputs.3d.linear.askew-all Tests/Slopes/inputs.3d.linear.askew-xy Tests/Slopes/inputs.3d.linear.askew-xz Tests/Slopes/inputs.3d.linear.askew-yz Tests/Slopes/main.cpp commit 0ab63f3f93950b5a90fb091c6eabbf3239886fcd Author: Weiqun Zhang Date: Mon Mar 29 09:32:23 2021 -0700 Fix bug in nodal solver that uses harmonic averageing (#1902) This is a minor bug that does not affect the final result. The convergence rate might be slightly affected. Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H commit dc8f72702c1ed4bb4443916ded01d9daff4f811f Author: asalmgren Date: Fri Mar 26 12:40:02 2021 -0700 Ensure center coefficient in an EB nodal solve has correct sign (#1905) It is possible for the center coefficient in an EB nodal solve to have the incorrect sign after coarsening -- here we protect against that by setting s0 to the max of (s0,0) -- this enables the BiCG to converge. Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H commit 9f297b9bc3a233553925feb5c0e2db784f1ddf9e Author: mic84 Date: Fri Mar 26 12:00:44 2021 -0700 CMake: add test_install (#1894) * CMake: add test_install * CMake: forgot to add test dir * CMake: specify compiler to use when running test_install * Update Doc .github/workflows/linux.yml CMakeLists.txt Docs/sphinx_documentation/source/BuildingAMReX.rst Tests/CMakeTestInstall/CMakeLists.txt Tools/CMake/AMReXInstallHelpers.cmake commit 146fc573967dbb2328be15d8e499a479f218c4e8 Author: jmsexton03 Date: Thu Mar 25 16:16:33 2021 -0700 Update Make.CNS and similar files to reflect Tutorials to Tests move (#1903) Tests/Amr/Advection_AmrCore/README Tests/Amr/Advection_AmrCore/README.md Tests/Amr/Advection_AmrLevel/Exec/Make.Adv Tests/CNS/Exec/Make.CNS Tests/FortranInterface/Advection_F/Exec/Make.Adv Tests/FortranInterface/Advection_octree_F/Exec/Make.Adv Tests/GPU/CNS/Exec/Make.CNS Tests/LinearSolvers/MultiComponent/MCNodalLinOp.H Tests/LinearSolvers/MultiComponent/MCNodalLinOp.cpp Tests/LinearSolvers/MultiComponent/main.cpp commit f9216a341e51404e935733a4c953525a6f503d47 Author: Andrew Myers Date: Thu Mar 25 14:42:37 2021 -0400 Fix AMREX_EXPORT (#1901) Src/EB/AMReX_EB2.H commit e57c1e8294c7c74a5e3a71821639a4cc1e48d895 Author: Andrew Myers Date: Thu Mar 25 13:52:56 2021 -0400 reimplement increment and incrementwithtotal to take advantage of GPUs and OpenMP (#1899) Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_Particles.H commit 62b3d7e4cafbc5b2f6453782bd9f4500014c4b63 Author: Andrew Myers Date: Thu Mar 25 13:46:49 2021 -0400 Export more global variables for windows support (#1900) Src/EB/AMReX_EB2.cpp commit 87c81d0ca70e65c9209386c03b3c85ab204667bb Author: Axel Huebl Date: Wed Mar 24 16:48:05 2021 -0700 CI: HIPCC as C Compiler (#1897) Since `hipcc` is a C++ compiler, we should assign it with `-x c` if used as C compiler. Or... we just use the corresponding `clang` for C files. .github/workflows/linux.yml commit f2443e0bc4aa433baa2154cf9b3b7ca4a672bf65 Author: Andrew Myers Date: Wed Mar 24 18:09:52 2021 -0400 Return const& in rdata and idata and disallow returning copies for rvalues. (#1895) Src/Particle/AMReX_Particle.H commit d226b807024fedd11eec4d751367524f75c01178 Author: Axel Huebl Date: Wed Mar 24 14:36:25 2021 -0700 CI: HIP w/o MPI (#1896) HIP 4.1 introduces issues with C targets which break the MPI feature test in CMake's `FindMPI.cmake` for C. .github/workflows/linux.yml commit 7c3e95210a1877c67139ea57c62fbcc717acc1c3 Author: Andrew Myers Date: Mon Mar 22 18:13:14 2021 -0400 Remove unnecessary abort. (#1892) Src/EB/AMReX_distFcnElement.cpp commit f3ddeadbb53049034f817d8bb9a8c21951ff494a Author: Weiqun Zhang Date: Mon Mar 22 14:53:44 2021 -0700 Update ALaplacian for multi-component and reuse (#1891) Src/LinearSolvers/MLMG/AMReX_MLABecLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLALap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLALaplacian.H Src/LinearSolvers/MLMG/AMReX_MLALaplacian.cpp commit d1468003318845a83fb59692bb4f06be95eb319a Author: Andrew Myers Date: Mon Mar 22 13:28:37 2021 -0400 Move tutorials (#1876) .github/workflows/linux.yml .github/workflows/macos.yml .github/workflows/windows.yml CMakeLists.txt Docs/sphinx_documentation/source/BuildingAMReX.rst Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_NFiles.H Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleUtil.H Tests/Amr/Advection_AmrCore/CMakeLists.txt Tests/Amr/Advection_AmrCore/Exec/GNUmakefile Tests/Amr/Advection_AmrCore/Exec/GNUmakefile_movie Tests/Amr/Advection_AmrCore/Exec/Make.Adv Tests/Amr/Advection_AmrCore/Exec/Make.package Tests/Amr/Advection_AmrCore/Exec/Prob.H Tests/Amr/Advection_AmrCore/Exec/inputs Tests/Amr/Advection_AmrCore/Exec/inputs_for_scaling Tests/Amr/Advection_AmrCore/Exec/paraview_amr101.py Tests/Amr/Advection_AmrCore/README Tests/Amr/Advection_AmrCore/README.md Tests/Amr/Advection_AmrCore/Source/AdvancePhiAllLevels.cpp Tests/Amr/Advection_AmrCore/Source/AdvancePhiAtLevel.cpp Tests/Amr/Advection_AmrCore/Source/AmrCoreAdv.H Tests/Amr/Advection_AmrCore/Source/AmrCoreAdv.cpp Tests/Amr/Advection_AmrCore/Source/DefineVelocity.cpp Tests/Amr/Advection_AmrCore/Source/Kernels.H Tests/Amr/Advection_AmrCore/Source/Make.package Tests/Amr/Advection_AmrCore/Source/Src_K/Adv_K.H Tests/Amr/Advection_AmrCore/Source/Src_K/Make.package Tests/Amr/Advection_AmrCore/Source/Src_K/compute_flux_2D_K.H Tests/Amr/Advection_AmrCore/Source/Src_K/compute_flux_3D_K.H Tests/Amr/Advection_AmrCore/Source/Src_K/slope_K.H Tests/Amr/Advection_AmrCore/Source/Tagging.H Tests/Amr/Advection_AmrCore/Source/bc_fill.H Tests/Amr/Advection_AmrCore/Source/face_velocity.H Tests/Amr/Advection_AmrCore/Source/main.cpp Tests/Amr/Advection_AmrLevel/CMakeLists.txt Tests/Amr/Advection_AmrLevel/Exec/Make.Adv Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/GNUmakefile Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/Make.package Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/Prob.f90 Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/face_velocity_2d.f90 Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/face_velocity_3d.f90 Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/inputs Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/inputs.tracers Tests/Amr/Advection_AmrLevel/Exec/SingleVortex/probin Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/GNUmakefile Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/Make.package Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/Prob.f90 Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/face_velocity_2d.f90 Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/face_velocity_3d.f90 Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/inputs Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/inputs.regt Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/probdata.f90 Tests/Amr/Advection_AmrLevel/Exec/UniformVelocity/probin Tests/Amr/Advection_AmrLevel/README Tests/Amr/Advection_AmrLevel/Source/Adv_F.H Tests/Amr/Advection_AmrLevel/Source/AmrLevelAdv.H Tests/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tests/Amr/Advection_AmrLevel/Source/LevelBldAdv.cpp Tests/Amr/Advection_AmrLevel/Source/Make.package Tests/Amr/Advection_AmrLevel/Source/Src_2d/Adv_2d.f90 Tests/Amr/Advection_AmrLevel/Source/Src_2d/Make.package Tests/Amr/Advection_AmrLevel/Source/Src_2d/compute_flux_2d.f90 Tests/Amr/Advection_AmrLevel/Source/Src_2d/slope_2d.f90 Tests/Amr/Advection_AmrLevel/Source/Src_3d/Adv_3d.f90 Tests/Amr/Advection_AmrLevel/Source/Src_3d/Make.package Tests/Amr/Advection_AmrLevel/Source/Src_3d/compute_flux_3d.f90 Tests/Amr/Advection_AmrLevel/Source/Src_3d/slope_3d.f90 Tests/Amr/Advection_AmrLevel/Source/Src_nd/Adv_nd.f90 Tests/Amr/Advection_AmrLevel/Source/Src_nd/Make.package Tests/Amr/Advection_AmrLevel/Source/Src_nd/Tagging_nd.f90 Tests/Amr/Advection_AmrLevel/Source/Src_nd/tagging_params.f90 Tests/Amr/Advection_AmrLevel/Source/main.cpp Tests/CMakeLists.txt Tests/CNS/CMakeLists.txt Tests/CNS/Exec/Combustor/GNUmakefile Tests/CNS/Exec/Combustor/Make.package Tests/CNS/Exec/Combustor/bc_fill_nd.F90 Tests/CNS/Exec/Combustor/bc_fill_nd.F90_jbb Tests/CNS/Exec/Combustor/cns_prob.F90 Tests/CNS/Exec/Combustor/cns_prob.F90_jbb Tests/CNS/Exec/Combustor/inputs Tests/CNS/Exec/Combustor/inputs.regt Tests/CNS/Exec/Make.CNS Tests/CNS/Exec/Pulse/GNUmakefile Tests/CNS/Exec/Pulse/Make.package Tests/CNS/Exec/Pulse/cns_prob.F90 Tests/CNS/Exec/Pulse/inputs Tests/CNS/Exec/Pulse/inputs.regt Tests/CNS/Exec/ShockRef/GNUmakefile Tests/CNS/Exec/ShockRef/Make.package Tests/CNS/Exec/ShockRef/cns_prob.F90 Tests/CNS/Exec/ShockRef/inputs Tests/CNS/Exec/ShockRef/inputs.amr Tests/CNS/Exec/ShockRef/inputs.regt Tests/CNS/Exec/Sod/GNUmakefile Tests/CNS/Exec/Sod/Make.package Tests/CNS/Exec/Sod/cns_prob.F90 Tests/CNS/Exec/Sod/inputs Tests/CNS/Source/CNS.H Tests/CNS/Source/CNS.cpp Tests/CNS/Source/CNSBld.cpp Tests/CNS/Source/CNS_F.H Tests/CNS/Source/CNS_advance.cpp Tests/CNS/Source/CNS_init_eb2.cpp Tests/CNS/Source/CNS_io.cpp Tests/CNS/Source/CNS_setup.cpp Tests/CNS/Source/Make.package Tests/CNS/Source/diffusion/Make.package Tests/CNS/Source/diffusion/cns_diff_mod.F90 Tests/CNS/Source/diffusion/cns_eb_diff_mod.F90 Tests/CNS/Source/diffusion/cns_eb_diff_wall.F90 Tests/CNS/Source/diffusion/diff_coef_mod.F90 Tests/CNS/Source/fortran/CNS_derive.F90 Tests/CNS/Source/fortran/CNS_divop.F90 Tests/CNS/Source/fortran/CNS_dudt.F90 Tests/CNS/Source/fortran/CNS_f.F90 Tests/CNS/Source/fortran/CNS_nd.F90 Tests/CNS/Source/fortran/CNS_physics.F90 Tests/CNS/Source/fortran/CNS_tagging.F90 Tests/CNS/Source/fortran/Make.package Tests/CNS/Source/fortran/bc_fill_nd.F90 Tests/CNS/Source/hydro/Hyp_gamma_MOL.F90 Tests/CNS/Source/hydro/Hyp_gamma_MOL_EB.F90 Tests/CNS/Source/hydro/Make.package Tests/CNS/Source/hydro/analriem3d.F90 Tests/CNS/Source/hydro/cns_eb_hyp_wall.F90 Tests/CNS/Source/hydro/slope_mol_3d_gamma.F90 Tests/CNS/Source/hydro/slope_mol_3d_gamma_EB.F90 Tests/CNS/Source/main.cpp Tests/FortranInterface/Advection_F/CMakeLists.txt Tests/FortranInterface/Advection_F/Exec/Make.Adv Tests/FortranInterface/Advection_F/Exec/SingleVortex/GNUmakefile Tests/FortranInterface/Advection_F/Exec/SingleVortex/Make.package Tests/FortranInterface/Advection_F/Exec/SingleVortex/Prob_2d.f90 Tests/FortranInterface/Advection_F/Exec/SingleVortex/Prob_3d.f90 Tests/FortranInterface/Advection_F/Exec/SingleVortex/face_velocity_2d.F90 Tests/FortranInterface/Advection_F/Exec/SingleVortex/face_velocity_3d.F90 Tests/FortranInterface/Advection_F/Exec/SingleVortex/inputs Tests/FortranInterface/Advection_F/Exec/SingleVortex/inputs.physbc Tests/FortranInterface/Advection_F/Exec/SingleVortex/inputs.rt Tests/FortranInterface/Advection_F/README Tests/FortranInterface/Advection_F/Source/Make.package Tests/FortranInterface/Advection_F/Source/Src_2d/Make.package Tests/FortranInterface/Advection_F/Source/Src_2d/advect_2d_mod.F90 Tests/FortranInterface/Advection_F/Source/Src_2d/compute_flux_2d.f90 Tests/FortranInterface/Advection_F/Source/Src_2d/slope_2d.f90 Tests/FortranInterface/Advection_F/Source/Src_3d/Make.package Tests/FortranInterface/Advection_F/Source/Src_3d/advect_3d_mod.F90 Tests/FortranInterface/Advection_F/Source/Src_3d/compute_flux_3d.f90 Tests/FortranInterface/Advection_F/Source/Src_3d/slope_3d.f90 Tests/FortranInterface/Advection_F/Source/amr_data_mod.F90 Tests/FortranInterface/Advection_F/Source/averagedown_mod.F90 Tests/FortranInterface/Advection_F/Source/bc_mod.F90 Tests/FortranInterface/Advection_F/Source/compute_dt_mod.F90 Tests/FortranInterface/Advection_F/Source/evolve_mod.F90 Tests/FortranInterface/Advection_F/Source/fillpatch_mod.F90 Tests/FortranInterface/Advection_F/Source/fmain.F90 Tests/FortranInterface/Advection_F/Source/initdata.F90 Tests/FortranInterface/Advection_F/Source/my_amr_mod.F90 Tests/FortranInterface/Advection_F/Source/plotfile_mod.F90 Tests/FortranInterface/Advection_F/Source/tagging_mod.F90 Tests/FortranInterface/Advection_octree_F/CMakeLists.txt Tests/FortranInterface/Advection_octree_F/Exec/Make.Adv Tests/FortranInterface/Advection_octree_F/Exec/SingleVortex/GNUmakefile Tests/FortranInterface/Advection_octree_F/Exec/SingleVortex/Make.package Tests/FortranInterface/Advection_octree_F/Exec/SingleVortex/Prob.f90 Tests/FortranInterface/Advection_octree_F/Exec/SingleVortex/face_velocity_2d.F90 Tests/FortranInterface/Advection_octree_F/Exec/SingleVortex/inputs Tests/FortranInterface/Advection_octree_F/Exec/SingleVortex/inputs.rt Tests/FortranInterface/Advection_octree_F/README Tests/FortranInterface/Advection_octree_F/Source/Make.package Tests/FortranInterface/Advection_octree_F/Source/Src_2d/Make.package Tests/FortranInterface/Advection_octree_F/Source/Src_2d/advect_2d_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/Src_2d/compute_flux_2d.f90 Tests/FortranInterface/Advection_octree_F/Source/Src_2d/slope_2d.f90 Tests/FortranInterface/Advection_octree_F/Source/amr_data_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/averagedown_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/bc_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/compute_dt_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/evolve_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/fillpatch_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/fmain.F90 Tests/FortranInterface/Advection_octree_F/Source/initdata.F90 Tests/FortranInterface/Advection_octree_F/Source/my_amr_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/plotfile_mod.F90 Tests/FortranInterface/Advection_octree_F/Source/tagging_mod.F90 Tests/GPU/CNS/CMakeLists.txt Tests/GPU/CNS/Exec/Make.CNS Tests/GPU/CNS/Exec/RT/GNUmakefile Tests/GPU/CNS/Exec/RT/Make.package Tests/GPU/CNS/Exec/RT/cns_prob.H Tests/GPU/CNS/Exec/RT/cns_prob.cpp Tests/GPU/CNS/Exec/RT/cns_prob_parm.H Tests/GPU/CNS/Exec/RT/inputs Tests/GPU/CNS/Exec/RT/inputs-rt Tests/GPU/CNS/Exec/Sod/GNUmakefile Tests/GPU/CNS/Exec/Sod/Make.package Tests/GPU/CNS/Exec/Sod/cns_prob.H Tests/GPU/CNS/Exec/Sod/cns_prob.cpp Tests/GPU/CNS/Exec/Sod/cns_prob_parm.H Tests/GPU/CNS/Exec/Sod/inputs Tests/GPU/CNS/Exec/Sod/inputs-rt Tests/GPU/CNS/Source/CNS.H Tests/GPU/CNS/Source/CNS.cpp Tests/GPU/CNS/Source/CNSBld.cpp Tests/GPU/CNS/Source/CNS_K.H Tests/GPU/CNS/Source/CNS_advance.cpp Tests/GPU/CNS/Source/CNS_bcfill.cpp Tests/GPU/CNS/Source/CNS_derive.H Tests/GPU/CNS/Source/CNS_derive.cpp Tests/GPU/CNS/Source/CNS_index_macros.H Tests/GPU/CNS/Source/CNS_io.cpp Tests/GPU/CNS/Source/CNS_parm.H Tests/GPU/CNS/Source/CNS_parm.cpp Tests/GPU/CNS/Source/CNS_setup.cpp Tests/GPU/CNS/Source/CNS_tagging.H Tests/GPU/CNS/Source/Make.package Tests/GPU/CNS/Source/diffusion/Make.package Tests/GPU/CNS/Source/hydro/CNS_hydro_K.H Tests/GPU/CNS/Source/hydro/Make.package Tests/GPU/CNS/Source/main.cpp Tests/GPU/Vector/inputs Tests/HDF5Benchmark/inputs Tests/LinearSolvers/ABecLaplacian_C/CMakeLists.txt Tests/LinearSolvers/ABecLaplacian_F/CMakeLists.txt Tests/LinearSolvers/MAC_Projection_EB/CMakeLists.txt Tests/LinearSolvers/NodalPoisson/CMakeLists.txt Tests/LinearSolvers/Nodal_Projection_EB/CMakeLists.txt Tests/LinearSolvers/NodeTensorLap/CMakeLists.txt Tests/MultiBlock/IndexType/main.cpp Tests/Particles/ParallelContext/CMakeLists.txt Tests/Particles/ParallelContext/inputs.rt Tests/Particles/ParallelContext/main.cpp Tests/Particles/Redistribute/inputs Tests/Particles/Redistribute/inputs.rt Tools/RegressionTesting/AMReX-cuda-tests.ini Tools/RegressionTesting/AMReX-dpcpp-tests.ini Tools/RegressionTesting/AMReX-hip-tests.ini Tools/RegressionTesting/AMReX-tests.ini Tutorials/Basic/Build_with_libamrex/GNUmakefile Tutorials/Basic/Build_with_libamrex/MyParams.H Tutorials/Basic/Build_with_libamrex/main.cpp Tutorials/Basic/Build_with_libamrex/my_func.f90 Tutorials/Basic/Build_with_libamrex/test_parameters.cpp Tutorials/Basic/HeatEquation_EX1_C/CMakeLists.txt Tutorials/Basic/HeatEquation_EX1_C/Exec/GNUmakefile Tutorials/Basic/HeatEquation_EX1_C/Exec/inputs Tutorials/Basic/HeatEquation_EX1_C/Source/Make.package Tutorials/Basic/HeatEquation_EX1_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX1_C/Source/myfunc.H Tutorials/Basic/HeatEquation_EX1_C/Source/myfunc.cpp Tutorials/Basic/HeatEquation_EX1_C/Source/mykernel.H Tutorials/Basic/HeatEquation_EX1_CF/CMakeLists.txt Tutorials/Basic/HeatEquation_EX1_CF/Exec/GNUmakefile Tutorials/Basic/HeatEquation_EX1_CF/Exec/inputs_2d Tutorials/Basic/HeatEquation_EX1_CF/Exec/inputs_3d Tutorials/Basic/HeatEquation_EX1_CF/Source/Make.package Tutorials/Basic/HeatEquation_EX1_CF/Source/advance.cpp Tutorials/Basic/HeatEquation_EX1_CF/Source/advance_2d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/advance_3d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/init_phi_2d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/init_phi_3d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/main.cpp Tutorials/Basic/HeatEquation_EX1_CF/Source/myfunc.H Tutorials/Basic/HeatEquation_EX1_CF/Source/myfunc_F.H Tutorials/Basic/HeatEquation_EX1_F/CMakeLists.txt Tutorials/Basic/HeatEquation_EX1_F/GNUmakefile Tutorials/Basic/HeatEquation_EX1_F/Make.package Tutorials/Basic/HeatEquation_EX1_F/advance.f90 Tutorials/Basic/HeatEquation_EX1_F/fmain.F90 Tutorials/Basic/HeatEquation_EX1_F/init_phi.f90 Tutorials/Basic/HeatEquation_EX1_F/inputs Tutorials/Basic/HeatEquation_EX2_C/CMakeLists.txt Tutorials/Basic/HeatEquation_EX2_C/Exec/GNUmakefile Tutorials/Basic/HeatEquation_EX2_C/Exec/inputs Tutorials/Basic/HeatEquation_EX2_C/Source/Make.package Tutorials/Basic/HeatEquation_EX2_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX2_C/Source/myfunc.H Tutorials/Basic/HeatEquation_EX2_C/Source/myfunc.cpp Tutorials/Basic/HeatEquation_EX2_C/Source/mykernel.H Tutorials/Basic/HeatEquation_EX2_CF/CMakeLists.txt Tutorials/Basic/HeatEquation_EX2_CF/Exec/GNUmakefile Tutorials/Basic/HeatEquation_EX2_CF/Exec/inputs_2d Tutorials/Basic/HeatEquation_EX2_CF/Exec/inputs_3d Tutorials/Basic/HeatEquation_EX2_CF/Source/Make.package Tutorials/Basic/HeatEquation_EX2_CF/Source/advance.cpp Tutorials/Basic/HeatEquation_EX2_CF/Source/advance_2d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/advance_3d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/init_phi_2d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/init_phi_3d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/main.cpp Tutorials/Basic/HeatEquation_EX2_CF/Source/myfunc.H Tutorials/Basic/HeatEquation_EX2_CF/Source/myfunc_F.H Tutorials/Basic/HeatEquation_EX3_C/CMakeLists.txt Tutorials/Basic/HeatEquation_EX3_C/Exec/GNUmakefile Tutorials/Basic/HeatEquation_EX3_C/Exec/inputs_2d Tutorials/Basic/HeatEquation_EX3_C/Exec/inputs_3d Tutorials/Basic/HeatEquation_EX3_C/Source/Make.package Tutorials/Basic/HeatEquation_EX3_C/Source/advance.cpp Tutorials/Basic/HeatEquation_EX3_C/Source/init_phi_2d.f90 Tutorials/Basic/HeatEquation_EX3_C/Source/init_phi_3d.f90 Tutorials/Basic/HeatEquation_EX3_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX3_C/Source/myfunc.H Tutorials/Basic/HeatEquation_EX3_C/Source/myfunc_F.H Tutorials/Basic/HelloWorld_C/CMakeLists.txt Tutorials/Basic/HelloWorld_C/GNUmakefile Tutorials/Basic/HelloWorld_C/Make.package Tutorials/Basic/HelloWorld_C/main.cpp Tutorials/Basic/HelloWorld_F/CMakeLists.txt Tutorials/Basic/HelloWorld_F/GNUmakefile Tutorials/Basic/HelloWorld_F/Make.package Tutorials/Basic/HelloWorld_F/fmain.f90 Tutorials/Basic/PrefixSum_MultiFab/CMakeLists.txt Tutorials/Basic/PrefixSum_MultiFab/GNUmakefile Tutorials/Basic/PrefixSum_MultiFab/Make.package Tutorials/Basic/PrefixSum_MultiFab/Parallel-Prefix-Sum.ipynb Tutorials/Basic/PrefixSum_MultiFab/README.md Tutorials/Basic/PrefixSum_MultiFab/inputs Tutorials/Basic/PrefixSum_MultiFab/main.cpp Tutorials/Basic/main_C/CMakeLists.txt Tutorials/Basic/main_C/GNUmakefile Tutorials/Basic/main_C/Make.package Tutorials/Basic/main_C/main.cpp Tutorials/Basic/main_F/CMakeLists.txt Tutorials/Basic/main_F/GNUmakefile Tutorials/Basic/main_F/Make.package Tutorials/Basic/main_F/main.F90 Tutorials/Blueprint/AssignMultiLevelDensity/CMakeLists.txt Tutorials/Blueprint/AssignMultiLevelDensity/GNUmakefile Tutorials/Blueprint/AssignMultiLevelDensity/Make.package Tutorials/Blueprint/AssignMultiLevelDensity/inputs Tutorials/Blueprint/AssignMultiLevelDensity/main.cpp Tutorials/Blueprint/CellSortedParticles/CMakeLists.txt Tutorials/Blueprint/CellSortedParticles/CellSortedPC.H Tutorials/Blueprint/CellSortedParticles/CellSortedPC.cpp Tutorials/Blueprint/CellSortedParticles/GNUmakefile Tutorials/Blueprint/CellSortedParticles/Make.package Tutorials/Blueprint/CellSortedParticles/cell_sorted_3d.F90 Tutorials/Blueprint/CellSortedParticles/cell_sorted_F.H Tutorials/Blueprint/CellSortedParticles/inputs Tutorials/Blueprint/CellSortedParticles/main.cpp Tutorials/Blueprint/HeatEquation_EX1_C/CMakeLists.txt Tutorials/Blueprint/HeatEquation_EX1_C/Exec/GNUmakefile Tutorials/Blueprint/HeatEquation_EX1_C/Exec/inputs_2d Tutorials/Blueprint/HeatEquation_EX1_C/Exec/inputs_3d Tutorials/Blueprint/HeatEquation_EX1_C/Source/Make.package Tutorials/Blueprint/HeatEquation_EX1_C/Source/advance.cpp Tutorials/Blueprint/HeatEquation_EX1_C/Source/advance_2d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/advance_3d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/init_phi_2d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/init_phi_3d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/main.cpp Tutorials/Blueprint/HeatEquation_EX1_C/Source/myfunc.H Tutorials/Blueprint/HeatEquation_EX1_C/Source/myfunc_F.H Tutorials/Blueprint/README.txt Tutorials/CMakeLists.txt Tutorials/EB/GeometryGeneration/CMakeLists.txt Tutorials/EB/GeometryGeneration/GNUmakefile Tutorials/EB/GeometryGeneration/Make.package Tutorials/EB/GeometryGeneration/main.cpp Tutorials/EB/MacProj/CMakeLists.txt Tutorials/EB/MacProj/GNUmakefile Tutorials/EB/MacProj/Make.package Tutorials/EB/MacProj/inputs Tutorials/EB/MacProj/main.cpp Tutorials/EB/Poisson/CMakeLists.txt Tutorials/EB/Poisson/GNUmakefile Tutorials/EB/Poisson/Make.package Tutorials/EB/Poisson/Poisson.H Tutorials/EB/Poisson/Poisson.cpp Tutorials/EB/Poisson/inputs Tutorials/EB/Poisson/main.cpp Tutorials/EB/STLtest/GNUmakefile Tutorials/EB/STLtest/Make.package Tutorials/EB/STLtest/airfoil.stl Tutorials/EB/STLtest/inputs Tutorials/EB/STLtest/main.cpp Tutorials/ForkJoin/MLMG/CMakeLists.txt Tutorials/ForkJoin/MLMG/GNUmakefile Tutorials/ForkJoin/MLMG/Make.package Tutorials/ForkJoin/MLMG/ff.f90 Tutorials/ForkJoin/MLMG/inputs Tutorials/ForkJoin/MLMG/main.cpp Tutorials/ForkJoin/Simple/CMakeLists.txt Tutorials/ForkJoin/Simple/GNUmakefile Tutorials/ForkJoin/Simple/Make.package Tutorials/ForkJoin/Simple/MyTest.H Tutorials/ForkJoin/Simple/MyTest.cpp Tutorials/ForkJoin/Simple/MyTest_F.H Tutorials/ForkJoin/Simple/inputs Tutorials/ForkJoin/Simple/main.cpp Tutorials/FortranInterface/Advection_octree_F2/CMakeLists.txt Tutorials/FortranInterface/Advection_octree_F2/Exec/Make.Adv Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/GNUmakefile Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/Make.package Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/Prob.f90 Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/face_velocity_2d.F90 Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/inputs Tutorials/FortranInterface/Advection_octree_F2/README Tutorials/FortranInterface/Advection_octree_F2/Source/Make.package Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/Make.package Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/advect_2d_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/compute_flux_2d.f90 Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/slope_2d.f90 Tutorials/FortranInterface/Advection_octree_F2/Source/amr_data_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/averagedown_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/bc_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/compute_dt_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/evolve_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/fillpatch_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/fmain.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/initdata.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/my_amr_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/plotfile_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/tagging_mod.F90 Tutorials/GPU/EBCNS/CMakeLists.txt Tutorials/GPU/EBCNS/Exec/Make.CNS Tutorials/GPU/EBCNS/Exec/Sod/GNUmakefile Tutorials/GPU/EBCNS/Exec/Sod/Make.package Tutorials/GPU/EBCNS/Exec/Sod/cns_prob.H Tutorials/GPU/EBCNS/Exec/Sod/cns_prob.cpp Tutorials/GPU/EBCNS/Exec/Sod/cns_prob_parm.H Tutorials/GPU/EBCNS/Exec/Sod/inputs Tutorials/GPU/EBCNS/README Tutorials/GPU/EBCNS/Source/CNS.H Tutorials/GPU/EBCNS/Source/CNS.cpp Tutorials/GPU/EBCNS/Source/CNSBld.cpp Tutorials/GPU/EBCNS/Source/CNS_K.H Tutorials/GPU/EBCNS/Source/CNS_advance.cpp Tutorials/GPU/EBCNS/Source/CNS_bcfill.cpp Tutorials/GPU/EBCNS/Source/CNS_derive.H Tutorials/GPU/EBCNS/Source/CNS_derive.cpp Tutorials/GPU/EBCNS/Source/CNS_index_macros.H Tutorials/GPU/EBCNS/Source/CNS_init_eb2.cpp Tutorials/GPU/EBCNS/Source/CNS_io.cpp Tutorials/GPU/EBCNS/Source/CNS_parm.H Tutorials/GPU/EBCNS/Source/CNS_parm.cpp Tutorials/GPU/EBCNS/Source/CNS_setup.cpp Tutorials/GPU/EBCNS/Source/CNS_tagging.H Tutorials/GPU/EBCNS/Source/Make.package Tutorials/GPU/EBCNS/Source/diffusion/Make.package Tutorials/GPU/EBCNS/Source/hydro/CNS_hydro_K.H Tutorials/GPU/EBCNS/Source/hydro/Make.package Tutorials/GPU/EBCNS/Source/main.cpp Tutorials/GPU/Launch/CMakeLists.txt Tutorials/GPU/Launch/GNUmakefile Tutorials/GPU/Launch/Make.package Tutorials/GPU/Launch/MyKernel_F.F90 Tutorials/GPU/Launch/MyKernel_F.H Tutorials/GPU/Launch/Readme.md Tutorials/GPU/Launch/main.cpp Tutorials/GPU/ParallelReduce/CMakeLists.txt Tutorials/GPU/ParallelReduce/GNUmakefile Tutorials/GPU/ParallelReduce/Make.package Tutorials/GPU/ParallelReduce/main.cpp Tutorials/GPU/ParallelScan/CMakeLists.txt Tutorials/GPU/ParallelScan/GNUmakefile Tutorials/GPU/ParallelScan/Make.package Tutorials/GPU/ParallelScan/main.cpp Tutorials/GPU/run.corigpu Tutorials/GPU/run.summit Tutorials/MUI/Exec_01/GNUmakefile Tutorials/MUI/Exec_02/GNUmakefile Tutorials/MUI/Exec_coupled/cmd_mpirun Tutorials/MUI/Exec_coupled/inputs Tutorials/MUI/Source_01/Make.package Tutorials/MUI/Source_01/init_phi_3d.f90 Tutorials/MUI/Source_01/main_01.cpp Tutorials/MUI/Source_01/myfunc.H Tutorials/MUI/Source_01/myfunc_F.H Tutorials/MUI/Source_02/Make.package Tutorials/MUI/Source_02/main_02.cpp Tutorials/MUI/Source_02/myfunc.H Tutorials/MUI/doc/GNUmakefile Tutorials/MUI/doc/MUIcouplingNotes.tex Tutorials/MUI/doc/iface_rect.png Tutorials/MUI/doc/vis_interface.m Tutorials/Particles/CellSortedParticles/CMakeLists.txt Tutorials/Particles/CellSortedParticles/CellSortedPC.H Tutorials/Particles/CellSortedParticles/CellSortedPC.cpp Tutorials/Particles/CellSortedParticles/GNUmakefile Tutorials/Particles/CellSortedParticles/Make.package Tutorials/Particles/CellSortedParticles/cell_sorted_3d.F90 Tutorials/Particles/CellSortedParticles/cell_sorted_F.H Tutorials/Particles/CellSortedParticles/inputs Tutorials/Particles/CellSortedParticles/main.cpp Tutorials/Particles/ElectromagneticPIC/CMakeLists.txt Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/EMParticleContainer.cpp Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/Evolve.cpp Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/GNUmakefile Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/Make.package Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/em_pic_K.H Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/inputs Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/EMParticleContainer.cpp Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/Evolve.cpp Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/GNUmakefile Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/Make.package Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/em_pic_3d.F90 Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/em_pic_F.H Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/inputs Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/script.sh Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/EMParticleContainer.cpp Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/Evolve.cpp Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/GNUmakefile Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/GNUmakefile.libamrex Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/Make.package Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/em_pic_3d.F90 Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/em_pic_F.H Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/inputs Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/run.summitdev Tutorials/Particles/ElectromagneticPIC/Make.EMPIC Tutorials/Particles/ElectromagneticPIC/Source/Constants.H Tutorials/Particles/ElectromagneticPIC/Source/EMParticleContainer.H Tutorials/Particles/ElectromagneticPIC/Source/EMParticleContainerInit.cpp Tutorials/Particles/ElectromagneticPIC/Source/Evolve.H Tutorials/Particles/ElectromagneticPIC/Source/IO.H Tutorials/Particles/ElectromagneticPIC/Source/IO.cpp Tutorials/Particles/ElectromagneticPIC/Source/Make.package Tutorials/Particles/ElectromagneticPIC/Source/NodalFlags.H Tutorials/Particles/ElectromagneticPIC/Source/NodalFlags.cpp Tutorials/Particles/ElectromagneticPIC/Source/main.cpp Tutorials/Particles/ElectrostaticPIC/CMakeLists.txt Tutorials/Particles/ElectrostaticPIC/ElectrostaticParticleContainer.H Tutorials/Particles/ElectrostaticPIC/ElectrostaticParticleContainer.cpp Tutorials/Particles/ElectrostaticPIC/GNUmakefile Tutorials/Particles/ElectrostaticPIC/Make.package Tutorials/Particles/ElectrostaticPIC/PhysConst.H Tutorials/Particles/ElectrostaticPIC/electrostatic_pic_2d.f90 Tutorials/Particles/ElectrostaticPIC/electrostatic_pic_3d.f90 Tutorials/Particles/ElectrostaticPIC/electrostatic_pic_F.H Tutorials/Particles/ElectrostaticPIC/inputs Tutorials/Particles/ElectrostaticPIC/main.cpp Tutorials/Particles/NeighborList/CMakeLists.txt Tutorials/Particles/NeighborList/CheckPair.H Tutorials/Particles/NeighborList/Constants.H Tutorials/Particles/NeighborList/GNUmakefile Tutorials/Particles/NeighborList/MDParticleContainer.H Tutorials/Particles/NeighborList/MDParticleContainer.cpp Tutorials/Particles/NeighborList/Make.package Tutorials/Particles/NeighborList/README.md Tutorials/Particles/NeighborList/inputs Tutorials/Particles/NeighborList/main.cpp Tutorials/Particles/NeighborList/script.sh Tutorials/README.md Tutorials/SDC/MISDC_ADR_2d/Exec/GNUmakefile Tutorials/SDC/MISDC_ADR_2d/Exec/inputs_2d Tutorials/SDC/MISDC_ADR_2d/README Tutorials/SDC/MISDC_ADR_2d/Source/Make.package Tutorials/SDC/MISDC_ADR_2d/Source/SDC_sweeper.cpp Tutorials/SDC/MISDC_ADR_2d/Source/functions_2d.f90 Tutorials/SDC/MISDC_ADR_2d/Source/init_phi_2d.f90 Tutorials/SDC/MISDC_ADR_2d/Source/main.cpp Tutorials/SDC/MISDC_ADR_2d/Source/myfunc.H Tutorials/SDC/MISDC_ADR_2d/Source/myfunc_F.H Tutorials/SWFFT/SWFFT_poisson/GNUmakefile Tutorials/SWFFT/SWFFT_poisson/Make.package Tutorials/SWFFT/SWFFT_poisson/README Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test.H Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test.cpp Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test_F.F90 Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test_F.H Tutorials/SWFFT/SWFFT_poisson/inputs.128 Tutorials/SWFFT/SWFFT_poisson/inputs.32 Tutorials/SWFFT/SWFFT_poisson/inputs.64 Tutorials/SWFFT/SWFFT_poisson/main.cpp Tutorials/SWFFT/SWFFT_poisson/run_me Tutorials/SWFFT/SWFFT_poisson/swfft_solver.cpp Tutorials/SWFFT/SWFFT_simple/GNUmakefile Tutorials/SWFFT/SWFFT_simple/Make.package Tutorials/SWFFT/SWFFT_simple/README Tutorials/SWFFT/SWFFT_simple/SWFFT_Test.H Tutorials/SWFFT/SWFFT_simple/SWFFT_Test.cpp Tutorials/SWFFT/SWFFT_simple/SWFFT_Test_F.F90 Tutorials/SWFFT/SWFFT_simple/SWFFT_Test_F.H Tutorials/SWFFT/SWFFT_simple/inputs.multipleGrids Tutorials/SWFFT/SWFFT_simple/inputs.oneGrid Tutorials/SWFFT/SWFFT_simple/main.cpp Tutorials/SWFFT/SWFFT_simple/run_me_2d Tutorials/SWFFT/SWFFT_simple/run_me_3d Tutorials/SWFFT/SWFFT_simple/swfft_compute.cpp commit 384ca2341889a8f1db78e87cb4b1735bd2d58554 Author: Axel Huebl Date: Mon Mar 22 10:09:53 2021 -0700 CMake: IPO/LTO (#1890) Add optional interprocedural optimization (IPO) aka link-time optimization (LTO) to AMReX. In my WarpX tests recently on x86_64 and ppc64le, using those options incurred a 2% performance hit (although one would expect a slight increase). Thus, they are not enabled by default. Also, enabling them with some toolchains needs users to set the linker properly. Nontheless, enabling IPO shrinks the binary size a lot. This is very valuable and worth the runtime penalty for our binary deployments, e.g. when shipping generic x86_64, aarch64 and ppc64le binaries on conda-forge to users. WarpX binary size (default build): * no IPO: 217MiB * WarpX IPO, AMReX no IPO: 155MiB * both WarpX and AMReX with IPO: 127M Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/CMake/AMReXBuildInfo.cmake Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXTypecheck.cmake Tools/CMake/AMReX_Config.cmake commit ef0eb9f5c4b065b38e3a3242a43263aba43aa874 Author: Weiqun Zhang Date: Sun Mar 21 14:35:07 2021 -0700 Header file order and iosfwd (#1883) Include amrex headers before standard C++ headers. Use iosfwd instead of iostream if we can. Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp Src/Amr/AMReX_AmrLevel.cpp Src/Amr/AMReX_Derive.H Src/Amr/AMReX_Derive.cpp Src/Amr/AMReX_StateData.H Src/Amr/AMReX_StateData.cpp Src/Amr/AMReX_StateDescriptor.H Src/Amr/AMReX_StateDescriptor.cpp Src/AmrCore/AMReX_AmrCore.H Src/AmrCore/AMReX_AmrCore.cpp Src/AmrCore/AMReX_Cluster.H Src/AmrCore/AMReX_Cluster.cpp Src/AmrCore/AMReX_ErrorList.H Src/AmrCore/AMReX_ErrorList.cpp Src/AmrCore/AMReX_FillPatchUtil.H Src/AmrCore/AMReX_Interpolater.cpp Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX.H Src/Base/AMReX_Array.H Src/Base/AMReX_Array4.H Src/Base/AMReX_BCRec.cpp Src/Base/AMReX_BLBackTrace.H Src/Base/AMReX_BLProfiler.H Src/Base/AMReX_BLProfiler.cpp Src/Base/AMReX_BaseFab.cpp Src/Base/AMReX_BaseUmap.H Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp Src/Base/AMReX_BoxDomain.H Src/Base/AMReX_BoxDomain.cpp Src/Base/AMReX_BoxIterator.H Src/Base/AMReX_BoxIterator.cpp Src/Base/AMReX_BoxList.H Src/Base/AMReX_BoxList.cpp Src/Base/AMReX_CoordSys.H Src/Base/AMReX_CoordSys.cpp Src/Base/AMReX_Dim3.H Src/Base/AMReX_Dim3.cpp Src/Base/AMReX_DistributionMapping.H Src/Base/AMReX_FACopyDescriptor.H Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_FabConv.H Src/Base/AMReX_Geometry.H Src/Base/AMReX_Geometry.cpp Src/Base/AMReX_GpuAllocators.H Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuComplex.H Src/Base/AMReX_GpuContainers.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuUtility.H Src/Base/AMReX_IndexType.H Src/Base/AMReX_IndexType.cpp Src/Base/AMReX_IntConv.H Src/Base/AMReX_IntVect.cpp Src/Base/AMReX_MFIter.H Src/Base/AMReX_MemPool.cpp Src/Base/AMReX_MemProfiler.H Src/Base/AMReX_MemProfiler.cpp Src/Base/AMReX_MultiFab.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_NFiles.H Src/Base/AMReX_NFiles.cpp Src/Base/AMReX_Orientation.H Src/Base/AMReX_Orientation.cpp Src/Base/AMReX_ParallelContext.H Src/Base/AMReX_ParallelContext.cpp Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelDescriptor.cpp Src/Base/AMReX_Periodicity.cpp Src/Base/AMReX_PlotFileDataImpl.H Src/Base/AMReX_PlotFileDataImpl.cpp Src/Base/AMReX_PlotFileUtil.H Src/Base/AMReX_PlotFileUtil.cpp Src/Base/AMReX_Print.H Src/Base/AMReX_Random.cpp Src/Base/AMReX_RealVect.H Src/Base/AMReX_RealVect.cpp Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_TinyProfiler.cpp Src/Base/AMReX_Tuple.H Src/Base/AMReX_TypeTraits.H Src/Base/AMReX_Utility.H Src/Base/AMReX_Vector.H Src/Base/AMReX_VectorIO.H Src/Base/AMReX_VectorIO.cpp Src/Base/AMReX_VisMF.H Src/Base/AMReX_VisMF.cpp Src/Base/AMReX_iMultiFab.cpp Src/Base/AMReX_parmparse_fi.cpp Src/Base/AMReX_parstream.H Src/Base/AMReX_parstream.cpp Src/Base/CMakeLists.txt Src/Base/Make.package Src/Boundary/AMReX_Mask.cpp Src/EB/AMReX_EB2_IF_Base.H Src/EB/AMReX_EB2_IF_Spline.H Src/EB/AMReX_EB2_Level.H Src/EB/AMReX_EBCellFlag.H Src/EB/AMReX_EBCellFlag.cpp Src/EB/AMReX_EBToPVD.H Src/EB/AMReX_EB_STL_utils.H Src/EB/AMReX_distFcnElement.H Src/Extern/HYPRE/AMReX_Hypre.H Src/Extern/HYPRE/AMReX_HypreABecLap.H Src/Extern/HYPRE/AMReX_HypreABecLap.cpp Src/Extern/HYPRE/AMReX_HypreIJIface.H Src/Extern/HYPRE/AMReX_HypreNodeLap.H Src/F_Interfaces/AmrCore/AMReX_FAmrCore.H Src/Particle/AMReX_Particle.H Src/Particle/AMReX_ParticleMPIUtil.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_TracerParticle_mod_K.H Src/Particle/AMReX_TracerParticles.cpp commit c984473e021aabdd5f5d2091dcc57481e8637c7c Author: Weiqun Zhang Date: Sun Mar 21 11:38:21 2021 -0700 Remove outdated notes (#1889) Docs/Notes/DPCPPWishlist.md Docs/Notes/HIPIssues.md Docs/Notes/Notes.io_implementation commit 15f5168a1ad9f49d87ff0ee25935e171fe1d0f19 Author: Weiqun Zhang Date: Fri Mar 19 20:32:22 2021 -0700 amrex::EnableIf_t -> std::enable_if_t (#1885) Since we require C++14 now, we can use std::enable_if_t now. Src/AmrCore/AMReX_FillPatchUtil.H Src/AmrCore/AMReX_FillPatchUtil_I.H Src/Base/AMReX_FBI.H Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_GpuFuse.H Src/Base/AMReX_GpuLaunchFunctsC.H Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBCImpl.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_Scan.H Src/Base/AMReX_Tuple.H Src/Base/AMReX_TypeTraits.H Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleMesh.H Src/Particle/AMReX_ParticleReduce.H Src/Particle/AMReX_ParticleTile.H Src/Particle/AMReX_ParticleTransformation.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_WriteBinaryParticleData.H commit 1dfb98c29afe574afe30101a8db82190d9eadaa6 Author: Andrew Myers Date: Fri Mar 19 23:31:57 2021 -0400 remove __attribute__((warn_unused_result)) from AMREX_NODISCARD (#1886) Src/Base/AMReX_Extension.H commit 352059135e3e87f3b35e237b1c7fb8961eabffa1 Author: Weiqun Zhang Date: Fri Mar 19 18:11:52 2021 -0700 Make FabArray::setFab safer (#1884) Instead of taking a raw pointer, it takes std::unique_ptr or rvalue FAB, so that the ownership is very clear. It also deletes the old FAB if not null. Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_ParallelDescriptor.H Src/EB/AMReX_EB2_MultiGFab.cpp Src/EB/AMReX_MultiCutFab.cpp Src/Extern/amrdata/AMReX_AmrData.cpp commit 9e1576fab02e20b19d54105e6e7a434bf6d6f040 Author: Weiqun Zhang Date: Fri Mar 19 17:00:02 2021 -0700 Fix typo in AMREX_DEBUG macro (#1881) Fix the call to CheckRcvStats. Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBCImpl.H commit ea9ba93abd20c9a4c79f2ae8c392cd62a553119f Author: Maikel Nadolski Date: Fri Mar 19 23:23:51 2021 +0100 Forward declare functions that are used in the implementation (#1882) This PR forward declares the functions local_copy_cpu unpack_recv_buffer_cpu local_copy_gpu unpack_recv_buffer_gpu before its first usage. Some compilers error out if a function is used before it was declared, even in template code. I am not sure, why it worked in the CI so far. Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBCImpl.H commit ca1839a8d9d805e915d6bd9d385a0c3087ad866d Author: Axel Huebl Date: Fri Mar 19 12:44:33 2021 -0700 Flags: RelWithDebInfo (#1879) Add warning control to CMake Build mode `RelWithDebugInfo` (currently default for WarpX). Tools/CMake/AMReXFlagsTargets.cmake commit 82664e56d3dbdefa75bec47ddd871996778805c3 Author: Axel Huebl Date: Fri Mar 19 08:47:04 2021 -0700 Mask: -Wfinal-dtor-non-final-class (#1880) ## Summary ``` Src/Boundary/AMReX_Mask.H:57:40: warning: class with destructor marked 'final' cannot be inherited from [-Wfinal-dtor-non-final-class] virtual ~Mask () noexcept override final {} ^ Src/Boundary/AMReX_Mask.H:26:7: note: mark 'amrex::Mask' as 'final' to silence this warning class Mask ``` Fix the warning in TagBox, CutFab and EBCellFlagFab as well. ## Additional background Seen with Clang 10.0 on Ubuntu 20.04 LTS. Co-authored-by: Weiqun Zhang Src/AmrCore/AMReX_TagBox.H Src/Boundary/AMReX_Mask.H Src/EB/AMReX_EBCellFlag.H Src/EB/AMReX_MultiCutFab.H commit 0e1acd6c559b8939ac323446963dfafbf7430052 Author: Andrew Myers Date: Fri Mar 19 01:50:12 2021 -0400 Fix Typedescriptor test for Windows (#1878) Tests/Particles/TypeDescriptor/main.cpp commit 281ff0fb583047355e0a726805c08d5d4e20cf67 Author: Axel Huebl Date: Thu Mar 18 11:42:30 2021 -0700 Build: Windows with Shared Libs (.dll) (#1847) * CI: Win with Shared Libs (.dll) Demonstrator of shared library issues on Windows. * Remove: warn-unresolved-symbols / undefined,warning * CMake typecheckobjs lib: export symbols Make this library behave like on Unix: symbol visibility by default. * Globals: dllimport/dllexport (Win) https://stackoverflow.com/questions/54560832/cmake-windows-export-all-symbols-does-not-cover-global-variables/54568678#54568678 .github/workflows/windows.yml Src/AmrCore/AMReX_Interpolater.H Src/Base/AMReX.H Src/Base/AMReX_Extension.H Src/Base/AMReX_MFIter.H Src/Base/AMReX_ParallelContext.H Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_VisMF.H Src/EB/AMReX_EBInterpolater.H Src/Particle/AMReX_ParticleContainerBase.H Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReXTypecheck.cmake Tools/CMake/AMReX_Config.H.in Tools/CMake/AMReX_Config.cmake Tools/GNUMake/tools/Make.sensei commit 17371b17ffd5f7794a91236de7fa464d4284fa86 Author: Andrew Myers Date: Thu Mar 18 13:24:45 2021 -0400 Remove tutorials documentation from main repo; it now lives at https://github.com/AMReX-Codes/amrex-tutorials (#1877) Docs/sphinx_tutorials/Makefile Docs/sphinx_tutorials/source/AMR_Tutorial.rst Docs/sphinx_tutorials/source/Basic_Tutorial.rst Docs/sphinx_tutorials/source/Blueprint_Tutorial.rst Docs/sphinx_tutorials/source/EB_Tutorial.rst Docs/sphinx_tutorials/source/ForkJoin_Tutorial.rst Docs/sphinx_tutorials/source/GPU_Tutorial.rst Docs/sphinx_tutorials/source/LinearSolvers_Tutorial.rst Docs/sphinx_tutorials/source/MUI_Tutorial.rst Docs/sphinx_tutorials/source/Particles_Tutorial.rst Docs/sphinx_tutorials/source/SDC_Tutorial.rst Docs/sphinx_tutorials/source/SWFFT/iface_rect.png Docs/sphinx_tutorials/source/SWFFT_Tutorial.rst Docs/sphinx_tutorials/source/_static/theme_overrides.css Docs/sphinx_tutorials/source/conf.py Docs/sphinx_tutorials/source/figs/fork_join_tasks.png Docs/sphinx_tutorials/source/figs/mf_remap_hires.png Docs/sphinx_tutorials/source/figs/nested_fork_join_tasks.png Docs/sphinx_tutorials/source/index.rst build_docs.sh commit b31d050d176c822d51dad418a868d62426c6a1b4 Author: Weiqun Zhang Date: Wed Mar 17 19:13:57 2021 -0700 2D Poisson and ALaplacian Solvers in 3D Build of AMReX (#1805) New capability of solving 2D cell-centered Poisson's and ALaplacian equation with 3D build of amrex. Src/Base/AMReX_Box.H Src/Base/AMReX_Box.cpp Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp Src/LinearSolvers/CMakeLists.txt Src/LinearSolvers/MLMG/AMReX_MLALap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_K.H Src/LinearSolvers/MLMG/AMReX_MLALaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLCGSolver.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLMG_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_K.H Src/LinearSolvers/MLMG/Make.package commit 14b6bff6954d677b496af065c8c9def6bacc2d43 Author: Weiqun Zhang Date: Wed Mar 17 18:26:39 2021 -0700 Add FabArray::release and Make BaseFab virtual (#1873) Add FabArray::release functions that release the ownership of the FAB at the given index and return a raw pointer to the FAB. Make BaseFab virtual so that it's safe to use the new release functions. Also declare a number of functions `[[nodiscard]]`. Fix some warnings in EB/CNS example. Src/AmrCore/AMReX_TagBox.H Src/Base/AMReX_BaseFab.H Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabFactory.H Src/Base/AMReX_IArrayBox.H Src/Boundary/AMReX_Mask.H Src/EB/AMReX_EBCellFlag.H Src/EB/AMReX_EBFabFactory.H Src/EB/AMReX_EBFabFactory.cpp Src/EB/AMReX_MultiCutFab.H Tutorials/EB/CNS/Source/CNS.cpp Tutorials/EB/CNS/Source/CNS_advance.cpp Tutorials/EB/CNS/Source/CNS_init_eb2.cpp commit 87b182a718574cc52a93750a4c484c2e5aa4264b Author: Weiqun Zhang Date: Wed Mar 17 15:16:57 2021 -0700 Robin BC in Linear Solver (#1848) Add Robin BC to MLABecLaplacian. There are two steps for setting up Robin BC. The first step is to call setDomainBC, and the second step is to call setLevelBC, which has been modified to take a, b and f in Robin BC, `a*phi + b*dphi/dn = f`. Docs/sphinx_documentation/source/LinearSolvers.rst Src/Boundary/AMReX_LO_BCTYPES.H Src/Boundary/AMReX_LO_BCTYPES.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.H Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp commit 5b235f13980b2602636eff689f95d96a99c75d06 Author: Weiqun Zhang Date: Wed Mar 17 10:06:45 2021 -0700 LevelBld (#1852) In Amr class, we need to get LevelBld* and use it to build AmrLevels. Previously, Amr called a function `LevelBld* getLevelBld()` to get that. The problem was that `getLevelBld` was not defined by amrex but it was being called by amrex. This caused issues in building amrex as a shared library on Mac and Windows, because they do not support weak symbols. In this PR, the constructors of Amr now take `LevelBld*` as an argument. This is unfortunately a breaking change. However, it's a compile time failure and it's easy to fix. See the changes in this commit for examples. Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp Src/Amr/AMReX_LevelBld.H Tutorials/Amr/Advection_AmrLevel/Source/main.cpp Tutorials/EB/CNS/Source/main.cpp Tutorials/GPU/CNS/Source/main.cpp Tutorials/GPU/EBCNS/Source/main.cpp commit 5d32c74750b59e06c5731e084a75eec6ab91aad9 Author: drangara <69211175+drangara@users.noreply.github.com> Date: Wed Mar 17 11:24:36 2021 -0400 Fix bug with face-centroid slopes when using multigrid (#1871) When computing the Least Square slopes on face-centroid stencil with multigrid, we need to grow `m_eb_phi` and make additional checks. This adds to the changes made in https://github.com/AMReX-Codes/amrex/pull/1707 Src/EB/AMReX_EB_LeastSquares_2D_K.H Src/EB/AMReX_EB_LeastSquares_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Tests/LinearSolvers/LeastSquares/inputs.2d.askew-x.mg Tests/LinearSolvers/LeastSquares/inputs.2d.askew-y.mg Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.askew-all.mg commit c28d91c2ee07ad6f970eb10bbc14d00af6adab4a Author: Weiqun Zhang Date: Tue Mar 16 14:59:05 2021 -0700 Avoid conflict with X11 (#1872) Bool -> B to avoid a conflict with an X11 header that defines ``` #define Bool int #define Status int #define True 1 #define False 0 ``` This should fix the compilation issue of Amrvis. Src/Base/AMReX_TypeTraits.H commit 44f77978de2bdaec40cc6136169b854a78ff167e Author: Andrew Myers Date: Tue Mar 16 11:23:18 2021 -0400 Add explanation of parameters to Redistribute docstring (#1868) Src/Particle/AMReX_Particles.H commit e1b123b2302c0377efb9ab99066a60f907c663c9 Author: Weiqun Zhang Date: Mon Mar 15 18:05:17 2021 -0700 Fix sign-compare and shadow warnings (#1867) The warning about shadowed varible in AMReX_NonLocalBC.H seems a compiler bug. Nevertheless, the source is modified to get rid of the warning. Src/Base/AMReX_NonLocalBC.H Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_ParticleContainerI.H commit 7d833e64ff06477953c9b3a2642b04e61116be93 Author: Weiqun Zhang Date: Mon Mar 15 12:42:05 2021 -0700 EditorConfig and trailing whitespaces (#1866) Trim trailing whitespaces in editorconfig for C/C++ codes and others. Allow md file to have trailing whitespaces because they are syntactically significant. .editorconfig .github/workflows/style/check_trailing_whitespaces.sh commit e5b06d59e3a6cf55c40c6498e42eafe940203cb9 Author: Weiqun Zhang Date: Mon Mar 15 12:28:06 2021 -0700 Optimization of CArena::freeUnused (#1865) Use `find` instead of manully looping over the set. Src/Base/AMReX_CArena.cpp commit fbef03f566fbbe3c583bd96db00b5cee391ad6b8 Author: Axel Huebl Date: Mon Mar 15 12:25:52 2021 -0700 CI: macOS Shared Lib (#1836) ## Summary CI for macOS: Build a shared library. This is the default in Spack. I see issues with AppleClang 12.0 raising: ``` ld: can't use -undefined warning or suppress with -twolevel_namespace clang: error: linker command failed with exit code 1 (use -v to see invocation) ``` @RemiLehe reported this again here: https://github.com/AMReX-Codes/amrex/issues/425#issuecomment-473153152 - [x] reproduce issue - [x] add fix ## Additional background Introduced in 1b6af3cb630 to fix #425 .github/workflows/dependencies/dependencies_mac.sh .github/workflows/macos.yml Src/Amr/AMReX_LevelBld.H Src/Amr/AMReX_PROB_AMR_F.H Src/Base/AMReX_Extension.H Tools/CMake/AMReX_Config.cmake commit 2789d017a7d1e4803fe9ca007dbbad611e42069f Author: Weiqun Zhang Date: Sun Mar 14 14:39:58 2021 -0700 Fix bug in #1861 (#1864) We cannot simply use std::set::erase on the free node list because the operator== only compares the starting block address. Src/Base/AMReX_CArena.cpp commit 107b5d4f36063b110539242928faba914f07e5b3 Author: Weiqun Zhang Date: Sun Mar 14 14:25:49 2021 -0700 Fix CUDA API calls in host callback functions (#1863) CUDA API calls are not allowed in host callback functions. Therefore, we have to move the call to free unused memory from `free` to `alloc`. Src/Base/AMReX_CArena.cpp commit d28caf8685f6c65639aee1d7d1731901480b3a14 Author: Weiqun Zhang Date: Sun Mar 14 14:19:48 2021 -0700 CI for Code Style (#1862) Check for tabs and trailing whitespaces at the end of line in CI. .github/workflows/style.yml .github/workflows/style/check_tabs.sh .github/workflows/style/check_trailing_whitespaces.sh .gitignore commit 486cd438aea970962a02c799c910eb8ddcd7dd71 Author: CCSE@LBNL Date: Sun Mar 14 08:38:31 2021 -0700 Remove tabs and trailing white spaces (#1860) The changes are purely white spaces only. `git diff -w development..HEAD` shows nothing. It was done with ``` find . -type d \( -name .git \ -o -path ./paper \ \) -prune -o \ -type f \( -name "*.H" -o -name "*.h" -o -name "*.hh" -o -name "*.hpp" \ -o -name "*.c" -o -name "*.cc" -o -name "*.cpp" -o -name "*.cxx" \ -o -name "*.f" -o -name "*.F" -o -name "*.f90" -o -name "*.F90" \ -o -name "*.py" \ -o -name "*.md" -o -name "*.rst" \ -o -name "*.sh" \ -o -name "*.tex" \ -o -name "*.txt" \ \) \ -exec grep -Iq . {} \; \ -exec sed -i 's/[[:blank:]]\+$//g' {} + \ -exec sed -i 's/\t/\ \ \ \ \ \ \ \ /g' {} + ``` We will add a CI check as a follow-up. CONTRIBUTING.md Docs/Migration/Migration.md Docs/sphinx_documentation/source/AMReX_Profiling_Tools_Chapter.rst Docs/sphinx_documentation/source/AmrCore.rst Docs/sphinx_documentation/source/AmrLevel.rst Docs/sphinx_documentation/source/Basics.rst Docs/sphinx_documentation/source/DualGrid.rst Docs/sphinx_documentation/source/EB.rst Docs/sphinx_documentation/source/External_Profiling_Tools.rst Docs/sphinx_documentation/source/Fortran.rst Docs/sphinx_documentation/source/GPU.rst Docs/sphinx_documentation/source/GPU_Chapter.rst Docs/sphinx_documentation/source/GettingStarted.rst Docs/sphinx_documentation/source/GridCreation.rst Docs/sphinx_documentation/source/IO.rst Docs/sphinx_documentation/source/InputsLoadBalancing.rst Docs/sphinx_documentation/source/Inputs_Chapter.rst Docs/sphinx_documentation/source/Introduction.rst Docs/sphinx_documentation/source/LinearSolvers.rst Docs/sphinx_documentation/source/LinearSolvers_Chapter.rst Docs/sphinx_documentation/source/LoadBalancing.rst Docs/sphinx_documentation/source/ManagingGridHierarchy_Chapter.rst Docs/sphinx_documentation/source/Particle.rst Docs/sphinx_documentation/source/SWFFT.rst Docs/sphinx_documentation/source/Testing.rst Docs/sphinx_documentation/source/Visualization.rst Docs/sphinx_documentation/source/index.rst Docs/sphinx_tutorials/source/AMR_Tutorial.rst Docs/sphinx_tutorials/source/Basic_Tutorial.rst Docs/sphinx_tutorials/source/Blueprint_Tutorial.rst Docs/sphinx_tutorials/source/GPU_Tutorial.rst Docs/sphinx_tutorials/source/LinearSolvers_Tutorial.rst Docs/sphinx_tutorials/source/MUI_Tutorial.rst Docs/sphinx_tutorials/source/Particles_Tutorial.rst Docs/sphinx_tutorials/source/index.rst README.md Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp Src/Amr/AMReX_AmrLevel.H Src/Amr/AMReX_AmrLevel.cpp Src/Amr/AMReX_AuxBoundaryData.H Src/Amr/AMReX_AuxBoundaryData.cpp Src/Amr/AMReX_Derive.H Src/Amr/AMReX_Derive.cpp Src/Amr/AMReX_StateData.H Src/Amr/AMReX_StateData.cpp Src/Amr/AMReX_StateDescriptor.H Src/Amr/AMReX_StateDescriptor.cpp Src/Amr/AMReX_extrapolater_3D_K.H Src/AmrCore/AMReX_AmrCore.cpp Src/AmrCore/AMReX_AmrMesh.cpp Src/AmrCore/AMReX_AmrParGDB.H Src/AmrCore/AMReX_AmrParticles.H Src/AmrCore/AMReX_Cluster.cpp Src/AmrCore/AMReX_ErrorList.H Src/AmrCore/AMReX_ErrorList.cpp Src/AmrCore/AMReX_FillPatchUtil.H Src/AmrCore/AMReX_FillPatchUtil_2d.F90 Src/AmrCore/AMReX_FillPatchUtil_3d.F90 Src/AmrCore/AMReX_FillPatchUtil_F.H Src/AmrCore/AMReX_FillPatchUtil_I.H Src/AmrCore/AMReX_FluxRegister.cpp Src/AmrCore/AMReX_INTERP_1D.F90 Src/AmrCore/AMReX_INTERP_2D.F90 Src/AmrCore/AMReX_INTERP_F.H Src/AmrCore/AMReX_Interp_1D_C.H Src/AmrCore/AMReX_Interp_2D_C.H Src/AmrCore/AMReX_Interp_3D_C.H Src/AmrCore/AMReX_Interpolater.H Src/AmrCore/AMReX_Interpolater.cpp Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX.H Src/Base/AMReX.cpp Src/Base/AMReX_BCRec.cpp Src/Base/AMReX_BLBackTrace.cpp Src/Base/AMReX_BLProfiler.H Src/Base/AMReX_BLProfiler.cpp Src/Base/AMReX_BLutil_F.F90 Src/Base/AMReX_BaseFab.H Src/Base/AMReX_BaseFab.cpp Src/Base/AMReX_BaseUmap_f.H Src/Base/AMReX_BlockMutex.H Src/Base/AMReX_BlockMutex.cpp Src/Base/AMReX_Box.H Src/Base/AMReX_Box.cpp Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp Src/Base/AMReX_BoxDomain.cpp Src/Base/AMReX_BoxList.H Src/Base/AMReX_BoxList.cpp Src/Base/AMReX_COORDSYS_1D_C.H Src/Base/AMReX_COORDSYS_2D_C.H Src/Base/AMReX_CoordSys.cpp Src/Base/AMReX_CudaGraph.H Src/Base/AMReX_CuptiTrace.H Src/Base/AMReX_CuptiTrace.cpp Src/Base/AMReX_DistributionMapping.H Src/Base/AMReX_DistributionMapping.cpp Src/Base/AMReX_Extension.H Src/Base/AMReX_FACopyDescriptor.H Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_FBI.H Src/Base/AMReX_FILCC_1D.F90 Src/Base/AMReX_FILCC_2D.F90 Src/Base/AMReX_FILCC_3D.F90 Src/Base/AMReX_FPC.cpp Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_FabConv.cpp Src/Base/AMReX_FileSystem.cpp Src/Base/AMReX_Geometry.H Src/Base/AMReX_Geometry.cpp Src/Base/AMReX_GpuAllocators.H Src/Base/AMReX_GpuComplex.H Src/Base/AMReX_GpuContainers.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuLaunch.H Src/Base/AMReX_GpuMemory.H Src/Base/AMReX_GpuUtility.H Src/Base/AMReX_IArrayBox.H Src/Base/AMReX_IArrayBox.cpp Src/Base/AMReX_INT.H Src/Base/AMReX_IntConv.H Src/Base/AMReX_IntConv.cpp Src/Base/AMReX_IntVect.H Src/Base/AMReX_Lazy.H Src/Base/AMReX_Lazy.cpp Src/Base/AMReX_MFCopyDescriptor.cpp Src/Base/AMReX_MFIter.H Src/Base/AMReX_MFIter.cpp Src/Base/AMReX_Machine.cpp Src/Base/AMReX_MemPool.cpp Src/Base/AMReX_MemProfiler.H Src/Base/AMReX_MemProfiler.cpp Src/Base/AMReX_MultiFab.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_MultiFabUtil_1D_C.H Src/Base/AMReX_NFiles.H Src/Base/AMReX_NFiles.cpp Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBC.cpp Src/Base/AMReX_NonLocalBCImpl.H Src/Base/AMReX_PODVector.H Src/Base/AMReX_ParallelContext.cpp Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelDescriptor.cpp Src/Base/AMReX_ParallelDescriptor_F.F90 Src/Base/AMReX_ParmParse.H Src/Base/AMReX_ParmParse.cpp Src/Base/AMReX_Partition.H Src/Base/AMReX_Periodicity.H Src/Base/AMReX_Periodicity.cpp Src/Base/AMReX_PlotFileUtil.H Src/Base/AMReX_PlotFileUtil.cpp Src/Base/AMReX_Print.H Src/Base/AMReX_REAL.H Src/Base/AMReX_RealBox.cpp Src/Base/AMReX_RealVect.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_SPACE.H Src/Base/AMReX_Scan.H Src/Base/AMReX_Slopes_K.H Src/Base/AMReX_ThirdPartyProfiling.H Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_TinyProfiler.cpp Src/Base/AMReX_TypeTraits.H Src/Base/AMReX_Utility.H Src/Base/AMReX_Utility.cpp Src/Base/AMReX_Vector.H Src/Base/AMReX_VectorIO.H Src/Base/AMReX_VectorIO.cpp Src/Base/AMReX_VisMF.H Src/Base/AMReX_VisMF.cpp Src/Base/AMReX_bc_types_mod.F90 Src/Base/AMReX_ccse-mpi.H Src/Base/AMReX_error_fi.cpp Src/Base/AMReX_error_mod.F90 Src/Base/AMReX_filcc_mod.F90 Src/Base/AMReX_iMultiFab.H Src/Base/AMReX_iMultiFab.cpp Src/Base/AMReX_mempool_mod.F90 Src/Base/AMReX_parmparse_fi.cpp Src/Base/AMReX_parmparse_mod.F90 Src/Boundary/AMReX_BndryData.H Src/Boundary/AMReX_BndryData.cpp Src/Boundary/AMReX_BndryRegister.H Src/Boundary/AMReX_BndryRegister.cpp Src/Boundary/AMReX_FabSet.H Src/Boundary/AMReX_FabSet.cpp Src/Boundary/AMReX_InterpBndryData.H Src/Boundary/AMReX_InterpBndryData.cpp Src/Boundary/AMReX_MacBndry.H Src/Boundary/AMReX_MacBndry.cpp Src/Boundary/AMReX_Mask.H Src/Boundary/AMReX_MultiMask.H Src/Boundary/AMReX_MultiMask.cpp Src/Boundary/AMReX_YAFluxRegister.cpp Src/EB/AMReX_EB2_3D_C.H Src/EB/AMReX_EB2_IF_Complement.H Src/EB/AMReX_EB2_IF_Lathe.H Src/EB/AMReX_EB2_IF_Sphere.H Src/EB/AMReX_EB2_Level.cpp Src/EB/AMReX_EBFluxRegister.cpp Src/EB/AMReX_EBInterpolater.cpp Src/EB/AMReX_EBMultiFabUtil.H Src/EB/AMReX_EBMultiFabUtil.cpp Src/EB/AMReX_EBMultiFabUtil_2D_C.H Src/EB/AMReX_EBMultiFabUtil_3D_C.H Src/EB/AMReX_EBToPVD.H Src/EB/AMReX_EBToPVD.cpp Src/EB/AMReX_EB_LeastSquares_2D_K.H Src/EB/AMReX_EB_LeastSquares_3D_K.H Src/EB/AMReX_EB_STL_utils.H Src/EB/AMReX_EB_STL_utils.cpp Src/EB/AMReX_EB_slopes_K.H Src/EB/AMReX_EB_triGeomOps_K.H Src/EB/AMReX_MultiCutFab.H Src/EB/AMReX_WriteEBSurface.H Src/EB/AMReX_WriteEBSurface.cpp Src/EB/AMReX_ebcellflag_mod.F90 Src/Extern/Conduit/AMReX_Conduit_Blueprint_ParticlesI.H Src/Extern/HYPRE/AMReX_Hypre.H Src/Extern/HYPRE/AMReX_Hypre.cpp Src/Extern/HYPRE/AMReX_HypreABecLap.H Src/Extern/HYPRE/AMReX_HypreABecLap.cpp Src/Extern/HYPRE/AMReX_HypreABecLap2.H Src/Extern/HYPRE/AMReX_HypreABecLap2.cpp Src/Extern/HYPRE/AMReX_HypreABecLap3.H Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/PETSc/AMReX_PETSc.H Src/Extern/PETSc/AMReX_PETSc.cpp Src/Extern/ProfParser/AMReX_AVGDOWN_1D.F Src/Extern/ProfParser/AMReX_AVGDOWN_2D.F Src/Extern/ProfParser/AMReX_AVGDOWN_3D.F Src/Extern/ProfParser/AMReX_AVGDOWN_F.H Src/Extern/ProfParser/AMReX_BLProfStats.H Src/Extern/ProfParser/AMReX_BLProfStats.cpp Src/Extern/ProfParser/AMReX_BLProfUtilities.H Src/Extern/ProfParser/AMReX_BLProfUtilities.cpp Src/Extern/ProfParser/AMReX_BLWritePlotFile.H Src/Extern/ProfParser/AMReX_BLWritePlotFile.cpp Src/Extern/ProfParser/AMReX_CommProfStats.H Src/Extern/ProfParser/AMReX_CommProfStats.cpp Src/Extern/ProfParser/AMReX_ProfParserBatch.cpp Src/Extern/ProfParser/AMReX_RegionsProfStats.H Src/Extern/ProfParser/AMReX_RegionsProfStats.cpp Src/Extern/SWFFT/AlignedAllocator.h Src/Extern/SWFFT/CheckDecomposition.c Src/Extern/SWFFT/Dfft.H Src/Extern/SWFFT/DfftC.cpp Src/Extern/SWFFT/Distribution.H Src/Extern/SWFFT/DistributionC.cpp Src/Extern/SWFFT/Error.h Src/Extern/SWFFT/TimingStats.h Src/Extern/SWFFT/complex-type.h Src/Extern/SWFFT/distribution.c Src/Extern/SWFFT/distribution_c.h Src/Extern/amrdata/AMReX_AmrData.H Src/Extern/amrdata/AMReX_AmrData.cpp Src/Extern/amrdata/AMReX_DataServices.H Src/Extern/amrdata/AMReX_DataServices.cpp Src/Extern/amrdata/AMReX_FABUTIL_1D.F Src/Extern/amrdata/AMReX_FABUTIL_2D.F Src/Extern/amrdata/AMReX_FABUTIL_3D.F Src/Extern/amrdata/AMReX_XYPlotDataList.H Src/Extern/amrdata/AMReX_XYPlotDataList.cpp Src/Extern/amrdata/CMakeLists.txt Src/Extern/hpgmg/BL_HPGMG.H Src/Extern/hpgmg/BL_HPGMG.cpp Src/F_Interfaces/AmrCore/AMReX_FAmrCore.H Src/F_Interfaces/AmrCore/AMReX_FAmrCore.cpp Src/F_Interfaces/AmrCore/AMReX_amrcore_fi.cpp Src/F_Interfaces/AmrCore/AMReX_amrcore_mod.F90 Src/F_Interfaces/AmrCore/AMReX_fillpatch_fi.cpp Src/F_Interfaces/AmrCore/AMReX_fillpatch_mod.F90 Src/F_Interfaces/AmrCore/AMReX_fluxregister_fi.cpp Src/F_Interfaces/AmrCore/AMReX_fluxregister_mod.F90 Src/F_Interfaces/AmrCore/AMReX_tagbox_fi.cpp Src/F_Interfaces/AmrCore/AMReX_tagbox_mod.F90 Src/F_Interfaces/Base/AMReX_FPhysBC.cpp Src/F_Interfaces/Base/AMReX_box_fi.cpp Src/F_Interfaces/Base/AMReX_box_mod.F90 Src/F_Interfaces/Base/AMReX_boxarray_fi.cpp Src/F_Interfaces/Base/AMReX_distromap_fi.cpp Src/F_Interfaces/Base/AMReX_distromap_mod.F90 Src/F_Interfaces/Base/AMReX_geometry_fi.cpp Src/F_Interfaces/Base/AMReX_geometry_mod.F90 Src/F_Interfaces/Base/AMReX_init_mod.F90 Src/F_Interfaces/Base/AMReX_multifab_fi.cpp Src/F_Interfaces/Base/AMReX_multifab_mod.F90 Src/F_Interfaces/Base/AMReX_multifabutil_fi.cpp Src/F_Interfaces/Base/AMReX_parallel_mod.F90 Src/F_Interfaces/Base/AMReX_physbc_fi.cpp Src/F_Interfaces/Base/AMReX_physbc_mod.F90 Src/F_Interfaces/Base/AMReX_plotfile_fi.cpp Src/F_Interfaces/Base/AMReX_vismf_fi.cpp Src/F_Interfaces/LinearSolvers/AMReX_linop_mod.F90 Src/F_Interfaces/LinearSolvers/AMReX_multigrid_fi.cpp Src/F_Interfaces/LinearSolvers/AMReX_multigrid_mod.F90 Src/F_Interfaces/LinearSolvers/AMReX_poisson_mod.F90 Src/F_Interfaces/Octree/AMReX_octree_fi.cpp Src/F_Interfaces/Particle/AMReX_particlecontainer_fi.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLALap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLALaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLCGSolver.H Src/LinearSolvers/MLMG/AMReX_MLCGSolver.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLEBTensor_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensor_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp_nd.F90 Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLMGBndry.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensorOp.H Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLTensor_3D_K.H Src/LinearSolvers/Projections/AMReX_MacProjector.H Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp Src/Particle/AMReX_ArrayOfStructs.H Src/Particle/AMReX_BinIterator.H Src/Particle/AMReX_NeighborParticlesGPUImpl.H Src/Particle/AMReX_ParGDB.H Src/Particle/AMReX_Particle.H Src/Particle/AMReX_ParticleCommunication.cpp Src/Particle/AMReX_ParticleContainerBase.H Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleInit.H Src/Particle/AMReX_ParticleMPIUtil.H Src/Particle/AMReX_ParticleMesh.H Src/Particle/AMReX_ParticleReduce.H Src/Particle/AMReX_ParticleTransformation.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_Particle_mod_K.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_SparseBins.H Src/Particle/AMReX_TracerParticle_mod_K.H Src/Particle/AMReX_TracerParticles.H Src/Particle/AMReX_TracerParticles.cpp Src/SDC/AMReX_SDCquadrature.F90 Src/SDC/AMReX_SDCstruct.H Src/SDC/AMReX_SDCstruct.cpp Tests/AsyncOut/multifab/main.cpp Tests/DivFreePatch/main.cpp Tests/FillBoundaryComparison/main.cpp Tests/GPU/AnyOf/main.cpp Tests/GPU/Fuse/main.cpp Tests/GPU/Vector/main.cpp Tests/HDF5Benchmark/main.cpp Tests/LinearSolvers/ABecLaplacian_C/MyTest.H Tests/LinearSolvers/ABecLaplacian_C/MyTest.cpp Tests/LinearSolvers/ABecLaplacian_C/initProb.cpp Tests/LinearSolvers/ABecLaplacian_C/initProb_K.H Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-16.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-32.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-8.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-1.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-16.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-2.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-32.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-4.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-64.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-8.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-1.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-16.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-2.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-32.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-4.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-64.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-8.sh Tests/LinearSolvers/ABecLaplacian_F/init_prob.F90 Tests/LinearSolvers/ABecLaplacian_F/mytest.F90 Tests/LinearSolvers/CellEB/MyTest.H Tests/LinearSolvers/CellEB/MyTest.cpp Tests/LinearSolvers/CellEB/initEB.cpp Tests/LinearSolvers/CellEB2/MyTest.H Tests/LinearSolvers/CellEB2/MyTest.cpp Tests/LinearSolvers/CellEB2/initEB.cpp Tests/LinearSolvers/EBConvergenceTest/BC_2D.F90 Tests/LinearSolvers/EBConvergenceTest/BC_3D.F90 Tests/LinearSolvers/EBConvergenceTest/Convergence_Data_Gen.sh Tests/LinearSolvers/EBConvergenceTest/MyTest.H Tests/LinearSolvers/EBConvergenceTest/MyTest.cpp Tests/LinearSolvers/EBConvergenceTest/MyTest_F.H Tests/LinearSolvers/EBConvergenceTest/RHS.F90 Tests/LinearSolvers/EBConvergenceTest/acoef.F90 Tests/LinearSolvers/EBConvergenceTest/bcoef.F90 Tests/LinearSolvers/EBTensor/MyTest.cpp Tests/LinearSolvers/EBTensor/MyTestPlotfile.cpp Tests/LinearSolvers/EBTensor/MyTest_2D_K.H Tests/LinearSolvers/EBflux_grad/MyTest.H Tests/LinearSolvers/EBflux_grad/MyTest.cpp Tests/LinearSolvers/EBflux_grad/initEB.cpp Tests/LinearSolvers/LeastSquares/MyTest.cpp Tests/LinearSolvers/LeastSquares/README.md Tests/LinearSolvers/LeastSquares/initData.cpp Tests/LinearSolvers/LeastSquares/initEB.cpp Tests/LinearSolvers/LeastSquares/initPoiseuilleData.cpp Tests/LinearSolvers/MAC_Projection_EB/main.cpp Tests/LinearSolvers/MLMG/fort_3d.F90 Tests/LinearSolvers/MLMG/init_prob.cpp Tests/LinearSolvers/MLMG/main.cpp Tests/LinearSolvers/MultiComponent/MCNodalLinOp.H Tests/LinearSolvers/MultiComponent/MCNodalLinOp.cpp Tests/LinearSolvers/MultiComponent/main.cpp Tests/LinearSolvers/Nodal_Projection_EB/main.cpp Tests/LinearSolvers/NodeEB/MyTest.cpp Tests/MultiBlock/Advection/main.cpp Tests/MultiBlock/IndexType/main.cpp Tests/Particles/AssignDensity/main.cpp Tests/Particles/AssignMultiLevelDensity/main.cpp Tests/Particles/AsyncIO/main.cpp Tests/Particles/GhostsAndVirtuals/main.cpp Tests/Particles/InitFromAscii/main.cpp Tests/Particles/Intersection/main.cpp Tests/Particles/NeighborParticles/CheckPair.H Tests/Particles/NeighborParticles/Constants.H Tests/Particles/NeighborParticles/MDParticleContainer.cpp Tests/Particles/NeighborParticles/README.md Tests/Particles/ParticleIterator/main.cpp Tests/Particles/ParticleMesh/main.cpp Tests/Particles/ParticleTransformations/main.cpp Tests/Particles/SparseBins/main.cpp Tests/Particles/TypeDescriptor/main.cpp Tools/AMRProfParser/TestCodes/AMRProfTest0.cpp Tools/AMRProfParser/TestCodes/ProfWaitTest.cpp Tools/AMRProfParser/TestCodes/SendTest0.cpp Tools/AMRProfParser/TestCodes/TokenizeTest.cpp Tools/C_util/AmrDeriveTecplot/AmrDeriveTecplot.cpp Tools/C_util/AppendToPlotFile.cpp Tools/C_util/AugmentPlotfile/AugmentPlotfile.cpp Tools/C_util/AugmentPlotfile/AugmentPlotfile_F.H Tools/C_util/Convergence/Add.cpp Tools/C_util/Convergence/ComparePlotfiles.cpp Tools/C_util/Convergence/ComputeAmrDataNorms.H Tools/C_util/Convergence/ComputeAmrDataNorms.cpp Tools/C_util/Convergence/DebugDump.H Tools/C_util/Convergence/DebugOut.H Tools/C_util/Convergence/DebugOut.cpp Tools/C_util/Convergence/DiffFab.cpp Tools/C_util/Convergence/DiffSameDomainRefined.cpp Tools/C_util/Convergence/DiffSameDomainRefinedComposite.cpp Tools/C_util/Convergence/DiffSameDomainRefinedFD.cpp Tools/C_util/Convergence/DiffSameDomainRefinedStag.cpp Tools/C_util/Convergence/DiffSameGrid.cpp Tools/C_util/Convergence/DiffSameGrid2.cpp Tools/C_util/Convergence/DiffSameGridRefined.cpp Tools/C_util/Convergence/DiffUniform.cpp Tools/C_util/Convergence/PltFileNorm.cpp Tools/C_util/Convergence/PltFileNormB.cpp Tools/C_util/Convergence/PltFileScalConvRate.cpp Tools/C_util/Convergence/RichardsonConvergenceTest.cpp Tools/C_util/DiffMultiFab/diffmultifab.cpp Tools/C_util/Statistics/AVGDOWN_2D.F Tools/C_util/Statistics/AVGDOWN_3D.F Tools/C_util/Statistics/AVGDOWN_F.H Tools/C_util/Statistics/ComputeAmrDataStat.H Tools/C_util/Statistics/ComputeAmrDataStat.cpp Tools/C_util/Statistics/PltFileFluxAve.H Tools/C_util/Statistics/PltFileFluxAve.cpp Tools/C_util/Statistics/PltFileList.cpp Tools/C_util/Statistics/PltFileStat.cpp Tools/C_util/Statistics/PltFileXAve.H Tools/C_util/Statistics/PltFileXAve.cpp Tools/C_util/TV_TempWrite.H Tools/C_util/ViewMF/MFNorm.cpp Tools/C_util/ViewMF/checkMFghostcells.cpp Tools/C_util/ViewMF/main.cpp Tools/C_util/ViewMF/mfMinMax.cpp Tools/C_util/ViewMF/viewMF.cpp Tools/C_util/ViewMF/viewMFcol.cpp Tools/C_util/ViewMF/viewMFdiff.cpp Tools/C_util/ViewMF/viewMFdiffHardWire.cpp Tools/C_util/WritePlotFile.H Tools/C_util/WritePlotFile.cpp Tools/C_util/dbgTools/crsGrids.cpp Tools/C_util/dbgTools/intersectGrids.cpp Tools/CompileTesting/compiletesting.py Tools/EBSurfaceTools/ConvertEBSurface.cpp Tools/EBSurfaceTools/isoToVTK.py Tools/F_scripts/fcheck.py Tools/GNUMake/README.md Tools/Plotfile/AMReX_PPMUtil.cpp Tools/Plotfile/fcompare.cpp Tools/Postprocessing/C_Src/HorizontalAvg.cpp Tools/Postprocessing/C_Src/IntegrateComp.cpp Tools/Postprocessing/C_Src/MultiFabToMatLab.cpp Tools/Postprocessing/C_Src/PlotfileToMatLab.cpp Tools/Postprocessing/C_Src/PlotfileToTurb.cpp Tools/Postprocessing/C_Src/PtwisePltTransform.cpp Tools/Postprocessing/C_Src/PtwisePltTransform_nd.f90 Tools/Postprocessing/C_Src/WritePlotfileToASCII.cpp Tools/Postprocessing/C_Src/cube_extract.cpp Tools/Postprocessing/C_Src/particle_compare.cpp Tools/Postprocessing/python/column_depth.py Tools/Postprocessing/python/conv_slopes.py Tools/Postprocessing/python/dumpparthistory.py Tools/Postprocessing/python/eos_data.txt Tools/Postprocessing/python/helmeos.py Tools/Postprocessing/python/parseparticles.py Tools/Postprocessing/python/test_helmeos.py Tools/Postprocessing/python/test_parseparticles.py Tools/Py_util/amrex_particles_to_vtp/amrex_binary_particles_to_vtp.py Tools/Py_util/amrex_particles_to_vtp/write_pview_file.py Tools/Py_util/plotsinglevar.py Tools/Release/ppCleanup.py Tools/Release/ppCleanupDir.py Tools/Release/release.py Tools/typechecker/typechecker.py Tutorials/Amr/Advection_AmrCore/Exec/Prob.H Tutorials/Amr/Advection_AmrCore/Exec/paraview_amr101.py Tutorials/Amr/Advection_AmrCore/README.md Tutorials/Amr/Advection_AmrCore/Source/AdvancePhiAllLevels.cpp Tutorials/Amr/Advection_AmrCore/Source/AdvancePhiAtLevel.cpp Tutorials/Amr/Advection_AmrCore/Source/AmrCoreAdv.H Tutorials/Amr/Advection_AmrCore/Source/AmrCoreAdv.cpp Tutorials/Amr/Advection_AmrCore/Source/DefineVelocity.cpp Tutorials/Amr/Advection_AmrCore/Source/Src_K/Adv_K.H Tutorials/Amr/Advection_AmrCore/Source/Src_K/compute_flux_3D_K.H Tutorials/Amr/Advection_AmrCore/Source/face_velocity.H Tutorials/Amr/Advection_AmrCore/Source/main.cpp Tutorials/Amr/Advection_AmrLevel/Exec/SingleVortex/Prob.f90 Tutorials/Amr/Advection_AmrLevel/Exec/SingleVortex/face_velocity_2d.f90 Tutorials/Amr/Advection_AmrLevel/Exec/SingleVortex/face_velocity_3d.f90 Tutorials/Amr/Advection_AmrLevel/Exec/UniformVelocity/Prob.f90 Tutorials/Amr/Advection_AmrLevel/Source/Adv_F.H Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.H Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tutorials/Amr/Advection_AmrLevel/Source/LevelBldAdv.cpp Tutorials/Amr/Advection_AmrLevel/Source/Src_2d/Adv_2d.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_2d/compute_flux_2d.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_2d/slope_2d.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_3d/Adv_3d.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_3d/compute_flux_3d.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_3d/slope_3d.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_nd/Tagging_nd.f90 Tutorials/Amr/Advection_AmrLevel/Source/Src_nd/tagging_params.f90 Tutorials/Amr/Advection_AmrLevel/Source/main.cpp Tutorials/Basic/Build_with_libamrex/main.cpp Tutorials/Basic/Build_with_libamrex/test_parameters.cpp Tutorials/Basic/HeatEquation_EX1_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX1_C/Source/myfunc.H Tutorials/Basic/HeatEquation_EX1_C/Source/myfunc.cpp Tutorials/Basic/HeatEquation_EX1_CF/Source/advance.cpp Tutorials/Basic/HeatEquation_EX1_CF/Source/advance_3d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/init_phi_2d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/init_phi_3d.f90 Tutorials/Basic/HeatEquation_EX1_CF/Source/main.cpp Tutorials/Basic/HeatEquation_EX1_CF/Source/myfunc.H Tutorials/Basic/HeatEquation_EX1_CF/Source/myfunc_F.H Tutorials/Basic/HeatEquation_EX1_F/advance.f90 Tutorials/Basic/HeatEquation_EX1_F/fmain.F90 Tutorials/Basic/HeatEquation_EX1_F/init_phi.f90 Tutorials/Basic/HeatEquation_EX2_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX2_C/Source/myfunc.H Tutorials/Basic/HeatEquation_EX2_C/Source/myfunc.cpp Tutorials/Basic/HeatEquation_EX2_C/Source/mykernel.H Tutorials/Basic/HeatEquation_EX2_CF/Source/advance.cpp Tutorials/Basic/HeatEquation_EX2_CF/Source/advance_3d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/init_phi_2d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/init_phi_3d.f90 Tutorials/Basic/HeatEquation_EX2_CF/Source/main.cpp Tutorials/Basic/HeatEquation_EX2_CF/Source/myfunc.H Tutorials/Basic/HeatEquation_EX2_CF/Source/myfunc_F.H Tutorials/Basic/HeatEquation_EX3_C/Source/advance.cpp Tutorials/Basic/HeatEquation_EX3_C/Source/init_phi_2d.f90 Tutorials/Basic/HeatEquation_EX3_C/Source/init_phi_3d.f90 Tutorials/Basic/HeatEquation_EX3_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX3_C/Source/myfunc.H Tutorials/Basic/HeatEquation_EX3_C/Source/myfunc_F.H Tutorials/Basic/main_C/main.cpp Tutorials/Basic/main_F/main.F90 Tutorials/Blueprint/AssignMultiLevelDensity/main.cpp Tutorials/Blueprint/CellSortedParticles/CellSortedPC.H Tutorials/Blueprint/CellSortedParticles/CellSortedPC.cpp Tutorials/Blueprint/CellSortedParticles/cell_sorted_3d.F90 Tutorials/Blueprint/CellSortedParticles/cell_sorted_F.H Tutorials/Blueprint/CellSortedParticles/main.cpp Tutorials/Blueprint/HeatEquation_EX1_C/Source/advance.cpp Tutorials/Blueprint/HeatEquation_EX1_C/Source/advance_3d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/init_phi_2d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/init_phi_3d.f90 Tutorials/Blueprint/HeatEquation_EX1_C/Source/main.cpp Tutorials/Blueprint/HeatEquation_EX1_C/Source/myfunc.H Tutorials/Blueprint/HeatEquation_EX1_C/Source/myfunc_F.H Tutorials/Blueprint/README.txt Tutorials/EB/CNS/Exec/Combustor/bc_fill_nd.F90 Tutorials/EB/CNS/Exec/Pulse/cns_prob.F90 Tutorials/EB/CNS/Exec/ShockRef/cns_prob.F90 Tutorials/EB/CNS/Exec/Sod/cns_prob.F90 Tutorials/EB/CNS/Source/CNS.H Tutorials/EB/CNS/Source/CNS.cpp Tutorials/EB/CNS/Source/CNS_F.H Tutorials/EB/CNS/Source/CNS_advance.cpp Tutorials/EB/CNS/Source/CNS_init_eb2.cpp Tutorials/EB/CNS/Source/CNS_io.cpp Tutorials/EB/CNS/Source/CNS_setup.cpp Tutorials/EB/CNS/Source/diffusion/cns_diff_mod.F90 Tutorials/EB/CNS/Source/diffusion/cns_eb_diff_wall.F90 Tutorials/EB/CNS/Source/diffusion/diff_coef_mod.F90 Tutorials/EB/CNS/Source/fortran/CNS_divop.F90 Tutorials/EB/CNS/Source/fortran/CNS_dudt.F90 Tutorials/EB/CNS/Source/fortran/CNS_f.F90 Tutorials/EB/CNS/Source/fortran/CNS_nd.F90 Tutorials/EB/CNS/Source/fortran/bc_fill_nd.F90 Tutorials/EB/CNS/Source/hydro/Hyp_gamma_MOL.F90 Tutorials/EB/CNS/Source/hydro/Hyp_gamma_MOL_EB.F90 Tutorials/EB/CNS/Source/hydro/analriem3d.F90 Tutorials/EB/CNS/Source/hydro/cns_eb_hyp_wall.F90 Tutorials/EB/CNS/Source/hydro/slope_mol_3d_gamma.F90 Tutorials/EB/CNS/Source/hydro/slope_mol_3d_gamma_EB.F90 Tutorials/EB/CNS/Source/main.cpp Tutorials/EB/GeometryGeneration/main.cpp Tutorials/EB/MacProj/main.cpp Tutorials/EB/Poisson/main.cpp Tutorials/ForkJoin/MLMG/ff.f90 Tutorials/FortranInterface/Advection_F/Exec/SingleVortex/Prob_2d.f90 Tutorials/FortranInterface/Advection_F/Exec/SingleVortex/Prob_3d.f90 Tutorials/FortranInterface/Advection_F/Exec/SingleVortex/face_velocity_2d.F90 Tutorials/FortranInterface/Advection_F/Exec/SingleVortex/face_velocity_3d.F90 Tutorials/FortranInterface/Advection_F/Source/Src_2d/advect_2d_mod.F90 Tutorials/FortranInterface/Advection_F/Source/Src_2d/compute_flux_2d.f90 Tutorials/FortranInterface/Advection_F/Source/Src_2d/slope_2d.f90 Tutorials/FortranInterface/Advection_F/Source/Src_3d/advect_3d_mod.F90 Tutorials/FortranInterface/Advection_F/Source/Src_3d/compute_flux_3d.f90 Tutorials/FortranInterface/Advection_F/Source/Src_3d/slope_3d.f90 Tutorials/FortranInterface/Advection_F/Source/amr_data_mod.F90 Tutorials/FortranInterface/Advection_F/Source/averagedown_mod.F90 Tutorials/FortranInterface/Advection_F/Source/bc_mod.F90 Tutorials/FortranInterface/Advection_F/Source/compute_dt_mod.F90 Tutorials/FortranInterface/Advection_F/Source/evolve_mod.F90 Tutorials/FortranInterface/Advection_F/Source/fillpatch_mod.F90 Tutorials/FortranInterface/Advection_F/Source/fmain.F90 Tutorials/FortranInterface/Advection_F/Source/initdata.F90 Tutorials/FortranInterface/Advection_F/Source/my_amr_mod.F90 Tutorials/FortranInterface/Advection_F/Source/plotfile_mod.F90 Tutorials/FortranInterface/Advection_F/Source/tagging_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Exec/SingleVortex/Prob.f90 Tutorials/FortranInterface/Advection_octree_F/Exec/SingleVortex/face_velocity_2d.F90 Tutorials/FortranInterface/Advection_octree_F/Source/Src_2d/advect_2d_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/Src_2d/compute_flux_2d.f90 Tutorials/FortranInterface/Advection_octree_F/Source/Src_2d/slope_2d.f90 Tutorials/FortranInterface/Advection_octree_F/Source/amr_data_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/averagedown_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/bc_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/compute_dt_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/evolve_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/fillpatch_mod.F90 Tutorials/FortranInterface/Advection_octree_F/Source/my_amr_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/Prob.f90 Tutorials/FortranInterface/Advection_octree_F2/Exec/SingleVortex/face_velocity_2d.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/advect_2d_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/compute_flux_2d.f90 Tutorials/FortranInterface/Advection_octree_F2/Source/Src_2d/slope_2d.f90 Tutorials/FortranInterface/Advection_octree_F2/Source/amr_data_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/averagedown_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/bc_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/compute_dt_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/evolve_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/fillpatch_mod.F90 Tutorials/FortranInterface/Advection_octree_F2/Source/my_amr_mod.F90 Tutorials/GPU/CNS/Source/CNS.H Tutorials/GPU/CNS/Source/CNS.cpp Tutorials/GPU/CNS/Source/CNS_K.H Tutorials/GPU/CNS/Source/CNS_advance.cpp Tutorials/GPU/CNS/Source/CNS_io.cpp Tutorials/GPU/CNS/Source/CNS_setup.cpp Tutorials/GPU/CNS/Source/hydro/CNS_hydro_K.H Tutorials/GPU/CNS/Source/main.cpp Tutorials/GPU/EBCNS/Source/CNS.H Tutorials/GPU/EBCNS/Source/CNS.cpp Tutorials/GPU/EBCNS/Source/CNS_advance.cpp Tutorials/GPU/EBCNS/Source/CNS_init_eb2.cpp Tutorials/GPU/EBCNS/Source/CNS_io.cpp Tutorials/GPU/EBCNS/Source/CNS_setup.cpp Tutorials/GPU/EBCNS/Source/hydro/CNS_hydro_K.H Tutorials/GPU/EBCNS/Source/main.cpp Tutorials/GPU/Launch/MyKernel_F.F90 Tutorials/GPU/ParallelScan/main.cpp Tutorials/MUI/Source_01/init_phi_3d.f90 Tutorials/MUI/Source_01/main_01.cpp Tutorials/MUI/Source_02/main_02.cpp Tutorials/MUI/doc/MUIcouplingNotes.tex Tutorials/Particles/CellSortedParticles/CellSortedPC.H Tutorials/Particles/CellSortedParticles/CellSortedPC.cpp Tutorials/Particles/CellSortedParticles/cell_sorted_3d.F90 Tutorials/Particles/CellSortedParticles/cell_sorted_F.H Tutorials/Particles/CellSortedParticles/main.cpp Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/EMParticleContainer.cpp Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/em_pic_K.H Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/EMParticleContainer.cpp Tutorials/Particles/ElectromagneticPIC/Exec/OpenACC/em_pic_3d.F90 Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/EMParticleContainer.cpp Tutorials/Particles/ElectromagneticPIC/Source/EMParticleContainer.H Tutorials/Particles/ElectromagneticPIC/Source/EMParticleContainerInit.cpp Tutorials/Particles/ElectromagneticPIC/Source/main.cpp Tutorials/Particles/ElectrostaticPIC/ElectrostaticParticleContainer.H Tutorials/Particles/ElectrostaticPIC/ElectrostaticParticleContainer.cpp Tutorials/Particles/ElectrostaticPIC/electrostatic_pic_2d.f90 Tutorials/Particles/ElectrostaticPIC/electrostatic_pic_3d.f90 Tutorials/Particles/ElectrostaticPIC/electrostatic_pic_F.H Tutorials/Particles/ElectrostaticPIC/main.cpp Tutorials/Particles/NeighborList/CheckPair.H Tutorials/Particles/NeighborList/Constants.H Tutorials/Particles/NeighborList/MDParticleContainer.H Tutorials/Particles/NeighborList/MDParticleContainer.cpp Tutorials/Particles/NeighborList/README.md Tutorials/Particles/NeighborList/main.cpp Tutorials/SDC/MISDC_ADR_2d/Source/SDC_sweeper.cpp Tutorials/SDC/MISDC_ADR_2d/Source/functions_2d.f90 Tutorials/SDC/MISDC_ADR_2d/Source/init_phi_2d.f90 Tutorials/SDC/MISDC_ADR_2d/Source/main.cpp Tutorials/SDC/MISDC_ADR_2d/Source/myfunc.H Tutorials/SDC/MISDC_ADR_2d/Source/myfunc_F.H Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test.H Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test.cpp Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test_F.F90 Tutorials/SWFFT/SWFFT_poisson/SWFFT_Test_F.H Tutorials/SWFFT/SWFFT_poisson/main.cpp Tutorials/SWFFT/SWFFT_poisson/swfft_solver.cpp Tutorials/SWFFT/SWFFT_simple/SWFFT_Test.H Tutorials/SWFFT/SWFFT_simple/SWFFT_Test.cpp Tutorials/SWFFT/SWFFT_simple/SWFFT_Test_F.H Tutorials/SWFFT/SWFFT_simple/main.cpp Tutorials/SWFFT/SWFFT_simple/swfft_compute.cpp commit d84f787548d8e5b89466cd44fbad9d1656e7d08a Author: Weiqun Zhang Date: Sun Mar 14 06:15:13 2021 -0700 Free unused memory (#1861) If Arena is asked to allocate more than what's available in the system, try to free unused memory first. Src/Base/AMReX_Arena.H Src/Base/AMReX_Arena.cpp Src/Base/AMReX_CArena.H Src/Base/AMReX_CArena.cpp commit 85609ec18f8dd3b86eed3bf805e48515e5ce6e7d Author: kngott <30483578+kngott@users.noreply.github.com> Date: Sat Mar 13 17:52:38 2021 -0800 Fortran Interfaces for new FillPatch (#1793) Fortran Interfaces for new FillPatch. Compiles, and I believe the indexing is correct, but didn't test a Fortran implementation. Src/F_Interfaces/AmrCore/AMReX_fillpatch_fi.cpp Src/F_Interfaces/AmrCore/AMReX_fillpatch_mod.F90 Src/F_Interfaces/AmrCore/AMReX_interpolater_mod.F90 commit 9b17f925887a9058a3efb91af26a5ac082d5f2c6 Author: Weiqun Zhang Date: Fri Mar 12 12:14:08 2021 -0800 The_Device_Arena -> The_Arena (#1858) Replace The_Device_Arena with The_Arena in various places. Using The_Arena should help reduce the memory footprint. We used The_Device_Arena in the early days to avoid the penalty of page fault caused by touching managed memory on CPU. We are much better in that regard now. In fact, AMReX itself no longer depends on managed memory. Src/Base/AMReX_BlockMutex.cpp Src/Base/AMReX_CudaGraph.H Src/Base/AMReX_GpuAllocators.H Src/Base/AMReX_GpuFuse.cpp Src/Base/AMReX_GpuMemory.H Src/Base/AMReX_MFIter.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_Scan.H commit 7e31cfd67ad9739b7779ee47d1a2750acf9c2cc1 Author: Weiqun Zhang Date: Fri Mar 12 12:13:42 2021 -0800 Arena Release Threshold (#1857) Release memory from Arena back to the system if it's above a threshold. For the pinned arena, the release threshold is set to the size of the global memory. For others, it's set to a huge number. These can be controlled with ParmParse parameters, `amrex.the_arena_release_threshold` and `the_[device|pinned|managed|async]_release_threshold`. The reason for setting the pinned arena's release threshold to the size of the global memory is that we received a report in the past that the pinned arena used by the communication functions wasted a huge amount of memory in the case that more and more grids were added over the run time. Src/Base/AMReX_Arena.H Src/Base/AMReX_Arena.cpp Src/Base/AMReX_CArena.H Src/Base/AMReX_CArena.cpp Src/Base/AMReX_DArena.H Src/Base/AMReX_DArena.cpp Src/Base/AMReX_EArena.H Src/Base/AMReX_EArena.cpp Src/Base/AMReX_PArena.cpp commit fafaccd7d4a7d0fbe90ec3507402e1988cfd00a1 Author: Maikel Nadolski Date: Fri Mar 12 17:07:57 2021 +0100 Replace is_pod with is_standard_layout && is_trivial (#1856) [`std::is_pod`](https://en.cppreference.com/w/cpp/types/is_pod) is deprecated in C++20 and should be replaced by `std::is_standard_layout && std::is_trivial`. This is the only occurrence of `std::is_pod` that I found in AMReX. If unchanged, this will trigger warnings in future compilers. This triggered a warning in our internal CI when using gcc 10.2 and the `-std=c++20` option. Src/Base/AMReX_GpuAsyncArray.H commit 7d594d2affc9188963ceb55a9ebca597d02462da Author: Maikel Nadolski Date: Thu Mar 11 17:49:00 2021 +0100 Feature/non local parallel copy (#1842) This PR introduces "Multiblock" capabilities to AMReX. Multiblock refers to having multiple distinct computational domains, known as blocks, that have some relation to each other. This PR proposes a generic NonLocalBC::ParallelCopy that performs copies between two FabArrays that might live on different blocks. The ParallelCopy respects - An index mapping such as already present in NonLocalBC (DTOS) - A projection function that will be called on either sender or receiver side to perform actions like e.g. swapping two components or changing the sign of velocity components The proposed usage is ``` NonLocalBC::ParallelCopy(dest, destbox, src, scomp, dcomp, ncomp, ngrow, index_mapping, fab_projection); ``` Notes - In the current implementation, `DTOS` needs to be a map between index spaces where inverse images of boxes are boxes again. - The name `NonLocalBC::ParallelCopy` is confusing if it does something else than `amrex::ParallelCopy`. Alternatives? - Parts of the implementation might be shared with other places in AMReX, such as in communication procedures of FabArray. But this is out of scope. - In addition to `SwapComponents`, a more generic `PermuteComponents` is useful. - The test is not commented yet. Src/Base/AMReX_Extension.H Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBC.cpp Src/Base/AMReX_NonLocalBCImpl.H Src/Base/AMReX_TypeTraits.H Src/Base/CMakeLists.txt Src/Base/Make.package Tests/CMakeLists.txt Tests/MultiBlock/Advection/CMakeLists.txt Tests/MultiBlock/Advection/GNUmakefile Tests/MultiBlock/Advection/Make.package Tests/MultiBlock/Advection/main.cpp Tests/MultiBlock/IndexType/CMakeLists.txt Tests/MultiBlock/IndexType/GNUmakefile Tests/MultiBlock/IndexType/Make.package Tests/MultiBlock/IndexType/main.cpp commit 4893049d2588261f235c1e05ef7d0121a9ad6b6c Author: Pankaj Jha Date: Wed Mar 10 14:35:57 2021 -0800 Include missing library 'pthread' for linking to amrex library (#1854) - [HOTFIX ] Include missing library 'pthread' for linking to amrex library in the tutorial 'Basic/Build_with_libamrex' Tutorials/Basic/Build_with_libamrex/GNUmakefile commit 6ec00666843dcc0ae794358c872dd92e30a3cd34 Author: Andrew Myers Date: Mon Mar 8 17:05:52 2021 -0500 Enable expanding MPI / GPU macros for doxygen. (#1853) Docs/Doxygen/doxygen.conf commit 9001a7486ffcdd68cec9f2c1212559c4982865d8 Author: Weiqun Zhang Date: Mon Mar 8 08:15:45 2021 -0800 Revert the use of codeplay_host_task (#1851) It results int NaNs. Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuElixir.cpp commit fff2224879807f73c5a4acac19609f5630f77a8e Author: asalmgren Date: Sun Mar 7 10:01:11 2021 -0800 Add harmonic averaging (#1850) Mostly adding functionality, but also fixed a bug for 2d multicomponent cc->fc averaging Src/Base/AMReX_MultiFabUtil.H Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_MultiFabUtil_1D_C.H Src/Base/AMReX_MultiFabUtil_2D_C.H Src/Base/AMReX_MultiFabUtil_3D_C.H commit 146f3f2b7596cbee73d2b2eb8d48d4576ecc3f7c Author: Candace Gilet Date: Sat Mar 6 21:36:28 2021 -0500 Fix typo in GNUMake/Make.defs introduced in PR#1846 (#1849) Tools/GNUMake/Make.defs commit 0fc630c5bbc08b65feda01cae000b2314772d240 Author: Weiqun Zhang Date: Fri Mar 5 14:53:57 2021 -0800 AMREX_NO_PROBINIT (#1846) For Amr/AmrLevel based codes, there is now a compile time option AMREX_NO_PROBINIT=[TRUE|FALSE] for the support of the probin file and amrex_probinit function. The default behavior of requiring a user defined amrex_probinit function unless amr.probin_file is empty does not change unless amrex is built into a shared library on non-GNU/Linux system. The reason for the inconsistency is there is no portable way of supporting a weak symbol (.e.g, undefined amrex_probinit in amrex itself) in a shared library. This will partially solve the shared library build issue on Mac and Windows. Co-authored-by: Michele Rosso Docs/sphinx_documentation/source/BuildingAMReX.rst Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp Tools/CMake/AMReXGenerateConfigHeader.cmake Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReX_Config.H.in Tools/GNUMake/Make.defs Tools/libamrex/configure.py commit e7a264ddc0b008310d80c98ecf578e292f3cf13d Author: Maikel Nadolski Date: Fri Mar 5 19:53:28 2021 +0100 Use unique_ptr instead of naked pointer (#1839) This commit replaces naked pointer to char with `std::unique_ptr` in `NonLocalBC::CommHandler`. This will lead to proper cleanup in case of any exceptions. Note, that `Comm_nowait` and `Comm_finish` are *not* marked noexcept. Furthermore, future PRs will add interpolation and some kind of projection for non-local data transactions. Error handling for those is still unclear. Maybe, we want to mark everything noexcept. This is merely an implementation change without nearly any noticable difference in user code. Src/Base/AMReX_Extension.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_NonLocalBCImpl.H commit ef3a757eae988bff7948a47c346a6fb47ffd9ac0 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Fri Mar 5 09:43:54 2021 -0800 Fix early returns in ParallelCopy. (#1845) Fixes a new segfault bug due to pc_src.size() being called when pc_src isn't set. Instead, tracks all ParallelCopy_nowait early returns and checks once at the top of the ParallelCopy_finish. Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayCommI.H commit 6c1296b2466263a5aba46b5df16aceda47457b11 Author: Candace Gilet Date: Fri Mar 5 11:49:42 2021 -0500 Revert "Change default for Amr::probin_file -> "" (#1837)" (#1844) This reverts commit 2519768caedcf46a05b0cfd2b552dff2148465f7. Src/Amr/AMReX_Amr.cpp commit 2fb82bf64838aba949f7a641988a0c58774e5a88 Author: mic84 Date: Thu Mar 4 13:25:13 2021 -0800 CMake: move AMReX_INSTALL option to AMReXOptions.cmake (#1840) * CMake: add missing AMReX_INSTALL option * CMake: remove AMReX_INSTALL from root CMakeLists.txt CMakeLists.txt Tools/CMake/AMReXOptions.cmake commit e01e1c73e670defda422b0f6a7b0a5fc45fb17af Author: Weiqun Zhang Date: Thu Mar 4 12:23:03 2021 -0800 Add EditorConfig (#1841) .editorconfig commit 076be729a1cc8fabc5c18f6836e134f85fa8b0c7 Author: asalmgren Date: Thu Mar 4 08:00:53 2021 -0800 Migrate more of Tutorials/LinearSolvers to Tests/LinearSolvers (#1835) Tests/LinearSolvers/ABecLaplacian_C/CMakeLists.txt Tests/LinearSolvers/ABecLaplacian_C/GNUmakefile Tests/LinearSolvers/ABecLaplacian_C/Make.package Tests/LinearSolvers/ABecLaplacian_C/MyTest.H Tests/LinearSolvers/ABecLaplacian_C/MyTest.cpp Tests/LinearSolvers/ABecLaplacian_C/MyTestPlotfile.cpp Tests/LinearSolvers/ABecLaplacian_C/initProb.cpp Tests/LinearSolvers/ABecLaplacian_C/initProb_K.H Tests/LinearSolvers/ABecLaplacian_C/inputs Tests/LinearSolvers/ABecLaplacian_C/inputs-inhomNeumann Tests/LinearSolvers/ABecLaplacian_C/inputs-rt-abeclap-com Tests/LinearSolvers/ABecLaplacian_C/inputs-rt-poisson-lev Tests/LinearSolvers/ABecLaplacian_C/inputs.hypre Tests/LinearSolvers/ABecLaplacian_C/inputs.petsc Tests/LinearSolvers/ABecLaplacian_C/main.cpp Tests/LinearSolvers/ABecLaplacian_C/scalingtest/inputs.test Tests/LinearSolvers/ABecLaplacian_C/scalingtest/main.diff Tests/LinearSolvers/ABecLaplacian_C/scalingtest/results.org Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-1.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-1024.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-128.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-16.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-2.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-2048.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-256.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-32.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-4.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-512.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-64.sh Tests/LinearSolvers/ABecLaplacian_C/scalingtest/run-8.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/inputs.test Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-1.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-16.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-2.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-32.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-4.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-64.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-mpi-8.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-1.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-16.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-2.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-32.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-4.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-64.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/knl-omp-8.sh Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/main.diff Tests/LinearSolvers/ABecLaplacian_C/threadmultiple_test/results.org Tests/LinearSolvers/ABecLaplacian_F/CMakeLists.txt Tests/LinearSolvers/ABecLaplacian_F/GNUmakefile Tests/LinearSolvers/ABecLaplacian_F/Make.package Tests/LinearSolvers/ABecLaplacian_F/README Tests/LinearSolvers/ABecLaplacian_F/init_prob.F90 Tests/LinearSolvers/ABecLaplacian_F/inputs Tests/LinearSolvers/ABecLaplacian_F/inputs-rt-abeclap-lev Tests/LinearSolvers/ABecLaplacian_F/inputs-rt-poisson-com Tests/LinearSolvers/ABecLaplacian_F/main.F90 Tests/LinearSolvers/ABecLaplacian_F/mytest.F90 Tests/LinearSolvers/MultiComponent/GNUmakefile Tests/LinearSolvers/MultiComponent/MCNodalLinOp.H Tests/LinearSolvers/MultiComponent/MCNodalLinOp.cpp Tests/LinearSolvers/MultiComponent/Make.package Tests/LinearSolvers/MultiComponent/inputs Tests/LinearSolvers/MultiComponent/main.cpp Tests/LinearSolvers/NodalPoisson/CMakeLists.txt Tests/LinearSolvers/NodalPoisson/GNUmakefile Tests/LinearSolvers/NodalPoisson/Make.package Tests/LinearSolvers/NodalPoisson/MyTest.H Tests/LinearSolvers/NodalPoisson/MyTest.cpp Tests/LinearSolvers/NodalPoisson/MyTestPlotfile.cpp Tests/LinearSolvers/NodalPoisson/inputs-rt Tests/LinearSolvers/NodalPoisson/main.cpp Tests/LinearSolvers/Nodal_Projection_EB/CMakeLists.txt Tests/LinearSolvers/Nodal_Projection_EB/GNUmakefile Tests/LinearSolvers/Nodal_Projection_EB/Make.package Tests/LinearSolvers/Nodal_Projection_EB/README Tests/LinearSolvers/Nodal_Projection_EB/inputs_3d Tests/LinearSolvers/Nodal_Projection_EB/main.cpp Tests/LinearSolvers/NodeTensorLap/CMakeLists.txt Tests/LinearSolvers/NodeTensorLap/GNUmakefile Tests/LinearSolvers/NodeTensorLap/Make.package Tests/LinearSolvers/NodeTensorLap/MyTest.H Tests/LinearSolvers/NodeTensorLap/MyTest.cpp Tests/LinearSolvers/NodeTensorLap/MyTestPlotfile.cpp Tests/LinearSolvers/NodeTensorLap/main.cpp Tools/RegressionTesting/AMReX-cuda-tests.ini Tools/RegressionTesting/AMReX-dpcpp-tests.ini Tools/RegressionTesting/AMReX-hip-tests.ini Tools/RegressionTesting/AMReX-tests.ini commit 2519768caedcf46a05b0cfd2b552dff2148465f7 Author: Candace Gilet Date: Thu Mar 4 11:00:18 2021 -0500 Change default for Amr::probin_file -> "" (#1837) Src/Amr/AMReX_Amr.cpp commit 86c58645a181907962bd9a72d7c12089d68c1a3a Author: Weiqun Zhang Date: Wed Mar 3 21:47:44 2021 -0800 Fix BoxArray in doxygen (#1838) Src/Base/AMReX_BoxArray.H commit 0ad9fd9c31e9ddfbbdbb57e659d7864b2f2a6ed8 Author: asalmgren Date: Wed Mar 3 14:35:12 2021 -0800 Move Tutorials/LinearSolvers/MAC_Projection_EB to Tests/LinearSolvers/MAC_Projection_EB (#1834) Tests/LinearSolvers/MAC_Projection_EB/CMakeLists.txt Tests/LinearSolvers/MAC_Projection_EB/GNUmakefile Tests/LinearSolvers/MAC_Projection_EB/Make.package Tests/LinearSolvers/MAC_Projection_EB/README Tests/LinearSolvers/MAC_Projection_EB/inputs_3d Tests/LinearSolvers/MAC_Projection_EB/main.cpp Tools/RegressionTesting/AMReX-cuda-tests.ini Tools/RegressionTesting/AMReX-hip-tests.ini Tools/RegressionTesting/AMReX-tests.ini commit 42c927d1cae12f90a841c07ccd88275d8a66d5cc Author: Axel Huebl Date: Wed Mar 3 09:16:46 2021 -0800 AMReX Install: Config Header (#1833) Found another header that still installed itself :) Tools/CMake/AMReXGenerateConfigHeader.cmake commit dd2ec76a5cc5db01caa17bdf51efb603c019346d Author: Weiqun Zhang Date: Tue Mar 2 10:30:48 2021 -0800 Fix ncomp bug in linear solver (#1832) Need to give m_ncomp a default value, otherwise default constructor followed by define(...) will not work. Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.H commit 28fd2ee530e4c0229f04e33e041989f8535c9d33 Author: Axel Huebl Date: Tue Mar 2 09:41:39 2021 -0800 CMake: AMReX_INSTALL (#1831) Add an `AMReX_INSTALL` option that controls `install()` generation. Such options are typically used in superbuilds by dependent projects to supress the install of AMReX headers, helpers and libs, e.g. if AMReX is fully consumed as private dependency. CMakeLists.txt Docs/sphinx_documentation/source/BuildingAMReX.rst commit b7ddf2d2677fce63a567612978e01ced288dbda2 Author: Weiqun Zhang Date: Mon Mar 1 11:18:19 2021 -0800 Update CHANGES for 21.03 (#1830) CHANGES commit 2794f2f8f89d7461e4d99f19edc805c0feb8d7fe Author: Weiqun Zhang Date: Mon Mar 1 08:24:18 2021 -0800 Fix a sign compare warning (#1827) Src/Base/AMReX_GpuLaunchFunctsG.H commit 0518bb82022da8e6d6250d65b11f7cdaa8b31b24 Author: asalmgren Date: Sun Feb 28 12:58:34 2021 -0800 Ncomp for average cc to fc (#1826) Src/Base/AMReX_MultiFabUtil.H Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_MultiFabUtil_1D_C.H Src/Base/AMReX_MultiFabUtil_2D_C.H Src/Base/AMReX_MultiFabUtil_3D_C.H commit 90efb8b28714a496a3cd9188bdba4794579c59a4 Author: asalmgren Date: Sun Feb 28 09:01:48 2021 -0800 Add n_comp as an optional argument to the constructor of AMReX_MLABecLaplacian (#1825) Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp commit 662231a896991dd4fc110aa2fae6740a1f511c59 Author: Weiqun Zhang Date: Sat Feb 27 08:37:59 2021 -0800 Arena Query Functions and Bug Fix in FArrayBox & IArrayBox (#1823) Add Arena::isDeviceAccessible, isHostAccessible, isManaged, isDevice, and isPinned. Use them in various places. Fix bugs in FArrayBox & IArrayBox that touch device memory allocated by The_Async_Arena() on the host. Src/Base/AMReX_Arena.H Src/Base/AMReX_Arena.cpp Src/Base/AMReX_BArena.H Src/Base/AMReX_BArena.cpp Src/Base/AMReX_BaseFab.H Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_GpuControl.H Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_IArrayBox.cpp Src/Base/AMReX_PArena.H Src/Base/AMReX_PArena.cpp Src/Base/AMReX_VisMF.cpp commit 272e9fc5e93b54f125e907b29e3b9db149c4ae63 Author: Weiqun Zhang Date: Sat Feb 27 08:09:28 2021 -0800 DPCPP: Fix bug in computing the number of active threads (#1822) Thank Intel for the help on debugging! Src/Base/AMReX_GpuLaunchFunctsG.H commit da6eca54acd9fc291fec189a9ee75b54d9727cab Author: mic84 Date: Fri Feb 26 09:51:54 2021 -0800 CMake: add -Wno-pass-failed to Clang-based compilers (#1815) * CMake: add -Wno-pass-failed to Clang-based compilers * CMake: forgot to add flag in one place * CMake: need that flag for Release too Tools/CMake/AMReXFlagsTargets.cmake Tools/CMake/AMReXSYCL.cmake commit f208bc92bc0c9c0b744423cf3cc847db23e5a0ba Author: drangara <69211175+drangara@users.noreply.github.com> Date: Fri Feb 26 11:55:37 2021 -0500 Cover multiple cuts (#1810) ## Summary When building the edge types, cover the cut edges that touch opposite regular corners in order to prevent the `too many cuts in a face error`. Logic is only enabled when flag is set to true in inputs file (default is false). ## Additional background When trying to resolve complex geometries with sharp edges one can run into cases where there are multiple EB cuts on a face. In this change, when the flag `cover_multiple_cuts` is set, we cover appropriate edges in order to avoid this error. Manually tested geometry generation on some complex geometries that were previously failing to resolve. Src/EB/AMReX_EB2_3D_C.H Src/EB/AMReX_EB2_Level.H Src/EB/AMReX_EB2_MultiGFab.H Src/EB/AMReX_EB2_MultiGFab.cpp commit 35d27f7b8ba4ac3b77faadf12af5132ecd1f24cc Author: Weiqun Zhang Date: Fri Feb 26 07:33:30 2021 -0800 Fix sign-compare warnings (#1817) Src/Base/AMReX_DistributionMapping.cpp Src/Base/AMReX_Machine.cpp Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_ParmParse.cpp Src/Base/AMReX_VisMF.cpp Src/EB/AMReX_EBToPVD.cpp Src/EB/AMReX_distFcnElement.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/Particle/AMReX_ParticleCommunication.cpp Src/Particle/AMReX_StructOfArrays.H Tools/GNUMake/comps/gnu.mak commit 8cdf250b04aa1c06ae745133100be2988e9a35aa Author: Erik Schnetter Date: Thu Feb 25 22:52:50 2021 -0500 AMReX_GpuFuse.H: Avoid compiler warning (#1816) Src/Base/AMReX_GpuFuse.H commit a2c92323ee40c46a1848438230ee2b6ac1cfefd8 Author: Erik Schnetter Date: Thu Feb 25 17:03:58 2021 -0500 Avoid compiler warning in FabArray::defined (#1814) Avoid compiler warning in `FabArray::defined`: Convert `size_t` to `int`. Src/Base/AMReX_FabArray.H commit 327c62b150bb23ce01633a4d7ae060291156c62d Author: Weiqun Zhang Date: Wed Feb 24 13:42:23 2021 -0800 Add regression testing script for HIP (#1812) Tools/RegressionTesting/AMReX-hip-tests.ini commit f2e46722e8a6d2a8fd383d268b14579c4bae2929 Author: Weiqun Zhang Date: Wed Feb 24 09:16:25 2021 -0800 Add Axel to the list of core developers (#1811) .zenodo.json CONTRIBUTING.md commit 47389357d5de37d84be69d9b06a294e64cb2a96b Author: Weiqun Zhang Date: Mon Feb 22 09:30:43 2021 -0800 Update docs for GitHub Discussions (#1808) CONTRIBUTING.md Docs/sphinx_documentation/source/index.rst commit 13aa70f0ae5e156b9afdec8ed9f8ef13ba86a0fb Author: Weiqun Zhang Date: Mon Feb 22 08:40:22 2021 -0800 The_Async_Arena and Elixir::append (#1804) Add The_Async_Arena that uses CUDA steam-ordered memory allocator, if available. If the feature is not available, use The_Arena to allocate and free memory and the free is done with a host callback function. The purpose of this Arena is to be used by a temporary FArrayBox inside an MFIter loop. Add Elixir::append that appends an rvalue Elixir to an existing one to reduce the number of host callback calls. Docs/sphinx_documentation/source/GPU.rst Src/Base/AMReX_Arena.H Src/Base/AMReX_Arena.cpp Src/Base/AMReX_BaseFab.H Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuElixir.H Src/Base/AMReX_GpuElixir.cpp Src/Base/AMReX_IArrayBox.H Src/Base/AMReX_IArrayBox.cpp Src/Base/AMReX_PArena.H Src/Base/AMReX_PArena.cpp Src/Base/CMakeLists.txt Src/Base/Make.package commit 110422d41aae2eee79772d890a52de4ee54f1d2c Author: kngott <30483578+kngott@users.noreply.github.com> Date: Sun Feb 21 20:13:08 2021 -0800 Add function name output option to TinyProfiler. (#1803) Use tiny_profiler.verbose=1 to output "Entering " and "Leaving " output. Primarily for hunting down hang issues. Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_TinyProfiler.cpp commit 85cad7492dfa14851cc47ed4c55e4d2a56d6d56c Author: Axel Huebl Date: Fri Feb 19 15:40:58 2021 -0800 CMake: More Particle Tests (#1801) Tests/Particles/AssignDensity/CMakeLists.txt Tests/Particles/AssignMultiLevelDensity/CMakeLists.txt Tests/Particles/AsyncIO/CMakeLists.txt Tests/Particles/AsyncIO/inputs Tests/Particles/GhostsAndVirtuals/CMakeLists.txt Tests/Particles/GhostsAndVirtuals/main.cpp Tests/Particles/InitFromAscii/CMakeLists.txt Tests/Particles/InitFromAscii/main.cpp Tests/Particles/Intersection/CMakeLists.txt Tests/Particles/Intersection/main.cpp Tests/Particles/ParallelContext/CMakeLists.txt Tests/Particles/ParticleArray/CMakeLists.txt Tests/Particles/ParticleIterator/CMakeLists.txt Tests/Particles/SparseBins/CMakeLists.txt Tests/Particles/TypeDescriptor/CMakeLists.txt commit 093275a3bbf26e45faa43971418bac597b8a7b85 Author: mic84 Date: Wed Feb 17 13:36:06 2021 -0800 Extrapolater: make assertion less restrictive (#1799) Src/Amr/AMReX_Extrapolater.cpp commit 86e26ca7bdcb64d5e694f325973541f721c568b9 Author: Weiqun Zhang Date: Wed Feb 17 10:57:36 2021 -0800 DPCPP: Use codeplay_host_task (#1797) In AsyncArray and Elixir, replace streamSynchronize with SYCL 2020's host_task. Note that the function name in the current oneAPI release is codeplay_host_task, because it started as a Codeplay extension. This will probably change in the future, since the extension has been accepted into the standard. Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuElixir.cpp commit d29a5709545b0d44051e82a895be37f5f821cbc6 Author: Weiqun Zhang Date: Wed Feb 17 10:47:06 2021 -0800 Remove functions about the number of host callbacks (#1798) We used to use the number of host callbacks to limit the number of GPU streams. But we no longer do that anymore, and the default number of streams has been reduced to 4. So the functions keeping track of the number of host callbacks can now be removed. Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_GpuElixir.cpp Src/Base/AMReX_MFIter.cpp commit 23c2fa054f21263220d1f1d384889af9593fc330 Author: Andrew Myers Date: Wed Feb 17 10:10:33 2021 -0800 Add ParticleArray classes (#1796) Add a class for storing arrays of particle data that can be toggled between AoS and SoA. Src/Base/AMReX_PODVector.H Src/Particle/AMReX_ParticleArray.H Src/Particle/CMakeLists.txt Src/Particle/Make.package Tests/Particles/ParticleArray/GNUmakefile Tests/Particles/ParticleArray/Make.package Tests/Particles/ParticleArray/main.cpp commit e611121acca7696320f3757982de857cbab74926 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Tue Feb 16 16:43:21 2021 -0800 ParallelCopy_nowait & ParallelCopy_finish (#1765) Most direct implementation of `ParallelCopy_nowait` and `ParallelCopy_finish`, with minimal changes. Previous version had a max component limit per message, set by the input parameter `fabarray.maxcomp`, 25 by default. This is preserved. If `ncomp > fabarray.maxcomp`, `ParallelCopy_nowait` calls `ParallelCopy_finish` internally for all iterations of the loop except the final iteration. For most apps using GPUs that haven't been manually setting `fabarray.maxcomp`, this will almost certainly mean behavior identical to `FillBoundary_nowait`, as `ncomp > 25` would require very small `MultiFabs` to fit on current GPUs. Keeps `ParallelCopy`, which simply calls `ParallelCopy_nowait` followed by `ParallelCopy_finish`. No doubt, there are potential optimizations available: e.g. if `ncomp>fabarray.maxcomp` and `ncomp%fabarray.maxcomp != 0` returning from `nowait` with a message with a full `fabarray.maxcomp` components may allow the largest possible overlap. But, this version gets the feature running while we discuss additional improvements such as this. Plan to follow up with a scaling study on the effect of message size on `ParallelCopy` timing on Summit by exploiting the `fabarray.maxcomp` feature to see if rewriting with more controlled buffer sizes yields enough benefit to justify a rewrite. Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayCommI.H commit 305770490a946deede9eee31ddd3814fe077ff67 Author: Weiqun Zhang Date: Tue Feb 16 14:40:47 2021 -0800 Fix a new bug in GNU Make (#1795) Must set CXXSTD for nvcc because we need to use it later. Tools/GNUMake/comps/nvcc.mak commit 446fb1e9ee4647604a56a68063f64cb7d94cfd50 Author: Weiqun Zhang Date: Tue Feb 16 10:50:25 2021 -0800 Add FabArray::tags() to return the tags. (#1794) Src/Base/AMReX_FabArray.H commit 4a332b5c3875a5eba705ad0d33f76d7f592bef80 Author: Andrew Myers Date: Tue Feb 16 09:30:16 2021 -0800 Bump minimum C++ standard from 11 to 14. (#1787) This is well-supported now and will allow a lot of the code in amrex to be simplified. I have also bumped the version on the Cuda 9 CI test to Cuda 9.2 and GCC 6.5.0, since the old combination did not work with C++14. .github/workflows/dependencies/dependencies_nvcc9.sh .github/workflows/linux.yml Docs/sphinx_documentation/source/BuildingAMReX.rst Docs/sphinx_documentation/source/BuildingAMReX_Chapter.rst INSTALL Tools/CMake/AMReX_Config.cmake Tools/GNUMake/comps/cray.mak Tools/GNUMake/comps/gnu.mak Tools/GNUMake/comps/intel.mak Tools/GNUMake/comps/llvm-flang.mak Tools/GNUMake/comps/nag.mak Tools/GNUMake/comps/nvcc.mak Tools/GNUMake/comps/nvhpc.mak Tools/GNUMake/comps/pgi.mak Tutorials/Basic/Build_with_libamrex/GNUmakefile commit 3ed36ff23c1d408a855573eadcce2686b51160d8 Author: asalmgren Date: Mon Feb 15 15:07:55 2021 -0800 fix table formatting in GPU.rst (#1790) Docs/sphinx_documentation/source/GPU.rst Docs/sphinx_documentation/source/Post_Processing.rst Docs/sphinx_documentation/source/Post_Processing_Chapter.rst commit e8b4d0739e4f21b44169a195a7aa40eaf0092c1b Author: asalmgren Date: Mon Feb 15 15:03:15 2021 -0800 Add an interface to the flux redistribution that takes the arguments (#1789) as Array4s rather than FABs or MultiFabs. Docs/sphinx_documentation/source/Post_Processing.rst Src/EB/AMReX_EB_utils.H Src/EB/AMReX_EB_utils.cpp commit 15a84595c5b88f4e289be505869786400bac49e3 Author: Max Katz Date: Mon Feb 15 15:00:58 2021 -0800 Update the preferred short name of NVIDIA HPC SDK to nvhpc (#1788) `nvhpc` is the preferred short name of the NVIDIA HPC SDK (for example when used in modules). Tools/GNUMake/Make.defs Tools/GNUMake/comps/nvhpc.mak commit 492c5524375858d46f4dd6ea97589cffef749c42 Author: Weiqun Zhang Date: Mon Feb 15 14:58:59 2021 -0800 Fix sphinx config for sphinx 3.5.0 (#1792) https://github.com/sphinx-doc/sphinx/issues/8885 Docs/sphinx_documentation/source/conf.py Docs/sphinx_tutorials/source/conf.py commit df5bbe922563bf3ae9eb97a566014205e1a1b2f4 Author: Weiqun Zhang Date: Thu Feb 11 10:07:38 2021 -0800 BoxArray::simplified() & simplified_list() (#1786) Make them public so that they can be used by application codes to regenerate grids covering exactly the same region but with a different max grid size. However, it should be emphasized that they only work with regular BoxArrays without anything like coarsening. Src/Base/AMReX_BoxArray.H commit 7742e1c99f5e7a5ac7a4fa3530dba3a30d51daeb Author: mic84 Date: Wed Feb 10 15:21:10 2021 -0800 CMake: bump up minimum required HYPRE version (#1785) Tools/CMake/AMReXThirdPartyLibraries.cmake commit 932836ad422ad742a7c3c9b6375bad9cffec76d7 Author: asalmgren Date: Tue Feb 9 18:17:01 2021 -0800 fix bug in EB redistribution which was accidentally including (#1784) contributions from outside a non-periodic domain Src/EB/AMReX_EB_utils.cpp commit d2c88194fac2b9e35e469cf332034bedbf361cf9 Author: Axel Huebl Date: Tue Feb 9 00:02:41 2021 -0800 Windows: WINDOWS_EXPORT_ALL_SYMBOLS (#1782) I am seeing some linker issues downstream when building Python bindings with WarpX on Windows. This nice option makes sure we don't need to add dll export symbols all over the place when building a shared AMReX library (dll). Kudos to the ROOT team at CERN for contributing this feature years ago to CMake. Tools/CMake/AMReXBuildInfo.cmake Tools/CMake/AMReX_Config.cmake commit 6219fa1df9b1980327da3f40bbcf4457a93a6602 Author: Weiqun Zhang Date: Mon Feb 8 18:35:42 2021 -0800 Fix precision issue in ParmParse::add (#1783) When adding values to ParmParse, we need to call `std::setprecision` so as not to lose precision. Src/Base/AMReX_ParmParse.cpp commit ba0f1c0e77515debd83fe651abd3045d87567dab Author: Weiqun Zhang Date: Mon Feb 8 14:57:30 2021 -0800 Fix bug in #1774 (#1779) A bug was introduced in #1774 that resulted in incorrect weights when making nodal RHS from cell-centered RHS. Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H commit d44e353bb9f7745973502e4d7b16bbde3d29ed09 Author: Andrew Myers Date: Mon Feb 8 11:45:59 2021 -0800 Fix up neighbor list for 2D (#1781) Src/Particle/AMReX_NeighborList.H commit 591f48db3c0539fa639e8d846cd4a036a7e63c53 Author: Weiqun Zhang Date: Sat Feb 6 17:38:19 2021 -0800 Remove managed memory from CNS (#1780) Tutorials/GPU/CNS/Exec/RT/cns_prob.cpp Tutorials/GPU/CNS/Exec/RT/cns_prob_parm.H Tutorials/GPU/CNS/Exec/Sod/cns_prob.cpp Tutorials/GPU/CNS/Exec/Sod/cns_prob_parm.H Tutorials/GPU/CNS/Source/CNS.H Tutorials/GPU/CNS/Source/CNS.cpp Tutorials/GPU/CNS/Source/CNS_advance.cpp Tutorials/GPU/CNS/Source/CNS_derive.cpp Tutorials/GPU/CNS/Source/CNS_parm.H Tutorials/GPU/CNS/Source/CNS_setup.cpp commit d31f9b4a36a1cb912d4dd52bb1807117fe9d79f6 Author: Weiqun Zhang Date: Fri Feb 5 19:17:43 2021 -0800 Overset Solver with Refinement Ratio of 4 (#1778) We need to define overset mask for on coarse MG levels of fine AMR levels. We didn't need to do it for refinement ratio of 2 because there are no coarse MG levels on fine AMR levels. Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp commit eb9b4242a07028a77bd95986f5a5686c81dce428 Author: Andrew Myers Date: Thu Feb 4 12:11:37 2021 -0800 Fix a circular include involving PODVector. (#1776) Src/Base/AMReX_PODVector.H commit 5e08d985d3a91ab8ec10d9aec8c318b38ff10849 Author: Andrew Myers Date: Thu Feb 4 12:09:15 2021 -0800 add missing template arguments to the particle HDF5 IO methods (#1777) Src/Particle/AMReX_ParticleHDF5.H commit f8808fcb56e172f9548954240468cf3d18223851 Author: Weiqun Zhang Date: Thu Feb 4 10:28:54 2021 -0800 Refinement Ratio of 4 Support in Nodal Solver (#1774) * Refinement Ratio of 4 Support in Nodal Solver This adds refinement ratio of 4 support in non-EB nodal linear solver. Roundoff errors are expected for multi-level solves with a refinement ratio of 2. * MacProjector refinement ratio of 4 bug Move setLevelBC to project and call it only if it has not been called. This fixes a refinement ratio of 4 bug, because setLevelBC should not be called before coarse/fine ratio is set. Src/Base/AMReX_Box.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLMG_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp Src/LinearSolvers/Projections/AMReX_MacProjector.H Src/LinearSolvers/Projections/AMReX_MacProjector.cpp Tutorials/LinearSolvers/NodalPoisson/MyTest.H Tutorials/LinearSolvers/NodalPoisson/MyTest.cpp commit c686e458ffccfa0032f231753c65ea514df8f122 Author: Candace Gilet Date: Wed Feb 3 23:48:45 2021 -0500 For FillPatchTwoLevels, BCs should be set on the coarse box (#1772) since they are only used on the coarse level (when computing slopes). This corrects certain cases where a level of refinement is very close to, but not actually touching an ext_dir or hoextrap boundary. Src/Amr/AMReX_AmrLevel.cpp Src/AmrCore/AMReX_FillPatchUtil_I.H commit ebf64aa13ff7d57013473bdd4df0acf7be144193 Author: Michael Zingale Date: Wed Feb 3 23:34:09 2021 -0500 remove f90cat.py (#1775) it was only ever used with Castro as an experiment Tools/F_scripts/f90cat.py commit c9265f4c85057e94b39db780df42491f20137d19 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Wed Feb 3 16:06:45 2021 -0800 Make.unknown MPI Detection Update (#1770) Simplify/robust-ify Make.unknown's MPI detection. Tools/GNUMake/sites/Make.unknown commit 39fe5870a75b7dfd9f3b0cdc531ab63f92a9f15b Author: Axel Huebl Date: Wed Feb 3 13:45:56 2021 -0800 MSVC: Proper __cplusplus macro (#1773) Modern MSVC versions finally set the __cplusplus macro correctly, but we need to request this with an additional flag. Tools/CMake/AMReX_Config.cmake commit b00475d69aec7304d18129d6b3ce9c2a2f4d835a Author: Michael Zingale Date: Tue Feb 2 16:03:12 2021 -0500 remove any trailing / from HIP_PATH via realpath (#1771) Tools/GNUMake/comps/hip.mak commit c68493447deeaf1675a6cdfef323d0adb62f4bbe Author: Andrew Myers Date: Tue Feb 2 08:55:09 2021 -0800 Use ParIter instead of MakeMFIter in WriteBinaryParticleDataAsync. (#1767) Src/Particle/AMReX_WriteBinaryParticleData.H commit d2270fab703098760806d9b9297a7502e8918733 Author: Axel Huebl Date: Tue Feb 2 08:35:34 2021 -0800 IntVect::toArray() (#1768) ## Summary Add a ::toArray() conversion member to IntVect that can be used to convert back to an array type and cast the result on the way, e.g. to unsigned. ## Additional background Functions in https://github.com/ECP-WarpX/WarpX/pull/1660 Src/Base/AMReX_IntVect.H commit 4a425335e40fbae69261987e8a97bc094712fb9f Author: Weiqun Zhang Date: Mon Feb 1 08:37:34 2021 -0800 Update CHANGES for 21.02 (#1766) CHANGES commit 466e3071713ced99cdfce4426c283d75da21b2c5 Author: Andrew Myers Date: Sat Jan 30 13:21:11 2021 -0800 Add a Github action that tries to build tutorials with SP particles and DP mesh. (#1764) Currently this combination is only caught by the Nyx CI. .github/workflows/linux.yml commit 302fad36faaa8f5bb65c207c608ca332b1a07116 Author: Candace Gilet Date: Sat Jan 30 16:18:05 2021 -0500 Abort when MLMG is detected as failing regardless of verbosity setting. (#1762) Move Abort() outside if( verbose >0 ) block. Src/LinearSolvers/MLMG/AMReX_MLMG.cpp commit f649f04fef6922374f2d2b31474ac87eda35a5f6 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Sat Jan 30 13:16:45 2021 -0800 Anyof (#1757) * DPC++ versions of AnyOf. Written to match CUDA/HIP version. Performance not explicitly tested. * Also, adjustment to AMREX_DEVICE_PRINTF to remove the need for AMREX_DEVICE_COMPILE. Src/Base/AMReX_GpuPrint.H Src/Base/AMReX_Reduce.H Tests/GPU/AnyOf/GNUmakefile Tests/GPU/AnyOf/Make.package Tests/GPU/AnyOf/inputs Tests/GPU/AnyOf/main.cpp commit 896916428ca28b012c7c4ed53812898f2c26f815 Author: jmsexton03 Date: Fri Jan 29 13:32:24 2021 -0800 Use ParticleReal for particle interpolation (#1763) Src/Particle/AMReX_TracerParticles.cpp commit adbb5695469183897014c05824bd44772252f398 Author: Andrew Myers Date: Fri Jan 29 11:59:32 2021 -0800 Fix new logic error with aggregation_type. (#1761) Src/Particle/AMReX_ParticleContainerBase.cpp commit ef3e68bbc1380065f9515ea5c0236fe2f18cda5f Author: Andrew Myers Date: Fri Jan 29 11:51:30 2021 -0800 Remove obsolete interpolation functions from AMReX (#1754) Src/Particle/AMReX_Particle.H Src/Particle/AMReX_TracerParticle_mod_K.H Src/Particle/AMReX_TracerParticles.cpp commit 5df10f661e060de45fca677026325fc278b5c70c Author: Andrew Myers Date: Fri Jan 29 11:06:53 2021 -0800 Move more PC methods that don't need the template parameters into base class. (#1760) Src/Particle/AMReX_ParticleContainerBase.H Src/Particle/AMReX_ParticleContainerBase.cpp Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_Particles.H commit f45ede1d49940035f84dc9a8562fef0a2ad3b720 Author: Andrew Myers Date: Thu Jan 28 13:28:26 2021 -0800 make pc.Geom always refer to the particle one. (#1470) Src/Particle/AMReX_Particles.H commit c143555e74e4096ec25400369651bf884b8561b3 Author: Andrew Myers Date: Wed Jan 27 16:04:37 2021 -0800 Fix 'potential null dereference warning' in StructOfArrays (#1759) Src/Particle/AMReX_StructOfArrays.H commit 2cd12635b7d7c2d41ee5ebce07c8003ef7fbe1b0 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Wed Jan 27 14:31:30 2021 -0800 Gpu Docs Update First Pass (#1758) First pass at updating the GPU docs. More detailed updates to follow. Docs/sphinx_documentation/source/GPU.rst Docs/sphinx_documentation/source/GPU_Chapter.rst commit 0b79c5f7d855ed0078b63bfe34cf982c0956f06e Author: Andrew Myers Date: Wed Jan 27 09:29:29 2021 -0800 Move DataPrefix and Version... (#1756) * DataPrefix and Version do not need the template arguments, so moving them to a base class. Src/Particle/AMReX_Particle.H Src/Particle/AMReX_ParticleContainerBase.H Src/Particle/AMReX_ParticleContainerBase.cpp Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleInit.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_WriteBinaryParticleData.H Src/Particle/CMakeLists.txt Src/Particle/Make.package commit b95a0f30003d6eb97fb6e33a5fde5a902221495d Author: Andrew Myers Date: Wed Jan 27 09:19:47 2021 -0800 Remove unused move random methods. (#1755) These methods are not used, and if needed, they should be implemented at the application level. Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_Particles.H commit 02c52cda1b70c77384807d898a7a5c1f705c96c4 Author: Andrew Myers Date: Tue Jan 26 08:50:02 2021 -0800 Add IntVect versions of numParticlesOutOfRange. (#1751) Src/Particle/AMReX_ParticleUtil.H commit 9896c7643f72bc091b119e63ab6f46e5d1a058d2 Author: Axel Huebl Date: Mon Jan 25 16:52:20 2021 -0800 Fix ParIter Ctor (#1752) With user-defined allocators, this lead to: ``` Src/Particle/AMReX_ParIter.H:114:84: error: type ‘amrex::ParIterBase’ is not a direct base of ‘amrex::ParIter<0, 0, 4, 0, amrex::PinnedArenaAllocator>’ 114 | : ParIterBase(pc,level) | ``` in non-const contexts. Src/Particle/AMReX_ParIter.H commit b560557c4f5876337c0eb47df82f0ead37cf52b6 Author: asalmgren Date: Mon Jan 25 13:40:42 2021 -0800 Improve the interpolation from cell centroids to face centroids: (#1750) 1) if the centroids are aligned tangentially to the face but not at equal distance to the face, use just those two cells for interpolation 2) in the case where the face centroid is at 0 but the cell centroids are not aligned, instead of testing on an area fraction tangential to the face to decide which way to go, in 2d, now test on the minimum volume for each option and choose the option with the larger minimum volume -- this should avoid using covered or very small cells. (3D already has logic to avoid using covered cells) Src/EB/AMReX_EBMultiFabUtil_2D_C.H Src/EB/AMReX_EBMultiFabUtil_3D_C.H commit 3069f1ba4dd899f40ac6092fff2916466af8d980 Author: asalmgren Date: Mon Jan 25 13:40:25 2021 -0800 Fix EB slopes in 3D when two of the domain boundaries with ext_dir bc's meet at an edge -- (#1749) this case was handled incorrectly before Src/EB/AMReX_EB_slopes_K.H commit e93c5705a6c013ffc602a16e877d22ea3afbf1fc Author: Don E. Willcox Date: Sun Jan 24 20:23:00 2021 -0800 In ParticleToMesh and MeshToParticle, use nGrowVect() to accommodate different a different number of grow cells in each dimension. (#1748) Src/Particle/AMReX_ParticleMesh.H commit 162bca40a4ecec77a5d8d1df68b1864bfd15ec6a Author: asalmgren Date: Sat Jan 23 12:50:54 2021 -0800 Add additional tests when interpolating from cell centroids to face centroids (#1746) when the face centroid is exactly at the center of the face we need an additional test to know which way we can look transversely and not hit covered cells. Src/EB/AMReX_EBMultiFabUtil_2D_C.H Src/EB/AMReX_EBMultiFabUtil_3D_C.H commit cbbd1c5bd623fc59b67a96e1d5cd5339f2b24cbb Author: Weiqun Zhang Date: Fri Jan 22 09:40:57 2021 -0800 Fix single precision build of CNS (#1745) Tutorials/GPU/CNS/Source/main.cpp commit e2b1108dde79e258f4d06e584e745e0ffd2bbd5f Author: Andrew Myers Date: Thu Jan 21 17:37:54 2021 -0800 Add missing template parameters in ParIter.H. (#1744) Src/Particle/AMReX_ParIter.H commit 1656d55a6e001f139f88c5ada93068a7b862b59c Author: Andrew Myers Date: Thu Jan 21 15:23:39 2021 -0800 Template copy/add particles based on src type. (#1741) Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleTile.H Src/Particle/AMReX_Particles.H commit a63941a8f13fd972b860297de2132185a739418e Author: Axel Huebl Date: Thu Jan 21 15:00:09 2021 -0800 HIP: New Warp Size Macro (#1742) Use new the HIP macro for the warp size on various AMD devices. Src/Base/AMReX_GpuDevice.H commit c2beebd492f1efb202d09cef2f62e5283998e1b7 Author: Andrew Myers Date: Wed Jan 20 17:21:05 2021 -0800 Add allocator template parameter to ParticleContainer (#1416) Src/AmrCore/AMReX_AmrParticles.H Src/Base/AMReX_TypeTraits.H Src/Particle/AMReX_ParIter.H Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleInit.H Src/Particle/AMReX_ParticleTile.H Src/Particle/AMReX_Particles.H commit bcf034a796031d362b85dba9c88fad24f30b36e6 Author: asalmgren Date: Wed Jan 20 15:21:05 2021 -0800 Reduce compiler warnings (#1740) Co-authored-by: Andrew Myers Src/EB/AMReX_EB_slopes_K.H Src/Particle/AMReX_NeighborList.H commit b2c1b6077a07a23201aa65156ef2a1033943482f Author: Andrew Myers Date: Wed Jan 20 11:44:24 2021 -0800 Fix some new warnings about 'pointless comparisons' with 0 for unsigned types. (#1737) Src/Particle/AMReX_StructOfArrays.H commit 1775ce65c769d0502f727aa1f1955366a172ff2c Author: drangara <69211175+drangara@users.noreply.github.com> Date: Wed Jan 20 14:16:39 2021 -0500 Add ability to calculate slopes on face centroids (#1707) And apply the explicit stencil when computing the laplacian These changes allow the computation of EB viscous laplacian operator when phi is defined on cell centroids by using the Least squares method for calculating slopes on face centroids. Currently only works when Dirichlet BC is set. Src/EB/AMReX_EB_LeastSquares_2D_K.H Src/EB/AMReX_EB_LeastSquares_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_3D_K.H Tests/LinearSolvers/LeastSquares/GNUmakefile Tests/LinearSolvers/LeastSquares/Make.package Tests/LinearSolvers/LeastSquares/MyEB.H Tests/LinearSolvers/LeastSquares/MyTest.H Tests/LinearSolvers/LeastSquares/MyTest.cpp Tests/LinearSolvers/LeastSquares/README.md Tests/LinearSolvers/LeastSquares/initData.cpp Tests/LinearSolvers/LeastSquares/initEB.cpp Tests/LinearSolvers/LeastSquares/initPoiseuilleData.cpp Tests/LinearSolvers/LeastSquares/initPoiseuilleDataFor2D.cpp Tests/LinearSolvers/LeastSquares/initPoiseuilleDataFor3D.cpp Tests/LinearSolvers/LeastSquares/inputs.2d.askew-x Tests/LinearSolvers/LeastSquares/inputs.2d.askew-y Tests/LinearSolvers/LeastSquares/inputs.2d.base Tests/LinearSolvers/LeastSquares/inputs.2d.fullyrotated Tests/LinearSolvers/LeastSquares/inputs.2d.other Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.aligned.xy-x Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.aligned.xy-y Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.aligned.xz-x Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.aligned.xz-z Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.aligned.yz-y Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.aligned.yz-z Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.askew-all Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.askew-xy Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.askew-xz Tests/LinearSolvers/LeastSquares/inputs.3d.poiseuille.askew-yz Tests/LinearSolvers/LeastSquares/main.cpp commit 9bda4f80dea74dcef0ccd4cc9dfa60c693613228 Author: Weiqun Zhang Date: Wed Jan 20 09:53:21 2021 -0800 HIP indirect function and kernel fusing (#1739) Add a GNU Make flag, HIP_INDIRECT_FUNCTION=[TRUE|FALSE], and modify the Gpu kernel fusing test. Currently HIP does not support indirect function calls via device function pointers, which our GPU fusing implementation depends on. Src/Base/AMReX_GpuFuse.H Src/Base/AMReX_GpuFuse.cpp Src/Base/AMReX_GpuLaunch.H Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_GpuLaunchMacrosG.H Src/Base/AMReX_Reduce.H Tests/GPU/Fuse/GNUmakefile Tests/GPU/Fuse/main.cpp Tools/GNUMake/Make.defs commit 9437e36505e115da96cebebab973d5499f357bb2 Author: Robert Maynard Date: Tue Jan 19 13:39:04 2021 -0500 When using CMake 3.15+ allow for MSVC runtime library changes (#1736) Enabling of CMP0091 needs to go before the first `project` call as that computes if MSVC_RUNTIME_LIBRARY information is used CMakeLists.txt Tools/CMake/AMReX_Config.cmake commit 5791c90593389dbf23502d10c52f01b35e77bfa6 Author: Weiqun Zhang Date: Mon Jan 18 20:49:38 2021 -0800 RoundRobin: option to not sort processes (#1721) Add an optional parameter `sort` to RoundRobin constructors. This makes it consistent with other DistributionMapping strategies. And this fixes an incorrect behavior when RoundRobin is used by KnapSack that has been given `sort=false`. This should be able to fix the hanging issue observed in WarpX simulations with load balancing enabled and more MPI processes than the number of boxes. Src/Base/AMReX_DistributionMapping.H Src/Base/AMReX_DistributionMapping.cpp commit 5d3e31a08af987cf9e127ab07375483c74724743 Author: Weiqun Zhang Date: Sun Jan 17 12:02:00 2021 -0800 EB: edge centroid (#1730) Add edge centroid to the database and make it available via EBFArrayBoxFactory. Src/EB/AMReX_EB2_2D_C.H Src/EB/AMReX_EB2_2D_C.cpp Src/EB/AMReX_EB2_3D_C.H Src/EB/AMReX_EB2_3D_C.cpp Src/EB/AMReX_EB2_C.H Src/EB/AMReX_EB2_GeometryShop.H Src/EB/AMReX_EB2_IF_Rotation.H Src/EB/AMReX_EB2_Level.H Src/EB/AMReX_EB2_Level.cpp Src/EB/AMReX_EB2_ND_C.cpp Src/EB/AMReX_EBDataCollection.H Src/EB/AMReX_EBDataCollection.cpp Src/EB/AMReX_EBFabFactory.H Src/EB/AMReX_EB_utils.cpp Src/EB/CMakeLists.txt Src/EB/Make.package commit 8b123354e9929bc2515b032c2069e2d924e88e59 Author: Weiqun Zhang Date: Sun Jan 17 11:58:36 2021 -0800 No Fortran needed for building plotfile tools (#1711) Tools/Plotfile/GNUmakefile commit d2ff769fa0ad57243a3c9d0010eb68e2ab049005 Author: Emmanuel Motheau Date: Sun Jan 17 11:57:30 2021 -0800 fix RegridOnly to regrid even with single level grids (#1722) Src/Amr/AMReX_Amr.cpp commit 832ce5e8bcdd224c8946392dfaeb8915e0e02eed Author: Andy Nonaka Date: Sat Jan 16 11:54:48 2021 -0800 Consolidate agglomerate (#1733) a few lines in the docs about consolidation and agglomeration options in the linear solvers Docs/sphinx_documentation/source/LinearSolvers.rst commit eab09708761036232dd650c5ee724af7f28249f4 Author: Axel Huebl Date: Sat Jan 16 11:52:55 2021 -0800 Doc: Break into Debugger (#1732) Describe how to disable AMReX' signal and backtrace handling. Thanks to @sayerhs for pointing me to the right place (again :) ). Docs/sphinx_documentation/source/Basics.rst commit f6b0abc7c32368bafffe8e81a2de18c492c8971a Author: Max Katz Date: Sat Jan 16 11:50:44 2021 -0800 Only add Hypre interfaces to MLMG in 2D/3D (#1734) This is necessary for compiling with both Hypre and MLMG in 1D. Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLMG.H Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.cpp commit 889af2f8a8e1ec191627f1fb5b28ac9ed6d5aa9a Author: Max Katz Date: Fri Jan 15 18:13:37 2021 -0800 Don't include Hypre files in GNU make for 1D (#1731) These files cannot be compiled in 1D, so there is no point in adding them. CMake treats this as fatal, but for Castro we would like to use the AMReX Hypre GNU make package even without the actual interfaces. Src/Extern/HYPRE/Make.package commit 549bb3d5675f7cac9fd2d116f8cde1daaa477a09 Author: Andrew Myers Date: Fri Jan 15 17:05:38 2021 -0800 Always assert that tiling is false in RedistributeGPU (#1729) Src/Particle/AMReX_ParticleContainerI.H commit 55b62cb42fb6afcfd2e78c8d68fab80c39180a9a Author: Axel Huebl Date: Fri Jan 15 16:33:01 2021 -0800 Define: _OPENMP -> AMREX_USE_OMP (#1560) Replace the define `_OPENMP` with `AMREX_USE_OMP` for all parallel "backend" implementations and control of MFIter loops. This avoids accidentially enabling OpenMP for the parallel compute components, i.e. when a users explicitly uses OpenMP in auxiliary or implicit dependent functionality, but does not request to also OpenMP-parallelize those sections. Co-authored-by: Weiqun Zhang Docs/sphinx_documentation/source/AmrCore.rst Docs/sphinx_documentation/source/Basics.rst Docs/sphinx_documentation/source/GPU.rst Src/Amr/AMReX_Amr.cpp Src/Amr/AMReX_AmrLevel.cpp Src/Amr/AMReX_Extrapolater.cpp Src/Amr/AMReX_StateData.cpp Src/Amr/AMReX_StateDescriptor.cpp Src/AmrCore/AMReX_AmrCore.cpp Src/AmrCore/AMReX_ErrorList.cpp Src/AmrCore/AMReX_FillPatchUtil.H Src/AmrCore/AMReX_FillPatchUtil.cpp Src/AmrCore/AMReX_FillPatchUtil_I.H Src/AmrCore/AMReX_FluxRegister.cpp Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX.cpp Src/Base/AMReX_BLBackTrace.H Src/Base/AMReX_BLBackTrace.cpp Src/Base/AMReX_BLProfiler.cpp Src/Base/AMReX_BaseFab.H Src/Base/AMReX_BaseFab.cpp Src/Base/AMReX_BaseUmap.H Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp Src/Base/AMReX_BoxList.cpp Src/Base/AMReX_DistributionMapping.cpp Src/Base/AMReX_Extension.H Src/Base/AMReX_FBI.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_Geometry.H Src/Base/AMReX_Geometry.cpp Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_GpuUtility.cpp Src/Base/AMReX_MFIter.H Src/Base/AMReX_MFIter.cpp Src/Base/AMReX_MemPool.cpp Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_MultiFabUtil.H Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_NonLocalBCImpl.H Src/Base/AMReX_OpenMP.H Src/Base/AMReX_PCI.H Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelDescriptor.cpp Src/Base/AMReX_PhysBCFunct.H Src/Base/AMReX_Print.H Src/Base/AMReX_Random.cpp Src/Base/AMReX_Reduce.H Src/Base/AMReX_TinyProfiler.cpp Src/Base/AMReX_Utility.cpp Src/Base/AMReX_iMultiFab.cpp Src/Base/AMReX_omp_mod.F90 Src/Boundary/AMReX_BndryRegister.cpp Src/Boundary/AMReX_FabSet.cpp Src/Boundary/AMReX_InterpBndryData.cpp Src/Boundary/AMReX_MultiMask.H Src/Boundary/AMReX_MultiMask.cpp Src/Boundary/AMReX_YAFluxRegister.cpp Src/EB/AMReX_EB2_Level.H Src/EB/AMReX_EB2_Level.cpp Src/EB/AMReX_EBAmrUtil.cpp Src/EB/AMReX_EBCellFlag.cpp Src/EB/AMReX_EBFluxRegister.cpp Src/EB/AMReX_EBMultiFabUtil.cpp Src/EB/AMReX_EB_utils.H Src/EB/AMReX_EB_utils.cpp Src/EB/AMReX_MultiCutFab.cpp Src/EB/AMReX_algoim.cpp Src/Extern/HYPRE/AMReX_HypreABecLap.cpp Src/Extern/HYPRE/AMReX_HypreABecLap2.cpp Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/Extern/PETSc/AMReX_PETSc.cpp Src/Extern/ProfParser/AMReX_BLProfStats.cpp Src/Extern/ProfParser/AMReX_CommProfStats.cpp Src/Extern/ProfParser/AMReX_RegionsProfStats.cpp Src/Extern/hpgmg/BL_HPGMG.cpp Src/F_Interfaces/Octree/AMReX_octree_fi.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLALaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLCGSolver.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLMGBndry.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp Src/Particle/AMReX_NeighborParticlesCPUImpl.H Src/Particle/AMReX_NeighborParticlesI.H Src/Particle/AMReX_ParIter.H Src/Particle/AMReX_Particle.H Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleMesh.H Src/Particle/AMReX_ParticleReduce.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_TracerParticles.cpp Tests/FillBoundaryComparison/main.cpp Tests/LinearSolvers/CellEB2/MyTest.cpp Tests/LinearSolvers/CellOverset/MyTest.cpp Tests/LinearSolvers/MLMG/init_prob.cpp Tests/LinearSolvers/TensorOverset/MyTest.cpp Tests/Particles/ParticleIterator/main.cpp Tools/Postprocessing/C_Src/IntegrateComp.cpp Tutorials/Amr/Advection_AmrCore/Source/AdvancePhiAllLevels.cpp Tutorials/Amr/Advection_AmrCore/Source/AdvancePhiAtLevel.cpp Tutorials/Amr/Advection_AmrCore/Source/AmrCoreAdv.H Tutorials/Amr/Advection_AmrCore/Source/AmrCoreAdv.cpp Tutorials/Amr/Advection_AmrCore/Source/DefineVelocity.cpp Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.H Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tutorials/Blueprint/CellSortedParticles/CellSortedPC.cpp Tutorials/EB/CNS/Source/CNS.cpp Tutorials/EB/CNS/Source/CNS_advance.cpp Tutorials/EB/Poisson/Poisson.cpp Tutorials/GPU/CNS/Source/CNS.cpp Tutorials/GPU/EBCNS/Source/CNS.cpp Tutorials/LinearSolvers/ABecLaplacian_C/initProb.cpp Tutorials/LinearSolvers/MultiComponent/MCNodalLinOp.cpp Tutorials/LinearSolvers/NodalPoisson/MyTest.cpp Tutorials/LinearSolvers/NodeTensorLap/MyTest.cpp Tutorials/Particles/CellSortedParticles/CellSortedPC.cpp commit 2536f1ae9f7b7e73bbeee71d5642aa22e02432fc Author: Robert Maynard Date: Fri Jan 15 14:12:12 2021 -0500 When using CMake 3.15 allow for MSVC runtime library changes (#1724) Tools/CMake/AMReX_Config.cmake commit 40dcbf1f69cb81fc875e4e1db49d33c3292bbe9f Author: Mark Meredith Date: Fri Jan 15 12:04:44 2021 -0500 fix -Werror=sign-compare (#1699) Src/Particle/AMReX_NeighborParticlesCPUImpl.H Src/Particle/AMReX_NeighborParticlesI.H Src/Particle/AMReX_StructOfArrays.H commit 633c971c3258d4a662f91d5620e44834bf8d4c92 Author: asalmgren Date: Thu Jan 14 17:25:58 2021 -0800 Quiet the compiler warnings coming from EB/AMReX_EB_slopes_K.H (#1728) Src/EB/AMReX_EB_slopes_K.H commit fd6f5ce3c255d449d5de3d664269d64d069cbdc9 Author: mic84 Date: Thu Jan 14 15:55:57 2021 -0800 CMake: link tests exes against alias library (#1727) Tests/CMakeLists.txt commit 873c4801ad9b1f262a8865f6b9fe198a4ead29ca Author: Jon Rood Date: Thu Jan 14 15:36:17 2021 -0700 Update Make.nrel to accomodate MPT MPI with CUDA enabled (#1726) For our main computer at NREL, Eagle, we mostly use HPE's MPT for MPI. Until these changes, CUDA did not work with MPT when using the GNU makefiles. Tools/GNUMake/Make.defs Tools/GNUMake/sites/Make.nrel commit bcbdb65b525644fcb1cd61dc57d4e77c19b8eac6 Author: Weiqun Zhang Date: Thu Jan 14 13:38:11 2021 -0800 Fix CNS tutorial for HIP (#1725) Use AMREX_GPU_HOST_DEVICE instead of AMREX_GPU_DEVICE because of we cannot detect the lambda type in HIP. Tutorials/GPU/CNS/Source/CNS.cpp commit 10f3a2348e8cb4d0f52e90d273f70426ef1fcbbc Author: Weiqun Zhang Date: Thu Jan 14 11:14:01 2021 -0800 Cell-centered Hypre and Petsc setup on GPU (#1709) Do cell-centered hypre and Petsc solver setup on GPU. Docs/sphinx_documentation/source/LinearSolvers.rst Src/Base/AMReX.cpp Src/Base/AMReX_BaseFabUtility.H Src/Extern/HYPRE/AMReX_Habec_2D_K.H Src/Extern/HYPRE/AMReX_Habec_3D_K.H Src/Extern/HYPRE/AMReX_Habec_K.H Src/Extern/HYPRE/AMReX_Hypre.H Src/Extern/HYPRE/AMReX_HypreABecLap.H Src/Extern/HYPRE/AMReX_HypreABecLap.cpp Src/Extern/HYPRE/AMReX_HypreABecLap2.H Src/Extern/HYPRE/AMReX_HypreABecLap2.cpp Src/Extern/HYPRE/AMReX_HypreABecLap3.H Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/Extern/PETSc/AMReX_PETSc.H Src/Extern/PETSc/AMReX_PETSc.cpp Src/LinearSolvers/MLMG/AMReX_MLMG.H Tools/GNUMake/packages/Make.hypre Tutorials/LinearSolvers/ABecLaplacian_C/initProb.cpp commit dd0a3ceb3c62f68ad096b0692352acc776c352ae Author: mic84 Date: Wed Jan 13 12:59:28 2021 -0800 Tests: replace D_DECL with AMREX_D_DECL (#1723) Tests/HDF5Benchmark/main.cpp Tests/Particles/AssignMultiLevelDensity/main.cpp Tests/Particles/AsyncIO/main.cpp Tests/Particles/GhostsAndVirtuals/main.cpp Tests/Particles/Intersection/main.cpp Tests/Particles/Redistribute/main.cpp Tests/Particles/SparseBins/main.cpp commit 07488476533e1bf4731f1bd7c17c1b14c0fd13d2 Author: Matt Larsen Date: Mon Jan 11 13:35:37 2021 -0800 fixing assert (#1720) Co-authored-by: Matt Larsen Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp commit 95661e553fbd92c4d41ffa88438272a3ca4f385f Author: Andrew Myers Date: Mon Jan 11 11:38:54 2021 -0800 Fix compiler warnings in CreateVirtualParticles (#1718) Src/Particle/AMReX_ParticleContainerI.H commit f35b856849f22909e2e4395ac73edb263b108434 Author: Andrew Myers Date: Mon Jan 11 11:17:50 2021 -0800 Fix compiler warnings in ParticleInit.H (#1719) Src/Particle/AMReX_ParticleInit.H commit cd56f543e5372230314f1f0acfb6566a754376ed Author: Weiqun Zhang Date: Sun Jan 10 12:46:38 2021 -0800 ParmParse::hasUnusedInputs & getUnusedInputs (#1716) Add functions to find if there are unused `ParmParse` parameters starting with `prefix.`. If `prefix` is empty, all unused `ParmParse` parameters are included. Src/Base/AMReX_ParmParse.H Src/Base/AMReX_ParmParse.cpp commit cf17e95632b8bc26107a9ecd9e8dcae75db47c4e Author: asalmgren Date: Sat Jan 9 13:39:48 2021 -0800 There is no longer a CVODE tutorial (#1715) Docs/sphinx_tutorials/source/CVODE_Tutorial.rst Docs/sphinx_tutorials/source/index.rst commit 4e7f200eb91b1d8689b38f31d2ea1e29535094c0 Author: mic84 Date: Fri Jan 8 17:00:27 2021 -0800 CMake: offer option to exclude Src/Amr from build (#1714) Docs/sphinx_documentation/source/BuildingAMReX.rst Src/CMakeLists.txt Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXOptions.cmake commit 42f045ea6d39b7a1f02b2a92612352cff585a9b7 Author: Weiqun Zhang Date: Fri Jan 8 16:45:55 2021 -0800 Always have Abort_device etc. defined for GPU build (#1712) It used to be these functions are defined only if NDEBUG is defined. However, the cmake build system does not store `NDEBUG` in `AMReX_Config.H. It's possible that AMReX is built with `NDEBUG` and `Abort_device` is not defined, but the user's code may be built without `NDEBUG`. Then `amrex::Abort` defined in `AMReX.H` would fail to compile. Always having these device functions defined can solve the issue. Src/Base/AMReX.H Src/Base/AMReX.cpp commit 147934bcca2551e24c16bb0b5d4a50575a8a2eec Author: Weiqun Zhang Date: Fri Jan 8 14:57:07 2021 -0800 Prefetch not supported on Windows (#1713) On-demand page migration is not supported on Windows. Src/Base/AMReX_BaseFab.H commit 7eb484bb8fdaec3c46f66921b9baefe7a3b00580 Author: Emmanuel Motheau Date: Fri Jan 8 13:10:39 2021 -0800 Update write plot file to ascii tool (#1700) A flag (fast = 1) has been added to the WritePlotfileToAscii tool to propose to export data via Amrex::Print() instead of the 4 i,j,k,n imbricated loops. Indeed, on large plotfiles, exporting data to an ascii file is extremely slow with the procedure originally implemented in the tool. Tools/Postprocessing/C_Src/WritePlotfileToASCII.cpp commit 3daa8ced947e315f8c3cf7506e2eec45c6f8fbf4 Author: Weiqun Zhang Date: Fri Jan 8 10:34:38 2021 -0800 Tweak some floating point comparisons (#1710) Src/AmrCore/AMReX_FillPatchUtil_I.H Src/Base/AMReX_RealBox.H Src/Base/AMReX_RealBox.cpp Src/EB/AMReX_EB_utils.cpp commit 6a773594766d842c866cd3cbb74f2fe6f09884d6 Author: Weiqun Zhang Date: Thu Jan 7 14:48:37 2021 -0800 Make bisect more robust (#1708) If the lower and upper bounds are almost equal, break out of the bisect iteration. Src/Base/AMReX_Algorithm.H Src/Base/AMReX_Geometry.cpp commit be30569c55ce9ecf1f6576b4d94cd9040b3d8490 Author: Weiqun Zhang Date: Thu Jan 7 14:17:12 2021 -0800 Support single precision build with configure (#1706) * Add new arguments `--single-precision` and `--single-precision-particles` to `configure`. * Add amrex::almostEqual for comparing floating point numbers. * Fix double to float conversion warnings. * Add CI testing single precision build .github/workflows/linux.yml Src/Amr/AMReX_Amr.cpp Src/Amr/AMReX_AmrLevel.cpp Src/Amr/AMReX_AuxBoundaryData.cpp Src/Amr/AMReX_StateData.H Src/Amr/AMReX_StateData.cpp Src/Amr/AMReX_extrapolater_3D_K.H Src/AmrCore/AMReX_AmrCore.cpp Src/AmrCore/AMReX_Cluster.H Src/AmrCore/AMReX_ErrorList.cpp Src/Base/AMReX_Algorithm.H Src/Base/AMReX_BaseFab.H Src/Base/AMReX_BoxList.cpp Src/Base/AMReX_COORDSYS_2D_C.H Src/Base/AMReX_CoordSys.cpp Src/Base/AMReX_DistributionMapping.cpp Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_FilCC_C.cpp Src/Base/AMReX_Geometry.cpp Src/Base/AMReX_MFCopyDescriptor.cpp Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_ParallelDescriptor.cpp Src/Base/AMReX_RealBox.cpp Src/Base/AMReX_VisMF.cpp Src/Boundary/AMReX_InterpBndryData_3D_K.H Src/Boundary/AMReX_MacBndry.cpp Src/F_Interfaces/AmrCore/AMReX_FlashFluxRegister.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLALap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp_K.H Src/LinearSolvers/MLMG/AMReX_MLMG.H Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLMGBndry.cpp Src/LinearSolvers/MLMG/AMReX_MLMG_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLTensor_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensor_3D_K.H Src/LinearSolvers/Projections/AMReX_MacProjector.cpp Src/LinearSolvers/Projections/AMReX_NodalProjector.H Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleInit.H Src/Particle/AMReX_TracerParticle_mod_K.H Src/Particle/AMReX_TracerParticles.cpp Tests/FillBoundaryComparison/main.cpp Tools/C_util/WritePlotFile.cpp Tools/libamrex/configure.py Tutorials/Amr/Advection_AmrCore/Source/main.cpp Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tutorials/Amr/Advection_AmrLevel/Source/main.cpp Tutorials/Basic/HeatEquation_EX1_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX1_CF/Source/main.cpp Tutorials/Basic/HeatEquation_EX2_C/Source/main.cpp Tutorials/Basic/HeatEquation_EX2_CF/Source/main.cpp Tutorials/Basic/HeatEquation_EX3_C/Source/main.cpp Tutorials/Blueprint/HeatEquation_EX1_C/Source/main.cpp Tutorials/EB/CNS/Source/CNS_advance.cpp Tutorials/EB/CNS/Source/main.cpp Tutorials/GPU/CNS/Source/main.cpp Tutorials/GPU/EBCNS/Source/main.cpp Tutorials/LinearSolvers/MAC_Projection_EB/main.cpp Tutorials/LinearSolvers/Nodal_Projection_EB/main.cpp commit f84b7b891cfa55affc25426288b4d3881e4be2a7 Author: Andrew Myers Date: Wed Jan 6 15:34:03 2021 -0800 Fix logic in applying periodic bcs when subcycling is on (#1705) Src/Particle/AMReX_ParticleUtil.H commit 20119b405e406040637daff2f4a591670de9093d Author: Axel Huebl Date: Wed Jan 6 11:55:01 2021 -0800 CMake<3.18: Fix Linker Regression (#1701) Roll-back to explicit `-Wl,` linker flags for CMake<3.18 Tools/CMake/AMReX_Config.cmake commit 887a4ce2a3e995592d9dcfd4318682526c297546 Author: Weiqun Zhang Date: Wed Jan 6 10:13:23 2021 -0800 ParallelDescriptor::ReduceReal wrappers (#1703) Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelDescriptor.cpp commit c36b253b295032f83e13edd9de78c40e0e660a10 Author: Luca Fedeli Date: Tue Jan 5 17:47:19 2021 +0100 Fix several warnings related to double-->float conversions (#1698) ## Summary I tried to compile `WarpX` in single precision and I noticed several warnings related to casts from `double` to `float` in `AMReX`. This PR should fix several of these warnings. ## Additional background I compiled `WarpX` with `make DIM=3 PRECISION=FLOAT USE_SINGLE_PRECISION_PARTICLES=TRUE` using the compiler `g++ (Ubuntu 10.2.0-13ubuntu1) 10.2.0` Src/Base/AMReX_FilCC_3D_C.H Src/Base/AMReX_MultiFabUtil_3D_C.H Src/LinearSolvers/MLMG/AMReX_MLMG.H Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleIO.H commit 6c6924e66f26586d03d4be661739e7ab600986fa Author: Robert Maynard Date: Tue Jan 5 11:21:23 2021 -0500 Support when CMake 3.18 policy 105 is set to new (#1696) When AMReX is using a newer version of CMake (3.18+) it sets policy `105` to new, causing `target_link_options` rules to be applied to CUDA device linking. When that happens we need to make sure that host only linking flags are excluded. Tools/CMake/AMReX_Config.cmake commit 8aa9810b22c37855f6b0602403138e7ffcea4a7b Author: kngott <30483578+kngott@users.noreply.github.com> Date: Tue Jan 5 08:20:42 2021 -0800 Update FabReduce-isum test in ParallelReduce tutorial. (#1695) Tutorials/GPU/ParallelReduce/main.cpp commit 00c116c09110fb93428624da5cf5a7f0cdaabe43 Author: Matt Larsen Date: Tue Jan 5 08:09:58 2021 -0800 changing blueprint ghost type to int (#1694) This MR changes the type of the ghost field used by conduit blueprint from `double` to `int`. Ascent expects integers, and this reduces the storage space required for the field. Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp commit 5baf0f7fee46a02caa349d4f8aaae26c192acd05 Author: Axel Huebl Date: Mon Jan 4 16:10:03 2021 -0800 CI: Brew Returns Non-Zero If Already Installed (#1697) Homebrew has the unusual behavior to return a non-zero error code in `brew install` if the package is already installed. Since the base image in CI is somewhat fluid and also the status that `brew update` leaves is very dynamic, we just change this ourselves now to be a non-error. .github/workflows/dependencies/dependencies_mac.sh commit 7eafbe22b93d3d9404efd1de5ea830be4915e372 Author: Weiqun Zhang Date: Sun Jan 3 10:12:03 2021 -0800 Optimized function for filling BaseFab of GpuArray (#1692) Add amrex::fill for BaseFab of GpuArray (a.k.a. AoS). The GPU implementation uses shared memory. Tests show it's more than 10x faster than a naive implementation for GpuArray. Also added are a few functions in DPC++ so that I do not have to look up the documentation every time I need to use things like threadIdx and blockIdx. Src/Base/AMReX_BaseFabUtility.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_GpuTypes.H Src/Base/AMReX_MultiFabUtil.H commit 693700e6afb62df0531601cb48c2a5ab272784ea Author: Weiqun Zhang Date: Sat Jan 2 09:42:15 2021 -0800 BaseFab::minmax (#1691) Add BaseFab::minmax function and use it in VisMF. Src/Base/AMReX_BaseFab.H Src/Base/AMReX_VisMF.cpp commit 9ce1e4e19e64d7ae57d49a4dd0d17aab5a77a376 Author: Emmanuel Motheau Date: Sat Jan 2 09:40:45 2021 -0800 New feature in the fextract tool to directly export in the csv format (#1634) This PR adds a feature to the fextract tool to directly export in the csv format. Docs/sphinx_documentation/source/Post_Processing.rst Tools/Plotfile/fextract.cpp commit 3e38f80ba5e42cd66d98692edd4cef7075661a5b Author: Weiqun Zhang Date: Sat Jan 2 09:25:05 2021 -0800 Update CHANGES (#1693) CHANGES commit 63a567f7b00e7e7592686a8268ba796c3b886481 Author: Weiqun Zhang Date: Mon Dec 28 09:23:53 2020 -0800 DPCPP: GNU Make USE_GPU variable (#1689) Set USE_GPU=TRUE for dpcpp in Make.defs just like cuda and hip. Tools/GNUMake/Make.defs commit 6309a8b7a142d9bdcaedf9e95df15d6db8d9a8f5 Author: Weiqun Zhang Date: Sat Dec 26 12:20:00 2020 -0800 Optimization of MLCellLinOp::applyBC (#1690) Launch one kernel per box to work on all faces and all components. If the number of boxes exceeds the kernel fusing threshold, the kernels from all boxes will be fused. .github/workflows/linux.yml Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX_Box.H Src/Base/AMReX_GpuContainers.H Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.H Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.cpp Src/Particle/AMReX_ParticleTransformation.H commit 8bcc31cf0b4319afe633c1c791cdbb30b7bd7118 Author: Max Katz Date: Thu Dec 24 09:12:05 2020 -0800 Remove old launch macros (#1687) Removes some old macros that were used exclusively for the GPU pragma. Also removes `box_threads_and_blocks()` which was only used by that functionality. Src/Base/AMReX_BLFort.H Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp commit d3fd6462b288249b992baa5f4e9c03d6d8945979 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Wed Dec 23 17:12:26 2020 -0800 unsigned long long iMultiFab sum. (#1684) Changes iMultiFab sum to use hardware supported unsigned long long. Using ParallelReduce, this changes sum time on Cori (default CUDA, 10.1) from 0.07546 to 0.001185, 63x increase. Src/Base/AMReX_iMultiFab.cpp commit 021a9a3d9a5678984afd64fdffdea0c5d4912c44 Author: Weiqun Zhang Date: Wed Dec 23 14:44:23 2020 -0800 Sanitize the input of mkconfig (#1685) The mkconfig.py script takes CPPFLAGS and generates AMReX_Config.H. If CPPFLAGS contains -I flags (as advised by homebrew) and the input is not santized, it would generate ill-formed code. Tools/libamrex/mkconfig.py commit b0977fd7aadb9021196807e9ffe1c8bfa2aaa53c Author: Andy Nonaka Date: Wed Dec 23 14:38:33 2020 -0800 WritePlotfileToASCII Dimensionality Checking (#1686) Added dimensionality error checking to WritePlotfileToASCII.cpp Also supports older FBoxLib plotfile types now. Tools/Postprocessing/C_Src/WritePlotfileToASCII.cpp commit c1713daf9e6ea60b54eb201ab0b18161a2e28c28 Author: Weiqun Zhang Date: Wed Dec 23 14:22:42 2020 -0800 Gpu::Reduce cleanup (#1681) Remove unused parameter in blockReduce_partial. Move the test for full block into Gpu::Handler class. Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_GpuTypes.H commit cf82b6bcd2fdae99c1d78e856e4ee06bcb762995 Author: Weiqun Zhang Date: Wed Dec 23 14:20:28 2020 -0800 New versions of Reduce::Min, Max and MinMax taking callable (#1679) ## Summary Previously, Reduce::Min, Max and MinMax take a pointer to the data. New versions these functions taking a callable object is added to provide more flexibility. Note that Reduce::Sum already has the callable version. Also modify the implementation of the CPU versions of these functions to use OpenMP reduction instead of atomic when we can. Add optional initial value to some functions. Fix the order of template parameters for consistence. ## Checklist The proposed changes: - [x] add new capabilities to AMReX Src/Base/AMReX_Reduce.H commit c98be7422796a0e29cb539f805d91d19c2a5ba01 Author: Johannes Blaschke Date: Wed Dec 23 11:43:48 2020 -0800 update SumNeighbors to reflect recent Particle API changes (#1680) Src/Particle/AMReX_NeighborParticlesCPUImpl.H commit 7e48821802e61c999dcd6bdc7c0471d82e2fee55 Author: Weiqun Zhang Date: Wed Dec 23 10:28:22 2020 -0800 Gpu::AsyncArray -> Gpu::Buffer (#1676) because there is no need for a host callback function to free memory. Src/Base/AMReX_FabArrayUtility.H commit d080afba2402f0f46d03d80ab6d3264ef1968bad Author: Max Katz Date: Tue Dec 22 20:31:14 2020 -0800 Only issue prefetch if arena uses managed memory (#1678) ## Summary Before issuing a managed memory prefetch we check that the arena uses managed memory. This allows the CUDA implementation to still work with `amrex.the_arena_is_managed=0` while calling this API. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX Src/Base/AMReX_BaseFab.H Src/Base/AMReX_FabArrayUtility.H commit 88abc1e88eb34f7f7db0ecf110d75347a9d384a0 Author: Weiqun Zhang Date: Tue Dec 22 15:22:00 2020 -0800 CUDA CI: add flags to catch common errors (#1677) ## Summary Add `--Werror ext-lambda-captures-this` (nvcc 11 only) and `--Werror cross-execution-space-call` to CUDA CI tests. .github/workflows/linux.yml commit 0f820789c57213729b6598f3fa912bd751916dea Author: Weiqun Zhang Date: Tue Dec 22 15:19:49 2020 -0800 GNU Make: Fix the library flags (#1641) Previously for GPU build, the GNU Make system was very sensitive on which variable to use for adding extra `libraries`. Before a certain location, `LIBRARIES` should be used, but after that `libraries` should be used. This is very error prone. This PR tries to fix this. Now the user should be able to use `LIBRARIES` anywhere. The use of `libraries` should be avoided because it is still the case that it can only be used after a certain location in the make file. Nevertheless, this PR is backward compatible and it does not force the use of `LIBRARIES`. Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules commit a4a5adc298abf8ff6b74306fea7448fdff8c7502 Author: Weiqun Zhang Date: Tue Dec 22 14:54:48 2020 -0800 DPCPP: add sync to mimic CUDA NULL stream behavior (#1675) Src/Base/AMReX_GpuDevice.H commit 6c18f6c72f7a965aeed920aba3bf3b6bd6cc64d0 Author: Axel Huebl Date: Tue Dec 22 14:51:07 2020 -0800 NVCC 11.2: CUDA/C++17 (#1665) This adds CI coverage for C++17 builds with NVCC. Several apps expressed interest to transition to C++17 code-bases in the near future and this adds test coverage. Co-authored-by: Weiqun Zhang .github/workflows/linux.yml Src/Particle/AMReX_NeighborList.H Src/Particle/AMReX_ParticleLocator.H Tutorials/GPU/CNS/Source/CNS.cpp commit c8d01767451b03df74afb6f3dfd3da609ca1ffdb Author: Andrew Myers Date: Tue Dec 22 14:31:51 2020 -0800 Remove noexcept from this functor to work around cuda 11 bug. (#1674) Also uses variadic form of `amrex::max`. Tutorials/Particles/NeighborList/MDParticleContainer.cpp commit c777f74b02667f8c9a4929d00f7fb9e835900c53 Author: Weiqun Zhang Date: Tue Dec 22 13:13:36 2020 -0800 PolymorphicArray4 (#1643) PolymorphicArray4 is a thin layer around Array4. It allows us to access both AoS and SoA data with operator(). For example, constexpr int ncomp = 2; Box box(...); BaseFab soa_fab(box, ncomp); auto soa_array = makePolymorphic(soa_fab.array()); int i = ...; int j = ...; int k = ...; soa_array(i,j,k,0) = 0; soa_array(i,j,k,1) = 1; BaseFab> aos_fab(box,1); auto aos_array = makePolymorphic(aos_fab.array()); aos_array(i,j,k,0) = 0; aos_array(i,j,k,1) = 1; Src/Base/AMReX_Array.H Src/Base/AMReX_Array4.H commit afda0382ee0598ea0018894c0a3495c693225bc2 Author: Weiqun Zhang Date: Tue Dec 22 09:33:54 2020 -0800 Remove the check on ref ratio being <= 12 (#1672) Src/Amr/AMReX_Amr.cpp Src/AmrCore/AMReX_AmrMesh.cpp commit 1a8e3e70691a4cdf2183a455d13f2ac529705535 Author: Axel Huebl Date: Mon Dec 21 22:44:25 2020 -0800 CI: OpenMP on Windows (Clang) (#1671) - fix MSVC entry back to MSVC - add OpenMP to Clang .github/workflows/windows.yml commit 6680398a1a4f00424a35cd6d989ffa0f22963197 Author: Weiqun Zhang Date: Mon Dec 21 21:01:01 2020 -0800 Unsigned long long for counting number of particles on GPU (#1670) Because long does not have native atomicAdd support, it's much slower than doing reduction using unsigned long long. Src/Particle/AMReX_ParticleContainerI.H commit c9a571f79d44daba2cae2a4be743be8e29ea348d Author: mic84 Date: Mon Dec 21 18:49:32 2020 -0800 CMake: dowstream projects must enable Fortran if AMReX_FORTRAN=ON (#1669) * CMake: disable Fortran by default * CMake: dowstream projects must enable Fortran if AMReX_FORTRAN=ON Docs/sphinx_documentation/source/BuildingAMReX.rst Src/CMakeLists.txt Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXOptions.cmake commit 4e8386ecf2a5575c7e5e7c7d8c741d4f361bb658 Author: Johannes Blaschke Date: Mon Dec 21 16:16:12 2020 -0800 fix neighbor particle 'GetNeighbors' return type (#1668) Src/Particle/AMReX_NeighborParticles.H commit 754451b9325690e63d0ab45f1d68ba1bae5e284a Author: kngott <30483578+kngott@users.noreply.github.com> Date: Mon Dec 21 14:36:16 2020 -0800 FaceLinear fused interpolater variation (with performance comparison!) (#1663) Add a `interp_arr` version of `FaceLinear` to test generality of the new API (and performance comparison, as `FaceDivFree `stencil can't be used in regular API for comparison). Also includes a bit of `interp_arr` cleanup. Src/AmrCore/AMReX_Interpolater.H Src/AmrCore/AMReX_Interpolater.cpp commit 7b9eb82bec95a5c0e729b3086d614cbdfe2bd71b Author: Andrew Myers Date: Mon Dec 21 14:24:42 2020 -0800 Allow alternate form of "check_pair" function (#1667) This allows users to use a different form of `check_pair` functor that passes the particle indices. This facilitates the creation of "half" neighbor lists. Src/Particle/AMReX_NeighborList.H commit 10ed0e029b6278a8dcf2e4af46623a53f00f1641 Author: Weiqun Zhang Date: Mon Dec 21 13:42:47 2020 -0800 ParallelFor for Reduction (#1658) Add capability for `ParallelFor` to safely do reduction using `deviceReduceSum`, `Min`, etc. The user passes `Gpu::KernelInfo{}.setReduction(true)` to notify `ParallelFor` that this is a parallel reduction, and gives `ParallelFor` a callable that takes `Gpu::Handler`. A `Gpu::Handler` is needed to call `deviceReduceSum`. Also add `Gpu::Buffer` class, whose data pointer can be used as a device destination for `deviceReduceSum`. It also has a `copyToHost` method to copy the device result back to the host. See `Tutorials/GPU/ParallelReduce` for examples of how to use `ParallelFor` for reduction. Also note that the reduction function is OpenMP CPU threads safe. Thus the same code can run on with OpenMP when it is not built for GPU. Co-authored-by: Andrew Myers Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX.cpp Src/Base/AMReX_Box.H Src/Base/AMReX_Gpu.H Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuAsyncArray.cpp Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuBuffer.H Src/Base/AMReX_GpuFuse.H Src/Base/AMReX_GpuKernelInfo.H Src/Base/AMReX_GpuLaunchFunctsC.H Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_GpuTypes.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_Scan.H Src/Base/CMakeLists.txt Src/Base/Make.package Src/EB/AMReX_EB2_GeometryShop.H Tutorials/GPU/ParallelReduce/GNUmakefile Tutorials/GPU/ParallelReduce/main.cpp commit c8cdfa61f443ecde21debc3ad52665bf48eded8b Author: Andrew Myers Date: Sat Dec 19 11:32:26 2020 -0800 Revert tiling change in #1655 (#1664) Src/Particle/AMReX_ParticleContainerI.H commit 875ebff912b5d3f4edc0e6ad7a983a56374a9e69 Author: Axel Huebl Date: Fri Dec 18 20:36:28 2020 -0800 CI: CUDA 9.1 & 11.0 (#1653) Add a recent CUDA 11 release for cutting-edge coverage, too. .github/workflows/dependencies/dependencies_nvcc11.sh .github/workflows/dependencies/dependencies_nvcc9.sh .github/workflows/linux.yml commit 25d06e82804e3e1b4f3f1b0cea4e9979ed37373c Author: Axel Huebl Date: Fri Dec 18 15:29:58 2020 -0800 CI: Lib + Plotfiles Install (#1662) Test the new install of plotfile tools works in CI. .github/workflows/linux.yml commit cc61f30f4266407225bd8c7354565931b275a07f Author: mic84 Date: Fri Dec 18 13:56:37 2020 -0800 CMake: streamline install of plotfile tools (#1661) CMakeLists.txt Docs/sphinx_documentation/source/BuildingAMReX.rst Src/CMakeLists.txt Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXInstallHelpers.cmake Tools/Plotfile/CMakeLists.txt commit 1c291c0b6bb52be354b77b2abb646216dcb62ff7 Author: kngott <30483578+kngott@users.noreply.github.com> Date: Fri Dec 18 10:39:38 2020 -0800 Add new clean commands to docs. (#1660) Docs/sphinx_documentation/source/BuildingAMReX.rst commit 3b897e1b7e04222ae718c00010085908f7693b41 Author: mic84 Date: Thu Dec 17 17:14:36 2020 -0800 Fix how fextract handle the tolerance options (#1659) Tools/Plotfile/fextract.cpp commit de1701ed9d6cbdf61a8c4c3015c1cf5a9b00f7f5 Author: jmsexton03 Date: Thu Dec 17 17:00:36 2020 -0500 AssignDensity and AssignCellDensitySingleLevel copy -> ParallelCopy (#1633) Src/AmrCore/AMReX_AmrParticles.H Src/Particle/AMReX_ParticleContainerI.H commit 07a21fd454317b7b434240d548765c4c4ae4464a Author: Andrew Myers Date: Thu Dec 17 13:29:53 2020 -0800 Have AssignCellDensitySingleLevel respect gpu launch guards. (#1655) Src/Particle/AMReX_ParticleContainerI.H commit e21901db47b85704e29b3800fcad7318b9544e76 Author: Weiqun Zhang Date: Thu Dec 17 12:27:08 2020 -0800 Clean up GPU pragma and CUDA Fortran (#1656) Clean up the remaining GPU pragma and CUDA Fortran. This also fixes a bug in #1650 for codes using the Fortran 77 version of filcc. The proposed changes: - [x] fix a bug or incorrect behavior in AMReX Docs/sphinx_documentation/source/GPU.rst Docs/sphinx_documentation/source/GPU_Chapter.rst Src/Base/AMReX_BCUtil.cpp Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_PhysBCFunct.H Src/Base/AMReX_PhysBCFunct.cpp Src/Base/AMReX_filcc_mod.F90 Src/Base/AMReX_fort_mod.F90 Src/Base/Make.package Tools/GNUMake/Make.defs Tools/GNUMake/comps/hpcsdk.mak Tools/GNUMake/comps/ibm.mak Tools/GNUMake/comps/pgi.mak Tutorials/GPU/Launch/Readme.md commit 63db5cbe23e2ddc419e6895ec69610388519ca9d Author: mic84 Date: Thu Dec 17 11:14:35 2020 -0800 Add features to fextract (#1657) Add the following features to `fextract`: - user option to set the number of decimals in the output of reals - user option to set a cut-off tolerance: anything smaller than the tolerance gets printed as zero - user option to disable the job summary report at the end of the output The job summary report is disabled by default in contrast with the current behaviour. Tools/Plotfile/fextract.cpp commit 1f8891e49f23397b5633b34dae4437619d3b46a8 Author: Andrew Myers Date: Thu Dec 17 10:40:55 2020 -0800 Fix parallel add bug when both USE_GPU and USE_OMP are defined. (#1654) We cannot assume copies are thread safe any time AMREX_USE_GPU is defined, whether or not USE_OMP is also on. Note that this only would have been an issue when compiling with OMP but running on only one thread. The proposed changes: - [x] fix a bug or incorrect behavior in AMReX Src/Base/AMReX_FabArrayBase.cpp commit e63397244cd19a67d1e76fdedce73667d41e8f2c Author: Andy Nonaka Date: Thu Dec 17 10:38:23 2020 -0800 Write a plotfile to ASCII (#1638) This utility reads in a single-level plotfile, copies it to a MultiFab with a single box, then writes out all the data in 'i j k comp ' format to the screen The user can modify this cpp file to write out on certain components, coordinates, row/column formatting, etc. Usage: ./WritePlotfileToASCII2d.gnu.MPI.ex infile=inputFileName Docs/sphinx_documentation/source/Post_Processing.rst Docs/sphinx_documentation/source/Post_Processing_Chapter.rst Docs/sphinx_documentation/source/index.rst Tools/Postprocessing/C_Src/GNUmakefile Tools/Postprocessing/C_Src/WritePlotfileToASCII.cpp commit 7b24139e3214b417b54f8d6238131dc484bc6e66 Author: mic84 Date: Wed Dec 16 12:31:01 2020 -0800 CMake: AMRDATA needs Fortran (#1645) Tools/CMake/AMReXOptions.cmake commit 75373313e44803c3732096a2dee413b52ed2d62e Author: mic84 Date: Wed Dec 16 12:30:36 2020 -0800 CMake: fix few quirks with GPU backends variables (#1561) Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXOptions.cmake commit b851ce78a2f3f26d53eb101aa0a0266284f44bfb Author: kngott <30483578+kngott@users.noreply.github.com> Date: Wed Dec 16 12:23:39 2020 -0800 FaceLinear: Avoid accessing face outside of coarse box. (#1651) ## Summary Fix an index out-of-bound trigger in FaceLinear interpolater. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/AmrCore/AMReX_Interp_1D_C.H Src/AmrCore/AMReX_Interp_2D_C.H Src/AmrCore/AMReX_Interp_3D_C.H commit 08e27d80f3cb0151566b05f964fb4fc30296f56c Author: Axel Huebl Date: Wed Dec 16 12:09:48 2020 -0800 CI: fix macOS (#1652) * CI: fix macOS brew did again some updates and things fail. let's revert our current work-around. * Brew: Add gfortran not pulled automatically anymore .github/workflows/dependencies/dependencies_mac.sh commit d7b6b97354c02f2e37761f8afae0fca36e7a4550 Author: Candace Gilet Date: Wed Dec 16 14:13:52 2020 -0500 Fix NodalProjector::getGradPhi for multilevel case (#1642) ## Summary Two bug fixes for multilevel in NodalProjector: 1. Ensure NodalProjector::getGradPhi() returns the correct result. Previously could return \sum{sigma grad phi} / (2**dim sigma_coarse). 2. Average down the complete projected velocity at the end. For alpha!=1, this ensures the projected velocity is averaged down correctly. This could change the results of multilevel regression tests. If the application uses the affected outputs from NodalProjector as is, then changes are expected. If the application subsequently averages down the outputs, then changes greater than roundoff level are not expected. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp commit 70b7ab9999522a51666027ae3f7f3dc527541290 Author: Max Katz Date: Wed Dec 16 09:10:42 2020 -0800 Remove GPU pragma support from GNU make build system (#1650) This is no longer used. Tools/F_scripts/gpu_fortran.py Tools/F_scripts/write_cuda_headers.py Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules Tools/GNUMake/sites/Make.llnl commit affcccf3c900d1d29349aaeb0c5f21c9dd63e637 Author: Max Katz Date: Wed Dec 16 09:08:45 2020 -0800 Remove GPU pragma macros from ArrayLim (#1649) These are no longer used. Src/Base/AMReX_ArrayLim.H commit 78eae95a21df8fa1d0e2a7c86b0437dd68e8aa7a Author: Max Katz Date: Wed Dec 16 09:06:31 2020 -0800 Remove custom MFIter reduction code (#1648) This is no longer in use by any of the astro codes. Src/Base/AMReX_BLFort.H Src/Base/AMReX_MFIter.H Src/Base/AMReX_MFIter.cpp commit 2f9277a64058c91a5d0153b77bd2476bd1f85690 Author: Axel Huebl Date: Tue Dec 15 17:06:11 2020 -0800 CMake: Fix Typos in SYCL Warnings (#1647) Just a typo and a missing space in a warning. Tools/CMake/AMReXOptions.cmake commit 006569febeae1329fc0a3e0c2fb17c77e42539c7 Author: Weiqun Zhang Date: Tue Dec 15 16:30:54 2020 -0800 DPCPP MPI wrapper (#1646) ## Summary Replace mpiicpx with mpiicpc because mpiicpc is shipped by Intel, whereas mpiicpx is a script that is only available on JLSE testbeds. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Co-authored-by: Michele Rosso Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSYCL.cmake Tools/GNUMake/Make.defs Tutorials/GPU/CNS/Source/CNS_bcfill.cpp Tutorials/GPU/CNS/Source/hydro/CNS_hydro_K.H commit cd2e06e47cb586d0b8e2ef3eb0c534548634ad97 Author: Weiqun Zhang Date: Tue Dec 15 10:44:19 2020 -0800 Remove support for old versions of DPCPP (#1644) Src/Base/AMReX_Extension.H Src/Base/AMReX_FBI.H Src/Base/AMReX_GpuPrint.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_RandomEngine.H Src/Base/AMReX_Scan.H commit cd31efb8aee323dafddb6040c0613667cfb5382c Author: Weiqun Zhang Date: Tue Dec 15 09:14:51 2020 -0800 HIP and DPCPP support in configure (#1640) Tools/GNUMake/comps/hip.mak Tools/libamrex/configure.py Tools/libamrex/mkconfig.py commit 66e52d44979d14859e007d769fb1b550ec591cc7 Author: Weiqun Zhang Date: Mon Dec 14 15:45:16 2020 -0800 Fused FillBoundary (#1570) ## Summary Add a new FillBoundary function for a Vector of FabArrays. There are two implementations in this PR. The one being used simply calls `FillBoundary_nowait` and `FillBoundary_finish`. The other version aggregates the MPI messages from multiple FabArrays into one. Tests on summit shows the latter is slower. But I don't want to lose the code, so it is kept for future investigation. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Array4.H Src/Base/AMReX_BaseFab.H Src/Base/AMReX_Box.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_MultiFab.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_NonLocalBCImpl.H Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelDescriptor.cpp commit 6a212600436d83fcfb656ce72d6c810c1b7c1cdd Author: kngott <30483578+kngott@users.noreply.github.com> Date: Mon Dec 14 13:51:13 2020 -0800 Divergence Free Stencil and API (#1483) API for multi-dimensional, face-based, coarse-to-fine interpolation (InterpFromCoarse and FillPatchTwoLevels). Includes a variation of Vanella et. al. stencil (doi:10.1016/j.jcp.2010.05.003, section 3.2), but instead of interpolating from a far coarse cell to solve interior closure problem (highly biased result dependent on chosen coarse cell), this uses least squares with initial guess equal to the average of fine face values across the cell. (Mathematica notebooks of solution are available). Src/AmrCore/AMReX_FillPatchUtil.H Src/AmrCore/AMReX_FillPatchUtil_1d.F90 Src/AmrCore/AMReX_FillPatchUtil_I.H Src/AmrCore/AMReX_INTERP_1D.F90 Src/AmrCore/AMReX_Interp_1D_C.H Src/AmrCore/AMReX_Interp_2D_C.H Src/AmrCore/AMReX_Interp_3D_C.H Src/AmrCore/AMReX_Interpolater.H Src/AmrCore/AMReX_Interpolater.cpp Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp Src/Base/AMReX_GpuLaunch.H Tests/DivFreePatch/GNUmakefile Tests/DivFreePatch/Make.package Tests/DivFreePatch/inputs Tests/DivFreePatch/main.cpp commit 78d6f64fadc606d65e30d4da5f218e941bc7ea59 Author: Weiqun Zhang Date: Mon Dec 14 10:19:51 2020 -0800 Constant coefficient MacProjector (#1636) Optimization of constant coefficient MAC projection by using constant coefficient Poisson solver. Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/Projections/AMReX_MacProjector.H Src/LinearSolvers/Projections/AMReX_MacProjector.cpp commit 4794c1d1c2e1fd83bc5ac6be8ff71c4893861128 Author: Weiqun Zhang Date: Mon Dec 14 08:39:07 2020 -0800 Option to allow multiple active MFIters (#1631) Add MFIter::allowMultipleMFIters(bool) so that users can disable assertion on the number of active MFIters. This is needed by FLASH because multiple MFIters (one for each level) are built in its iterator. Docs/sphinx_documentation/source/Basics.rst Src/Base/AMReX_MFIter.H Src/Base/AMReX_MFIter.cpp Src/F_Interfaces/Base/AMReX_multifab_fi.cpp Src/F_Interfaces/Base/AMReX_multifab_mod.F90 commit 8e73aa00126634cd83d4ca9c46fbac20da924e4e Author: drangara <69211175+drangara@users.noreply.github.com> Date: Mon Dec 14 11:37:42 2020 -0500 Convert AMReX_eb_to_pvd.F90 to cpp (#1627) ## Summary Convert the fortran file that writes the EB surface for paraview visualization to cpp ## Additional background Tested with the visualization files produced by `mfix/benchamrk/07-hopper` and `mfix/tutorials/clr/prototype`. Paraview visualization seems to look "good". When comparing the actual data from the files I do see some differences b/w the fortran and cpp version, but I believe the differences can be attributed to the precision of datatypes used in the fortran vs cpp code. These changes should not affect the solution results, just affects visualization. Src/EB/AMReX_EBToPVD.H Src/EB/AMReX_EBToPVD.cpp Src/EB/AMReX_WriteEBSurface.cpp Src/EB/AMReX_WriteEB_F.H Src/EB/AMReX_eb_to_pvd.F90 Src/EB/CMakeLists.txt Src/EB/Make.package commit c0a931f0ca46d62210e5b8f2e92c6952cfd60685 Author: Weiqun Zhang Date: Sat Dec 12 14:42:57 2020 -0800 Fix EB bug in #1629 (#1635) For EB nodal solver with constant sigma, the domain ghost cells of sigma should be set to zero. Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp commit f6f883bfedf2ecf3900c7f79f1f4df3ec92cca76 Author: Weiqun Zhang Date: Sat Dec 12 10:45:05 2020 -0800 Nodal solver with constant coefficient (#1629) * Add specialization for constant coefficient case in the nodal solver. * Update NodalProjector. * Fix some warnings in 1D. Src/LinearSolvers/MLMG/AMReX_MLABecLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp Src/LinearSolvers/Projections/AMReX_NodalProjector.H Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp commit 85a5f3be91eb9d58845f96c2907af8ee65661f87 Author: Andrew Myers Date: Sat Dec 12 09:44:58 2020 -0800 fix bug in pshifting neighbors introduced in #1590 (#1632) Src/Particle/AMReX_ParticleCommunication.H commit 52d32c6a07cd14498129ec3f03ae69a06a27573e Author: asalmgren Date: Fri Dec 11 21:10:38 2020 -0800 When we create a new level of refinement, we need to set the "level_s… (#1630) …teps" of that new level appropriately. ## Summary ## Additional background ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Amr/AMReX_Amr.cpp commit 389a0cc49f018ea86df0067bc8182336426fe414 Author: Andrew Myers Date: Fri Dec 11 16:24:59 2020 -0800 Expose the index of the neighboring particle to users of the iterator. (#1628) The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Particle/AMReX_NeighborList.H commit 73509be24826d16680289e5555cdb8db1e2217e5 Author: Weiqun Zhang Date: Thu Dec 10 10:02:10 2020 -0800 Documentation on avoiding continue and return inside GPU launch macros. (#1617) Docs/sphinx_documentation/source/GPU.rst commit 728db75fa70fcf10fb7fcb53210ba988ce24f1e4 Author: Weiqun Zhang Date: Thu Dec 10 10:01:16 2020 -0800 Implement overset mask for MLPoisson too (#1604) Src/LinearSolvers/CMakeLists.txt Src/LinearSolvers/MLMG/AMReX_MLABecLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson.H Src/LinearSolvers/MLMG/AMReX_MLPoisson.cpp Src/LinearSolvers/MLMG/AMReX_MLPoisson_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_3D_K.H Src/LinearSolvers/MLMG/Make.package commit 8db1187e6faa4d6628d47ddb28dcc23200516451 Author: Axel Huebl Date: Thu Dec 10 09:47:03 2020 -0800 Warning in SP: large ICC float constant (#1626) Fix an ICC warning for a too-large float constant when compiling for single precision in WarpX. Src/Amr/AMReX_Amr.cpp commit cd88adc2e64456116fcaf0dcce4669921bdb2296 Author: Weiqun Zhang Date: Wed Dec 9 18:57:46 2020 -0800 Fuse nodal jacobi smoother for GPU (#1625) ## Summary Fuse nodal jacobi smoother kernels by switch the order of the sweep and MFIter loops. For Amr-Wind's ABL test with `amr.n_cell = 256 256 64` and `amr.max_grid_size = 32` on one V100, the nodal project is about 2x faster. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/LinearSolvers/MLMG/AMReX_MLNodeLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp commit b846969f308e0402e188b95756b478261a31a8d2 Author: Matt Larsen Date: Wed Dec 9 16:26:11 2020 -0800 don't use geom.size() as the number of levels (#1624) ## Summary The conduit adapter was using the `geom.size()` to determine the number of levels, instead of believing `n_levels`. There was a case where the max levels was greater than the actual refinement level. In this case, geoms was the size of the max_levels in the input deck, but mfs was the size of the active levels. resolves #1623 The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp commit 923752e81d08a87d8edd7aaf01e92359125e4f86 Author: OscarAntepara <52221614+OscarAntepara@users.noreply.github.com> Date: Wed Dec 9 08:15:17 2020 -0800 Adding eb slopes limiter (#1594) * Adding eb slope limiters * Adding comments and making 3d adjustments * Fixing face centroids computation in the z-direction and correcting the comments Src/EB/AMReX_EB_slopes_K.H commit ffa5387dd6d873987386cd9ebbea3bab9592c9fd Author: Andrew Myers Date: Tue Dec 8 13:36:03 2020 -0800 Support for SoA data for neighbor particles. (#1590) Src/Particle/AMReX_NeighborList.H Src/Particle/AMReX_NeighborParticles.H Src/Particle/AMReX_NeighborParticlesCPUImpl.H Src/Particle/AMReX_NeighborParticlesGPUImpl.H Src/Particle/AMReX_NeighborParticlesI.H Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_Particles.H Tests/Particles/NeighborParticles/MDParticleContainer.H Tests/Particles/NeighborParticles/MDParticleContainer.cpp Tests/Particles/NeighborParticles/main.cpp Tests/Particles/Redistribute/main.cpp commit a28b02f1d194a0bce76cf784c6dc98cad1339015 Author: Axel Huebl Date: Tue Dec 8 11:13:42 2020 -0800 CMake: Verify DPC++ Compiler ID and at best warn (#1619) * CMake: less stringed MPI+DPCPP Check `find_package(MPI)` is capable to find Intel's MPI with a dpcpp CXX compiler. I would relax this check. Also, `CMAKE_CXX_COMPILER` can be an absolute path, e.g. `-DCMAKE_CXX_COMPILER=$(which dpcpp)`, which is not covered by the original syntax. * DPCPP Compiler Check: Only Warn on ID * Update recommendation * keep leading \n (consistency) Tools/CMake/AMReXOptions.cmake commit e0896cbd9e0432f642d484675e2a4f090ce93e33 Author: Houjun Date: Tue Dec 8 10:53:32 2020 -0800 Fix a bug that causes errors when writing HDF5 plotfile and particle (#1621) * Add HDF5 optimizations * Fix an issue that causes incorrect HDF5 data write with Tutorials/Basic/HeatEquation_EX1_C example code * Minor HDF5 tuning parameter change Co-authored-by: Houjun Tang Src/Base/AMReX_PlotFileUtil.cpp Src/Particle/AMReX_ParticleHDF5.H commit 82ca8fa508cad15c3f63c6233c8b028da5b4e347 Author: Michael Zingale Date: Tue Dec 8 12:34:11 2020 -0500 the A64fx Cray 10.0.1 compilers need the old-style options (#1620) ## Summary The default cray compilers on the Ookami A64fx machine use the old-style options for cray compilers. This logic allows us to build there. ## Additional background ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/GNUMake/comps/cray.mak commit 91a48a24777dabe03b1ff32bfce05536ce806200 Author: Weiqun Zhang Date: Mon Dec 7 20:00:29 2020 -0800 Fortran Inteface: Porting more MultiFab functions (#1609) Port IntVect ghost cell version of MultiFab::Add, Subtruct, Multiply, Divide, Saxpy, and LinComb to Fortran Interface. Src/F_Interfaces/Base/AMReX_multifab_fi.cpp Src/F_Interfaces/Base/AMReX_multifab_mod.F90 commit 975135ebdec9e0d96f9225a0196fade8ccbd2196 Author: Andrew Myers Date: Mon Dec 7 14:46:19 2020 -0800 [CI] turn off CLEAN option to keep existing files in the deploy target. (#1618) .github/workflows/docs.yml commit 8d45dd95dde0b6cb579c41b13ed3b0b69add297b Author: Axel Huebl Date: Mon Dec 7 13:40:33 2020 -0800 Conduit: Fix unused variables (#1615) ## Summary Removes four (4) unused variable warnings. cc @cyrush @mclarsen ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp commit 774ec481be4fdf28bd80f5a381af9b1c97d5b007 Author: Axel Huebl Date: Mon Dec 7 13:32:59 2020 -0800 DPC++: Re-Enable with Work-around. (#1613) My former colleagues shared a work-around that we can use to overwrite the compiler identification: https://gitlab.kitware.com/cmake/cmake/-/issues/21551#note_869580 This regression will also be fixed in CMake 3.19.2 .github/workflows/linux.yml commit ef6ebdbe62366e78d542fa13739e6d0557a3cbf4 Author: Andrew Myers Date: Mon Dec 7 11:48:19 2020 -0800 deploy to docs to seperate 'build' directory. (#1614) .github/workflows/docs.yml commit 2bbaf1bd81b0b8fee3dbd1bc95f2fc5c22009ac3 Author: Andrew Myers Date: Mon Dec 7 10:55:56 2020 -0800 fix and simplify script that builds the documentation (#1612) build_docs.sh commit 6fb23a385bf2fa0bc2e3b64a24e4940b4c0ea16d Author: Andrew Myers Date: Mon Dec 7 10:32:11 2020 -0800 need to specify the target directory on the website repository (#1611) .github/workflows/docs.yml commit 0a402eef6c0f7bdcb609e786d3140a991e99b7b3 Author: Andrew Myers Date: Mon Dec 7 08:59:32 2020 -0800 Migrate building and deploying the documentation to Github CI. (#1603) .github/workflows/dependencies/documentation.sh .github/workflows/docs.yml .travis.yml Docs/sphinx_tutorials/Makefile build_and_deploy.sh build_docs.sh deploy_key.enc commit 81a56bd628b92745c2baef58d8e7b044306cc34c Author: Weiqun Zhang Date: Sun Dec 6 18:46:48 2020 -0800 Add check for plotfile tools (#1610) .github/workflows/linux.yml Tools/Plotfile/fboxinfo.cpp Tools/Plotfile/fextract.cpp Tools/Plotfile/fextrema.cpp commit 65615e97ef62a2cac302ecbe30482d35f0806542 Author: Weiqun Zhang Date: Sun Dec 6 16:05:16 2020 -0800 Fix error in #1599 (#1608) The proposed change fixes an error in #1599. Tools/Plotfile/fsnapshot.cpp commit 88236accbcc263cbf3d4b103189bc7b522e06dfd Author: Andrew Myers Date: Sun Dec 6 14:33:50 2020 -0800 Remove alternative operators from AMReX (#1599) Operators like `and`, `or`, and `not` are allowed in the C++ standard; however, they cause problems on Windows and with our clang Windows CI. This PR removes them from AMReX in favor of `&&`, `||`, and `!`, and also configures the CI to throw a compilation error if they are re-introduced into the codebase. Application codes are still free to use these if they want. Co-authored-by: Weiqun Zhang .github/workflows/linux.yml .github/workflows/macos.yml Src/AmrCore/AMReX_AmrCore.cpp Src/AmrCore/AMReX_AmrParGDB.H Src/AmrCore/AMReX_Cluster.cpp Src/AmrCore/AMReX_FillPatchUtil_I.H Src/Base/AMReX_Array4.H Src/Base/AMReX_AsyncOut.cpp Src/Base/AMReX_BaseFab.H Src/Base/AMReX_BlockMutex.cpp Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxArray.cpp Src/Base/AMReX_CuptiTrace.cpp Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FArrayBox.cpp Src/Base/AMReX_FBI.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_FilCC_C.cpp Src/Base/AMReX_FileSystem.cpp Src/Base/AMReX_Geometry.cpp Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_GpuLaunchMacrosG.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_IArrayBox.cpp Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_PCI.H Src/Base/AMReX_ParallelDescriptor.cpp Src/Base/AMReX_PlotFileDataImpl.cpp Src/Base/AMReX_Reduce.H Src/Base/AMReX_Slopes_K.H Src/Base/AMReX_TypeTraits.H Src/Base/AMReX_Vector.H Src/Base/AMReX_VisMF.cpp Src/Boundary/AMReX_InterpBndryData_2D_K.H Src/Boundary/AMReX_InterpBndryData_3D_K.H Src/EB/AMReX_EB2_2D_C.H Src/EB/AMReX_EB2_2D_C.cpp Src/EB/AMReX_EB2_3D_C.H Src/EB/AMReX_EB2_3D_C.cpp Src/EB/AMReX_EB2_Level.H Src/EB/AMReX_EBAmrUtil.cpp Src/EB/AMReX_EBFluxRegister_2D_C.H Src/EB/AMReX_EBFluxRegister_3D_C.H Src/EB/AMReX_EBMultiFabUtil.cpp Src/EB/AMReX_EBMultiFabUtil_2D_C.H Src/EB/AMReX_EBMultiFabUtil_3D_C.H Src/EB/AMReX_EB_slopes_K.H Src/EB/AMReX_EB_triGeomOps_K.H Src/EB/AMReX_EB_utils.cpp Src/EB/AMReX_WriteEBSurface.cpp Src/EB/AMReX_algoim_K.H Src/Extern/HYPRE/AMReX_Habec_2D_K.H Src/Extern/HYPRE/AMReX_Habec_3D_K.H Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/F_Interfaces/AmrCore/AMReX_FlashFluxRegister.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLALap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensor_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensor_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp_K.H Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLMG_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLTensor_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensor_3D_K.H Src/Particle/AMReX_NeighborParticles.H Src/Particle/AMReX_NeighborParticlesCPUImpl.H Src/Particle/AMReX_NeighborParticlesGPUImpl.H Src/Particle/AMReX_NeighborParticlesI.H Src/Particle/AMReX_Particle.H Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_ParticleCommunication.cpp Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleLocator.H Src/Particle/AMReX_ParticleTile.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_WriteBinaryParticleData.H Tests/LinearSolvers/CellOverset/MyTest.cpp Tests/LinearSolvers/EBTensor/MyTest.cpp Tests/LinearSolvers/NodalOverset/MyTest.cpp Tests/LinearSolvers/TensorOverset/MyTest.cpp Tools/Plotfile/fcompare.cpp Tools/Plotfile/fsnapshot.cpp Tools/Postprocessing/C_Src/particle_compare.cpp Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.cpp Tutorials/LinearSolvers/ABecLaplacian_C/MyTest.cpp Tutorials/Particles/CellSortedParticles/CellSortedPC.cpp Tutorials/Particles/NeighborList/MDParticleContainer.cpp commit ad7974de5b3e68fe2f8fa7c114ffaa460ad166a2 Author: hsitaram Date: Sun Dec 6 14:08:33 2020 -0700 Eb from stl (#1592) ## Summary This is a work-in-progress PR that adds a preliminary set of tools and helper functions for using triangulated STL files in amrex based applications. Coupling with AMReX's EB framework will be added in later pull requests. This PR is done early so as to provide a software framework for incremental changes. As of now, I have added one helper function that provides a nodal multifab with a marker value indicating inside or outside the STL geometry. I guess it can be used for particle-wall collisions. I have also added a test case in the EB folder that generates a nodal multifab with the blanked values for an airfoil. The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Co-authored-by: Hariswaran Sitaraman Co-authored-by: Hariswaran Sitaraman Src/EB/AMReX_EB_STL_utils.H Src/EB/AMReX_EB_STL_utils.cpp Src/EB/AMReX_EB_triGeomOps_K.H Src/EB/CMakeLists.txt Src/EB/Make.package Tutorials/EB/STLtest/GNUmakefile Tutorials/EB/STLtest/Make.package Tutorials/EB/STLtest/airfoil.stl Tutorials/EB/STLtest/inputs Tutorials/EB/STLtest/main.cpp commit 8dc48ad8385cd395310567bf72a36470236e0ce2 Author: Weiqun Zhang Date: Fri Dec 4 17:18:32 2020 -0800 DPCPP: run stencil rap on device (#1606) An earlier compiler bug has been fixed. The stencil rap kernel can run on GPU now. Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp commit 0490136d291cc3b354558f091391277f16a46c5a Author: Weiqun Zhang Date: Fri Dec 4 17:11:29 2020 -0800 Remove DPCPP hacks (#1605) Because the Intel graphics driver has increased the kernel parameters to 2K by default, these hacks are no longer needed. Src/Base/AMReX_GpuLaunchFunctsG.H Src/EB/AMReX_EB2_3D_C.cpp Src/EB/AMReX_EB2_Level.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.cpp Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLEBTensorOp.cpp Src/LinearSolvers/MLMG/AMReX_MLTensorOp.cpp commit 13d67fd18e736e96bfe4e67a6387750380900384 Author: Axel Huebl Date: Fri Dec 4 10:25:55 2020 -0800 CI: Disable DPC++ oneAPI 2020-11 (#1602) Someone patched in a dedicated CMake compiler ID for Intel DPC++ but did not add full compiler support. Release: 2021.1.1 Ref.: https://gitlab.kitware.com/cmake/cmake/-/issues/21551 .github/workflows/linux.yml commit a168f5927cee0d87ff839b23128c8fc878622313 Author: mic84 Date: Thu Dec 3 16:19:42 2020 -0800 CMake: bring back check on CUDA computing capabilities (#1600) Tools/CMake/AMReX_SetupCUDA.cmake commit 2a070723e249e3e4cf2befd61a576330fc280d80 Author: Andrew Myers Date: Thu Dec 3 14:05:14 2020 -0800 move the memcpy wrapper into amrex::Gpu (#1598) Src/Base/AMReX_GpuUtility.H commit 5b436831f47158a50738dcd64a6fe579c9ef8861 Author: Andrew Myers Date: Thu Dec 3 12:35:20 2020 -0800 Add wrapper for memcpy. (#1597) `std::memcpy` does not seem to work in device code in our HIP CI. This adds a wrapper that reverts to the global namespace `memcpy` in HIP device code. The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_GpuUtility.H commit ef665e2a2fcc245162767b3613aba48bbc59f958 Author: Axel Huebl Date: Thu Dec 3 11:16:38 2020 -0800 Fix: Windows and/or includes (iso646) (#1593) ## Summary Add missing includes to `AMReX_Extension.H`, which pulls the `` include on Windows for support for `and`/`or`. ## Additional background Seen with Clang 11.0.0 on x64 Windows with Visual Studio 2017 on conda-forge during a WarpX build. See also #947 ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Extension.H Src/Base/AMReX_Vector.H Src/Particle/AMReX_ParticleTile.H commit 5c1addce382f86f0701e6bddc209522cfe70c52d Author: Axel Huebl Date: Thu Dec 3 10:33:42 2020 -0800 CI: Tutorial with OMP (#1591) Making sure the OpenMP Tutorials build as well. .github/workflows/linux.yml Tutorials/Particles/ElectromagneticPIC/CMakeLists.txt commit 0d000b5397b5f6ef94924d85b65846c5811faf86 Author: Weiqun Zhang Date: Thu Dec 3 10:23:15 2020 -0800 AMReX_Config.H (#1566) Based on discussions with @sayerhs, @ax3l, and @mic84, We have decided to make the following changes to the build systems. All headers in Src/ will include AMReX_Config.H, which will be generated by the build systems (both gnu make and cmake). The file will include all the macros we currently pass as compiler options. The purpose of this is to solve some inconsistency issues in the current systems. We support both super build and using AMReX as a library. Moreover, we do not want to force libamrex users to use the same build tool that is used to build the library. In this PR, we are put all the macro definitions in AMReX_Config.H that is included by every AMReX header. Co-authored-by: Axel Huebl Co-authored-by: Michele Rosso Src/Amr/AMReX_Amr.H Src/Amr/AMReX_AmrLevel.H Src/Amr/AMReX_AuxBoundaryData.H Src/Amr/AMReX_Derive.H Src/Amr/AMReX_Extrapolater.H Src/Amr/AMReX_LevelBld.H Src/Amr/AMReX_PROB_AMR_F.H Src/Amr/AMReX_StateData.H Src/Amr/AMReX_StateDescriptor.H Src/Amr/AMReX_extrapolater_1D_K.H Src/Amr/AMReX_extrapolater_2D_K.H Src/Amr/AMReX_extrapolater_3D_K.H Src/Amr/AMReX_extrapolater_K.H Src/AmrCore/AMReX_AmrCore.H Src/AmrCore/AMReX_AmrMesh.H Src/AmrCore/AMReX_AmrParGDB.H Src/AmrCore/AMReX_AmrParticles.H Src/AmrCore/AMReX_Cluster.H Src/AmrCore/AMReX_ErrorList.H Src/AmrCore/AMReX_FLUXREG_F.H Src/AmrCore/AMReX_FillPatchUtil.H Src/AmrCore/AMReX_FillPatchUtil_F.H Src/AmrCore/AMReX_FillPatchUtil_I.H Src/AmrCore/AMReX_FluxReg_1D_C.H Src/AmrCore/AMReX_FluxReg_2D_C.H Src/AmrCore/AMReX_FluxReg_3D_C.H Src/AmrCore/AMReX_FluxReg_C.H Src/AmrCore/AMReX_FluxRegister.H Src/AmrCore/AMReX_INTERP_F.H Src/AmrCore/AMReX_Interp_1D_C.H Src/AmrCore/AMReX_Interp_2D_C.H Src/AmrCore/AMReX_Interp_3D_C.H Src/AmrCore/AMReX_Interp_C.H Src/AmrCore/AMReX_Interpolater.H Src/AmrCore/AMReX_TagBox.H Src/Base/AMReX.H Src/Base/AMReX_Algorithm.H Src/Base/AMReX_Arena.H Src/Base/AMReX_Array.H Src/Base/AMReX_Array4.H Src/Base/AMReX_ArrayLim.H Src/Base/AMReX_AsyncOut.H Src/Base/AMReX_BArena.H Src/Base/AMReX_BCRec.H Src/Base/AMReX_BCUtil.H Src/Base/AMReX_BC_TYPES.H Src/Base/AMReX_BLBackTrace.H Src/Base/AMReX_BLFort.H Src/Base/AMReX_BLProfiler.H Src/Base/AMReX_BLassert.H Src/Base/AMReX_BackgroundThread.H Src/Base/AMReX_BaseFab.H Src/Base/AMReX_BaseFabUtility.H Src/Base/AMReX_BaseUmap.H Src/Base/AMReX_BaseUmap_f.H Src/Base/AMReX_BlockMutex.H Src/Base/AMReX_Box.H Src/Base/AMReX_BoxArray.H Src/Base/AMReX_BoxDomain.H Src/Base/AMReX_BoxIterator.H Src/Base/AMReX_BoxList.H Src/Base/AMReX_CArena.H Src/Base/AMReX_CONSTANTS.H Src/Base/AMReX_COORDSYS_1D_C.H Src/Base/AMReX_COORDSYS_2D_C.H Src/Base/AMReX_COORDSYS_3D_C.H Src/Base/AMReX_COORDSYS_C.H Src/Base/AMReX_CoordSys.H Src/Base/AMReX_CudaGraph.H Src/Base/AMReX_CuptiTrace.H Src/Base/AMReX_DArena.H Src/Base/AMReX_Dim3.H Src/Base/AMReX_DistributionMapping.H Src/Base/AMReX_EArena.H Src/Base/AMReX_Exception.H Src/Base/AMReX_Extension.H Src/Base/AMReX_FACopyDescriptor.H Src/Base/AMReX_FArrayBox.H Src/Base/AMReX_FPC.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayUtility.H Src/Base/AMReX_FabConv.H Src/Base/AMReX_FabFactory.H Src/Base/AMReX_FilCC_1D_C.H Src/Base/AMReX_FilCC_2D_C.H Src/Base/AMReX_FilCC_3D_C.H Src/Base/AMReX_FilCC_C.H Src/Base/AMReX_FilND_C.H Src/Base/AMReX_FileSystem.H Src/Base/AMReX_ForkJoin.H Src/Base/AMReX_Functional.H Src/Base/AMReX_Geometry.H Src/Base/AMReX_Gpu.H Src/Base/AMReX_GpuAllocators.H Src/Base/AMReX_GpuAssert.H Src/Base/AMReX_GpuAsyncArray.H Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuComplex.H Src/Base/AMReX_GpuContainers.H Src/Base/AMReX_GpuControl.H Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuElixir.H Src/Base/AMReX_GpuError.H Src/Base/AMReX_GpuFuse.H Src/Base/AMReX_GpuKernelInfo.H Src/Base/AMReX_GpuLaunch.H Src/Base/AMReX_GpuLaunchFunctsC.H Src/Base/AMReX_GpuLaunchFunctsG.H Src/Base/AMReX_GpuLaunchGlobal.H Src/Base/AMReX_GpuLaunchMacrosC.H Src/Base/AMReX_GpuLaunchMacrosG.H Src/Base/AMReX_GpuMemory.H Src/Base/AMReX_GpuPrint.H Src/Base/AMReX_GpuQualifiers.H Src/Base/AMReX_GpuRange.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_GpuTypes.H Src/Base/AMReX_GpuUtility.H Src/Base/AMReX_IArrayBox.H Src/Base/AMReX_INT.H Src/Base/AMReX_IndexSequence.H Src/Base/AMReX_IndexType.H Src/Base/AMReX_IntConv.H Src/Base/AMReX_IntVect.H Src/Base/AMReX_LayoutData.H Src/Base/AMReX_Lazy.H Src/Base/AMReX_Loop.H Src/Base/AMReX_MFCopyDescriptor.H Src/Base/AMReX_MFIter.H Src/Base/AMReX_Machine.H Src/Base/AMReX_Machine.cpp Src/Base/AMReX_MakeType.H Src/Base/AMReX_Math.H Src/Base/AMReX_MemPool.H Src/Base/AMReX_MemProfiler.H Src/Base/AMReX_MultiFab.H Src/Base/AMReX_MultiFabUtil.H Src/Base/AMReX_MultiFabUtil_1D_C.H Src/Base/AMReX_MultiFabUtil_2D_C.H Src/Base/AMReX_MultiFabUtil_3D_C.H Src/Base/AMReX_MultiFabUtil_C.H Src/Base/AMReX_MultiFabUtil_nd_C.H Src/Base/AMReX_NFiles.H Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBCImpl.H Src/Base/AMReX_OpenMP.H Src/Base/AMReX_Orientation.H Src/Base/AMReX_PODVector.H Src/Base/AMReX_ParallelContext.H Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelReduce.H Src/Base/AMReX_ParmParse.H Src/Base/AMReX_Partition.H Src/Base/AMReX_Periodicity.H Src/Base/AMReX_PhysBCFunct.H Src/Base/AMReX_PlotFileDataImpl.H Src/Base/AMReX_PlotFileUtil.H Src/Base/AMReX_Print.H Src/Base/AMReX_REAL.H Src/Base/AMReX_Random.H Src/Base/AMReX_RandomEngine.H Src/Base/AMReX_RealBox.H Src/Base/AMReX_RealVect.H Src/Base/AMReX_Reduce.H Src/Base/AMReX_SPACE.H Src/Base/AMReX_Scan.H Src/Base/AMReX_Slopes_K.H Src/Base/AMReX_ThirdPartyProfiling.H Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_Tuple.H Src/Base/AMReX_TypeTraits.H Src/Base/AMReX_Utility.H Src/Base/AMReX_Vector.H Src/Base/AMReX_VectorIO.H Src/Base/AMReX_VisMF.H Src/Base/AMReX_ccse-mpi.H Src/Base/AMReX_filcc_f.H Src/Base/AMReX_iMultiFab.H Src/Base/AMReX_parstream.H Src/Boundary/AMReX_BndryData.H Src/Boundary/AMReX_BndryRegister.H Src/Boundary/AMReX_BoundCond.H Src/Boundary/AMReX_FabSet.H Src/Boundary/AMReX_InterpBndryData.H Src/Boundary/AMReX_InterpBndryData_1D_K.H Src/Boundary/AMReX_InterpBndryData_2D_K.H Src/Boundary/AMReX_InterpBndryData_3D_K.H Src/Boundary/AMReX_InterpBndryData_K.H Src/Boundary/AMReX_LOUtil_K.H Src/Boundary/AMReX_LO_BCTYPES.H Src/Boundary/AMReX_MacBndry.H Src/Boundary/AMReX_Mask.H Src/Boundary/AMReX_MultiMask.H Src/Boundary/AMReX_YAFluxRegister.H Src/Boundary/AMReX_YAFluxRegister_1D_K.H Src/Boundary/AMReX_YAFluxRegister_2D_K.H Src/Boundary/AMReX_YAFluxRegister_3D_K.H Src/Boundary/AMReX_YAFluxRegister_K.H Src/CMakeLists.txt Src/EB/AMReX_EB2.H Src/EB/AMReX_EB2_2D_C.H Src/EB/AMReX_EB2_3D_C.H Src/EB/AMReX_EB2_C.H Src/EB/AMReX_EB2_GeometryShop.H Src/EB/AMReX_EB2_Graph.H Src/EB/AMReX_EB2_IF.H Src/EB/AMReX_EB2_IF_AllRegular.H Src/EB/AMReX_EB2_IF_Base.H Src/EB/AMReX_EB2_IF_Box.H Src/EB/AMReX_EB2_IF_Complement.H Src/EB/AMReX_EB2_IF_Cylinder.H Src/EB/AMReX_EB2_IF_Difference.H Src/EB/AMReX_EB2_IF_Ellipsoid.H Src/EB/AMReX_EB2_IF_Extrusion.H Src/EB/AMReX_EB2_IF_Intersection.H Src/EB/AMReX_EB2_IF_Lathe.H Src/EB/AMReX_EB2_IF_Plane.H Src/EB/AMReX_EB2_IF_Polynomial.H Src/EB/AMReX_EB2_IF_Rotation.H Src/EB/AMReX_EB2_IF_Scale.H Src/EB/AMReX_EB2_IF_Sphere.H Src/EB/AMReX_EB2_IF_Spline.H Src/EB/AMReX_EB2_IF_Torus.H Src/EB/AMReX_EB2_IF_Translation.H Src/EB/AMReX_EB2_IF_Union.H Src/EB/AMReX_EB2_Level.H Src/EB/AMReX_EB2_MultiGFab.H Src/EB/AMReX_EBAmrUtil.H Src/EB/AMReX_EBCellFlag.H Src/EB/AMReX_EBDataCollection.H Src/EB/AMReX_EBFArrayBox.H Src/EB/AMReX_EBFabFactory.H Src/EB/AMReX_EBFluxRegister.H Src/EB/AMReX_EBFluxRegister_2D_C.H Src/EB/AMReX_EBFluxRegister_3D_C.H Src/EB/AMReX_EBFluxRegister_C.H Src/EB/AMReX_EBInterpolater.H Src/EB/AMReX_EBMultiFabUtil.H Src/EB/AMReX_EBMultiFabUtil_2D_C.H Src/EB/AMReX_EBMultiFabUtil_3D_C.H Src/EB/AMReX_EBMultiFabUtil_C.H Src/EB/AMReX_EBSupport.H Src/EB/AMReX_EB_slopes_K.H Src/EB/AMReX_EB_utils.H Src/EB/AMReX_MultiCutFab.H Src/EB/AMReX_WriteEBSurface.H Src/EB/AMReX_WriteEB_F.H Src/EB/AMReX_algoim.H Src/EB/AMReX_algoim_K.H Src/EB/AMReX_distFcnElement.H Src/Extern/Conduit/AMReX_Conduit_Blueprint.H Src/Extern/HYPRE/AMReX_Habec_2D_K.H Src/Extern/HYPRE/AMReX_Habec_3D_K.H Src/Extern/HYPRE/AMReX_Habec_K.H Src/Extern/HYPRE/AMReX_Hypre.H Src/Extern/HYPRE/AMReX_HypreABecLap.H Src/Extern/HYPRE/AMReX_HypreABecLap2.H Src/Extern/HYPRE/AMReX_HypreABecLap3.H Src/Extern/HYPRE/AMReX_HypreIJIface.H Src/Extern/HYPRE/AMReX_HypreNodeLap.H Src/Extern/HYPRE/CMakeLists.txt Src/Extern/PETSc/AMReX_PETSc.H Src/Extern/PETSc/CMakeLists.txt Src/Extern/ProfParser/AMReX_AVGDOWN_F.H Src/Extern/ProfParser/AMReX_BLProfStats.H Src/Extern/ProfParser/AMReX_BLProfUtilities.H Src/Extern/ProfParser/AMReX_BLWritePlotFile.H Src/Extern/ProfParser/AMReX_CommProfStats.H Src/Extern/ProfParser/AMReX_RegionsProfStats.H Src/Extern/ProfParser/CMakeLists.txt Src/Extern/SENSEI/AMReX_AmrDataAdaptor.H Src/Extern/SENSEI/AMReX_AmrInSituBridge.H Src/Extern/SENSEI/AMReX_AmrMeshDataAdaptor.H Src/Extern/SENSEI/AMReX_AmrMeshInSituBridge.H Src/Extern/SENSEI/AMReX_InSituBridge.H Src/Extern/SENSEI/AMReX_InSituUtils.H Src/Extern/SENSEI/CMakeLists.txt Src/Extern/SWFFT/Dfft.H Src/Extern/amrdata/AMReX_AmrData.H Src/Extern/amrdata/AMReX_AmrvisConstants.H Src/Extern/amrdata/AMReX_DataServices.H Src/Extern/amrdata/AMReX_XYPlotDataList.H Src/Extern/hpgmg/BL_HPGMG.H Src/F_Interfaces/AmrCore/AMReX_FAmrCore.H Src/F_Interfaces/AmrCore/AMReX_FlashFluxRegister.H Src/F_Interfaces/Base/AMReX_FPhysBC.H Src/F_Interfaces/CMakeLists.txt Src/LinearSolvers/MLMG/AMReX_MLABecLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLap_K.H Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLALap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLALap_K.H Src/LinearSolvers/MLMG/AMReX_MLALaplacian.H Src/LinearSolvers/MLMG/AMReX_MLCGSolver.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.H Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensorOp.H Src/LinearSolvers/MLMG/AMReX_MLEBTensor_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensor_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLEBTensor_K.H Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLLinOp_F.H Src/LinearSolvers/MLMG/AMReX_MLLinOp_K.H Src/LinearSolvers/MLMG/AMReX_MLMG.H Src/LinearSolvers/MLMG/AMReX_MLMGBndry.H Src/LinearSolvers/MLMG/AMReX_MLMG_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLMG_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLPoisson.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLPoisson_K.H Src/LinearSolvers/MLMG/AMReX_MLTensorOp.H Src/LinearSolvers/MLMG/AMReX_MLTensor_1D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensor_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensor_3D_K.H Src/LinearSolvers/MLMG/AMReX_MLTensor_K.H Src/LinearSolvers/Projections/AMReX_MacProjector.H Src/LinearSolvers/Projections/AMReX_NodalProjector.H Src/Particle/AMReX_ArrayOfStructs.H Src/Particle/AMReX_BinIterator.H Src/Particle/AMReX_DenseBins.H Src/Particle/AMReX_NeighborList.H Src/Particle/AMReX_NeighborParticles.H Src/Particle/AMReX_NeighborParticlesCPUImpl.H Src/Particle/AMReX_NeighborParticlesGPUImpl.H Src/Particle/AMReX_ParGDB.H Src/Particle/AMReX_ParIter.H Src/Particle/AMReX_Particle.H Src/Particle/AMReX_ParticleBufferMap.H Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_ParticleHDF5.H Src/Particle/AMReX_ParticleIO.H Src/Particle/AMReX_ParticleInit.H Src/Particle/AMReX_ParticleLocator.H Src/Particle/AMReX_ParticleMPIUtil.H Src/Particle/AMReX_ParticleMesh.H Src/Particle/AMReX_ParticleReduce.H Src/Particle/AMReX_ParticleTile.H Src/Particle/AMReX_ParticleTransformation.H Src/Particle/AMReX_ParticleUtil.H Src/Particle/AMReX_Particle_mod_K.H Src/Particle/AMReX_Particles.H Src/Particle/AMReX_SparseBins.H Src/Particle/AMReX_StructOfArrays.H Src/Particle/AMReX_TracerParticle_mod_K.H Src/Particle/AMReX_TracerParticles.H Src/Particle/AMReX_WriteBinaryParticleData.H Src/Particle/CMakeLists.txt Src/SDC/AMReX_SDCstruct.H Tools/CMake/AMReXGenerateConfigHeader.cmake Tools/CMake/AMReXInstallHelpers.cmake Tools/CMake/AMReXSetDefines.cmake Tools/CMake/AMReX_Config.H.in Tools/CMake/AMReX_Config.cmake Tools/CMake/AMReX_Defines.cmake Tools/CMake/AMReX_Utils.cmake Tools/CMake/modify_installed_headers.cmake Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules Tools/libamrex/mkconfig.py Tutorials/Amr/Advection_AmrLevel/Source/AmrLevelAdv.H commit 5fb87ea35fa521e9ce97dc9d36f21eda5e2343be Author: Shreyas Ananthan Date: Thu Dec 3 09:19:23 2020 -0700 Restore reproducibility of solutions with MacProjector reuse (#1596) Src/LinearSolvers/Projections/AMReX_MacProjector.cpp commit ae4cc9a1488719e284409b078887866fe65709d9 Author: mic84 Date: Wed Dec 2 15:12:33 2020 -0800 CMake: output missing components list when AMReX is not found (#1572) Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXInstallHelpers.cmake commit 75c0896594cd38d883d7907466657eac3ca1a188 Author: Axel Huebl Date: Wed Dec 2 15:09:17 2020 -0800 CI: Windows + Clang (#1588) Add Clang as Windows target. This compiler offers potentially a route to modern OpenMP. (MSVC: 2.0 standard, LLVM/Clang: 3.1 standard) .github/workflows/windows.yml commit 91bdc795d598b8bbf58d640d572f27dfa5ae31a5 Author: Shreyas Ananthan Date: Wed Dec 2 14:44:56 2020 -0700 MacProjector: Update setDomainBC API (#1589) Allow `setDomainBC` to be called before `setUMAC` has been called. Src/LinearSolvers/Projections/AMReX_MacProjector.cpp commit 820a30d7966426a96e24e3963b37f3789eec258f Author: Andrew Myers Date: Wed Dec 2 13:26:03 2020 -0800 Fix InitNRandomPerCell for single precision particles (#1582) * due to roundoff errors, we need to redraw these numbers sometimes when single precision particles are used. * missing ; Src/Particle/AMReX_ParticleInit.H commit dc0ec74c4d120509f9754fb7d43af6d275d9c1a4 Author: Shreyas Ananthan Date: Wed Dec 2 13:35:05 2020 -0700 DPCPP: Update CMake options to allow building with MPI+DPCPP (#1586) Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXSYCL.cmake commit 89220ed73d9fe74618386f8b1bfafd30760d6fde Author: Andrew Myers Date: Tue Dec 1 10:31:23 2020 -0800 Reimplement NumParticlesAtLevel (#1581) This removes a workaround that had this function fall back to CPUs for CUDA (but not HIP or DPC++). Additionally, this re-implements the function in terms of ReduceTuple to avoid multiple memcpy operations in the case of multiple boxes. Src/Particle/AMReX_ParticleContainerI.H Tests/Particles/ParticleReduce/main.cpp commit bbcce736ea9497cdb4722e8b4f4c2ee0d7c193ca Author: Robert Maynard Date: Tue Dec 1 13:24:16 2020 -0500 eval_genex correctly replaces the matching compiler version (#1580) Previously it would place the resulting output in the variable `1` instead of `_in`. Additionally it would try to add an extra `>`. Tools/CMake/AMReXGenexHelpers.cmake commit 9a523cf95dc3046081b0830cf435a3455929728d Author: Robert Maynard Date: Tue Dec 1 13:20:19 2020 -0500 Add msvc cuda support (#1573) ## Summary This set of changes has allowed me to build AMReX on windows using CUDA 11.X and VS2019. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/AmrCore/AMReX_ErrorList.cpp Src/Base/AMReX.H Src/Base/AMReX.cpp Src/Base/AMReX_BLBackTrace.cpp Src/Base/AMReX_Extension.H Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_MultiFab.H Tools/CMake/AMReX_Config.cmake commit 1cb99250ecd4da445cfc8afe325664030c49c4f0 Author: Weiqun Zhang Date: Tue Dec 1 10:12:38 2020 -0800 Update CHANGES (#1584) CHANGES commit 94ca530cb43c497cf22dfbfbd13dfd087f7d910d Author: Andrew Myers Date: Mon Nov 30 21:55:26 2020 -0500 Fix bug affecting async IO + single precision particles (#1583) The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Particle/AMReX_WriteBinaryParticleData.H commit d9af9ab7f134b04bc5e3cc57362e82a64fecef0a Author: Weiqun Zhang Date: Mon Nov 30 12:49:50 2020 -0800 Fix warnings for DPCPP (#1579) Fix unused variable. Do not pass `-fsycl-unnamed-lambda` to `mpiicpx` for linking to get rid of warning on unknown argument. Src/Base/AMReX_FBI.H Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules Tutorials/GPU/CNS/Source/CNS.H commit 07afa8ec8f59403ac54c2b2c566c1c88b1863112 Author: Weiqun Zhang Date: Mon Nov 30 10:25:56 2020 -0800 Template specialization of MPI communication functions for char (#1569) * Add template specialization of MPI communication functions for char. Select a proper data type according to the size. * Use the new functions in FabArray and Particle communication. And this also fixes a bug in Particle communication. (The counts should be divided by the size of data type.) * Due to a gcc bug on explicit specialization in a namespace, workaround was implemented in AMReX_ParallelDescriptor.H and .cpp. Src/Base/AMReX_Arena.H Src/Base/AMReX_Arena.cpp Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_ParallelDescriptor.H Src/Base/AMReX_ParallelDescriptor.cpp Src/Particle/AMReX_ParticleCommunication.H commit 50c40c46382a83cdc98fd59e337c5037aba07c1e Author: Weiqun Zhang Date: Mon Nov 30 10:16:16 2020 -0800 EB2::PolyIF (#1567) The existing PolynomialIF contains a dynamic size vector. This makes it hard to support it on GPU. We have used an undocumented feature to get this run on GPU. However, this code can no longer compile, because the GNU make system now makes cross execution space calls an error. So we disable GPU support for PolynomialIF and add a new PolyIF class with fixed size array and therefore GPU support. Src/EB/AMReX_EB2_IF_Polynomial.H commit 26e4d30f346cb89921502e409da1b933a60f9c81 Author: Weiqun Zhang Date: Mon Nov 30 10:11:37 2020 -0800 Add .gitattributes (#1578) Make github detect *.H as C++. This file is borrowed from Castro. .gitattributes commit a77968674f3d9ec1d4298d99a506dd1951f88c67 Author: Shreyas Ananthan Date: Mon Nov 30 09:17:48 2020 -0700 Tools/Plotfile: Fix build errors in single precision mode (#1577) This PR fixes one build error and one warning observed when compiling `Tools/Plotfile` utilities with `-DAMREX_USE_FLOAT` Tools/Plotfile/fcompare.cpp Tools/Plotfile/fextract.cpp commit 82eceae29419ceec24450926af1f5445ac425d00 Author: Shreyas Ananthan Date: Mon Nov 30 09:14:54 2020 -0700 Arena: Add accessor to query Arena attributes through info object (#1575) ## Summary This PR adds an accessor to the `Arena` class that can be used to query the attributes of the Arena (e.g., `device_use_managed_memory` flag) from the application codes. ## Additional background Currently, there is no easy way to determine if `amrex::The_Arena()` is using managed memory. Applications would have to use a combination of `AMREX_USE_HIP` compile-time definition and `amrex.the_arena_is_managed` ParmParse variable to determine this. Testing individual pointers with `amrex::Gpu::isManaged()` is possible only on CUDA/HIP but not DPC++. Also it is costly compared to just querying the `Arena` object. Allowing read-only access to `ArenaInfo` object will allow a wide variety of checks in application code. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [X] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Arena.H commit acdb7eac1d5a1744e87c0caafd1c43cc5e1171e9 Author: Luca Fedeli Date: Mon Nov 30 16:58:32 2020 +0100 Fix some warnings related to double-->float conversions (#1571) I tried to compile `WarpX` in single precision and I noticed several warnings related to casts from `double` to `float` in `AMReX`. This PR should fix some of these warnings. I plan to continue in the next days with other PRs, if you agree. Src/AmrCore/AMReX_AmrMesh.H Src/Base/AMReX_CoordSys.H Src/Base/AMReX_FilCC_2D_C.H Src/Base/AMReX_Geometry.H Src/Base/AMReX_MultiFabUtil_2D_C.H Src/Base/AMReX_MultiFabUtil_nd_C.H commit abdab63376660238396626ee064130a5c1826024 Author: Shreyas Ananthan Date: Sun Nov 29 14:04:44 2020 -0700 Fix assertion check for MacProjection (#1576) Fixes bug introduced in #1574 Src/LinearSolvers/Projections/AMReX_MacProjector.cpp commit abeb357db57261fe8a8d811e161da0c6c2814d92 Author: Shreyas Ananthan Date: Sun Nov 29 09:53:42 2020 -0700 Allow MacProjection reuse (#1574) This commit modifies the MacProjector interface to allow reuse of the MacProjector. The changes to the API are backwards compatible. - Add a new constructor that does not require specification of beta, umac etc. - Add `initProjector` to explicitly initialize the projection instance at app level - Add methods to set MAC velocities, divU etc. Src/LinearSolvers/Projections/AMReX_MacProjector.H Src/LinearSolvers/Projections/AMReX_MacProjector.cpp commit cc8f323c426f3957619c93477af2b9e32e3b49c2 Author: Axel Huebl Date: Mon Nov 23 22:27:06 2020 -0800 CI: Make DPC++ Pretty (#1568) * CI: Make DPC++ Pretty Just combing the show pony. * oneAPI Activation: still needs set +e .github/workflows/linux.yml commit f068164b513c12828dac069510d64f8b4912028e Author: mic84 Date: Mon Nov 23 14:48:58 2020 -0800 Remove SUNDIALS support (#1559) ## Summary SUNDIALS support in AMReX consisted of only few Fortran interfaces to SUNDIALS functions for the convenience of app codes. Since there are no AMReX functionalities depending on SUNDIALS, support for SUNDIALS will be removed. ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Docs/sphinx_documentation/source/BuildingAMReX.rst Docs/sphinx_documentation/source/SUNDIALS.rst Docs/sphinx_documentation/source/SUNDIALS_CVODE.rst Docs/sphinx_documentation/source/SUNDIALS_top.rst Docs/sphinx_tutorials/source/SUNDIALS_Tutorial.rst Src/CMakeLists.txt Src/Extern/CVODE/Make.package Src/Extern/CVODE/cvode_interface.f90 Src/Extern/CVODE/fnvector_serial.f90 Src/Extern/CVODE/integrator_stats.f90 Src/Extern/CVODE/sundials_fdlsmat.f90 Src/Extern/SUNDIALS/CMakeLists.txt Src/Extern/SUNDIALS/Make.package Src/Extern/SUNDIALS/arkode_interface.f90 Src/Extern/SUNDIALS/cvode_interface.f90 Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXThirdPartyLibraries.cmake Tools/CMake/FindSUNDIALS.cmake Tools/GNUMake/Make.defs Tools/GNUMake/packages/Make.cvode Tools/GNUMake/packages/Make.sundials Tutorials/CMakeLists.txt Tutorials/CVODE/EX1_F/GNUmakefile Tutorials/CVODE/EX1_F/Make.package Tutorials/CVODE/EX1_F/inputs Tutorials/CVODE/EX1_F/integrate_ode.f90 Tutorials/CVODE/EX1_F/main.cpp Tutorials/CVODE/EX1_F/myfunc_F.H Tutorials/CVODE/EX1_F/ode_mod.f90 Tutorials/CVODE/EX2_F/GNUmakefile Tutorials/CVODE/EX2_F/Make.package Tutorials/CVODE/EX2_F/inputs Tutorials/CVODE/EX2_F/integrate_ode_no_jac.f90 Tutorials/CVODE/EX2_F/integrate_ode_with_jac.f90 Tutorials/CVODE/EX2_F/main.cpp Tutorials/CVODE/EX2_F/myfunc_F.H Tutorials/CVODE/EX2_F/ode_mod.f90 Tutorials/SUNDIALS/EX-CUSOLVER/GNUmakefile Tutorials/SUNDIALS/EX-CUSOLVER/Make.CVODE Tutorials/SUNDIALS/EX-CUSOLVER/Make.package Tutorials/SUNDIALS/EX-CUSOLVER/README.md Tutorials/SUNDIALS/EX-CUSOLVER/extern_probin.template Tutorials/SUNDIALS/EX-CUSOLVER/inputs Tutorials/SUNDIALS/EX-CUSOLVER/inputs_128 Tutorials/SUNDIALS/EX-CUSOLVER/inputs_256 Tutorials/SUNDIALS/EX-CUSOLVER/inputs_32 Tutorials/SUNDIALS/EX-CUSOLVER/inputs_64 Tutorials/SUNDIALS/EX-CUSOLVER/main.cpp Tutorials/SUNDIALS/EX-CUSOLVER/make_cuda.sh Tutorials/SUNDIALS/EX-CUSOLVER/make_cuda_cusolver.sh Tutorials/SUNDIALS/EX-CUSOLVER/make_serial.sh Tutorials/SUNDIALS/EX-CUSOLVER/react_cuda.cpp Tutorials/SUNDIALS/EX-CUSOLVER/react_cuda_cusolver.cpp Tutorials/SUNDIALS/EX-CUSOLVER/react_serial.cpp Tutorials/SUNDIALS/EX-CUSOLVER/react_system.F90 Tutorials/SUNDIALS/EX-CUSOLVER/react_utils.F90 Tutorials/SUNDIALS/EX-CUSOLVER/test_react.H Tutorials/SUNDIALS/EX-CUSOLVER/test_react_F.H Tutorials/SUNDIALS/EX1_C/CMakeLists.txt Tutorials/SUNDIALS/EX1_C/GNUmakefile Tutorials/SUNDIALS/EX1_C/Make.package Tutorials/SUNDIALS/EX1_C/SetIC.f90 Tutorials/SUNDIALS/EX1_C/inputs Tutorials/SUNDIALS/EX1_C/inputs_2box Tutorials/SUNDIALS/EX1_C/inputs_non_vectorized Tutorials/SUNDIALS/EX1_C/main.cpp Tutorials/SUNDIALS/EX1_C/myfunc_F.H Tutorials/SUNDIALS/EX1_CUDA/GNUmakefile Tutorials/SUNDIALS/EX1_CUDA/Make.package Tutorials/SUNDIALS/EX1_CUDA/SetIC.f90 Tutorials/SUNDIALS/EX1_CUDA/inputs Tutorials/SUNDIALS/EX1_CUDA/main.cpp Tutorials/SUNDIALS/EX1_CUDA/myfunc_F.H Tutorials/SUNDIALS/EX1_CUDA/ode_mod.f90 Tutorials/SUNDIALS/EX1_F/GNUmakefile Tutorials/SUNDIALS/EX1_F/Make.package Tutorials/SUNDIALS/EX1_F/inputs Tutorials/SUNDIALS/EX1_F/integrate_ode.f90 Tutorials/SUNDIALS/EX1_F/main.cpp Tutorials/SUNDIALS/EX1_F/myfunc_F.H Tutorials/SUNDIALS/EX1_F/ode_mod.f90 Tutorials/SUNDIALS/EX2_F/GNUmakefile Tutorials/SUNDIALS/EX2_F/Make.package Tutorials/SUNDIALS/EX2_F/inputs Tutorials/SUNDIALS/EX2_F/integrate_ode_no_jac.f90 Tutorials/SUNDIALS/EX2_F/integrate_ode_with_jac.f90 Tutorials/SUNDIALS/EX2_F/main.cpp Tutorials/SUNDIALS/EX2_F/myfunc_F.H Tutorials/SUNDIALS/EX2_F/ode_mod.f90 commit e1ff59c8d9207a135f25284380a1ba8fc15abb1f Author: Roberto Porcu <53792251+rporcu@users.noreply.github.com> Date: Sun Nov 22 23:19:06 2020 -0500 allow MLEBABecLap to solve for multiple components (#1557) ## Summary MLLinOp::getNComp() is returning 1 and right now MLEBABecLap is working only with MultiFabs having 1 component. ## Additional background This PR has the goal to generalize MLEBABecLap solver for a generic N number of components. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Co-authored-by: Weiqun Zhang Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.H Src/LinearSolvers/MLMG/AMReX_MLEBABecLap.cpp commit 1f8163c397e1aa33fd633219166b87d4633a0b46 Author: Weiqun Zhang Date: Sun Nov 22 19:40:00 2020 -0800 DPCPP: MPI (#1556) Allow MPI for dpcpp build in gnu make assuming mpiicpx is used. Tools/GNUMake/Make.defs Tools/GNUMake/Make.machines Tools/GNUMake/comps/dpcpp.mak Tools/GNUMake/sites/Make.alcf commit a85fd8d7971f9448b8a00e796a392bbe84ae0f97 Author: Axel Huebl Date: Fri Nov 20 10:28:25 2020 -0800 CMake: OpenMP for Fortran (#1563) * CMake: OpenMP for Fortran Fix missing activation. * Update Tools/CMake/AMReXParallelBackends.cmake Co-authored-by: mic84 * CMakeL OpenMP Fortran fix Config, too Co-authored-by: mic84 Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXParallelBackends.cmake commit 7a0fea34be764441014e2ba7dac44fecbdbbabfb Author: Weiqun Zhang Date: Thu Nov 19 11:01:07 2020 -0800 Non-local boundary conditions (#1544) * Non-local boundary conditions * Rotate90 fills the lo-x and lo-y boundary regions by rotating the data around (x=0,y=0) by 90 degrees in either direction. It also fills the corner of lo-x and lo-y boundary region by rotating the data by 180 degrees. * Rotate180 fills the lo-x boundary by rotating the data around (x=0,y=L_y/2) by 180 degrees. * Fill the polar boundaries of the spherical coordinates (theta, phi, r). The lo-x boundary is filled with f(-x,y) = f(x,mod(y+pi,2*pi)), and the hi-x boundary is filled with f(pi+x,y) = f(pi-x,mod(y+pi,2*pi)). * GPU support for NonLocalBC Src/Base/AMReX_FBI.H Src/Base/AMReX_FabArrayBase.H Src/Base/AMReX_FabArrayBase.cpp Src/Base/AMReX_NonLocalBC.H Src/Base/AMReX_NonLocalBCImpl.H Src/Base/CMakeLists.txt Src/Base/Make.package commit b5e08d92541f24e3ee3cb8d46f5b49c0147538bd Author: Weiqun Zhang Date: Thu Nov 19 10:28:53 2020 -0800 Remove deprecated MFUtil::convert function. One could use amrex::cast for FabArray casting. (#1523) Src/Base/AMReX_MultiFabUtilI.H commit 907baaede901d52601f120d606224bccd04d9611 Author: Andrew Myers Date: Thu Nov 19 13:24:16 2020 -0500 If TheZeroVector() is passed in to SortParticlesByBin, do nothing instead of crashing. (#1564) Src/Particle/AMReX_ParticleContainerI.H Src/Particle/AMReX_Particles.H commit 6038a3102fc55db861d69b242eecc35119675b96 Author: Weiqun Zhang Date: Wed Nov 18 09:49:08 2020 -0800 Scan size type (#1521) Make the size type a template parameter to support the scan of more than INT_MAX elements. Src/Base/AMReX_Scan.H commit ff92f1c129ea149eead13dc264eabf15a705e74d Author: mic84 Date: Wed Nov 18 09:14:51 2020 -0800 CMake: AMReX_MPI options depends on SYCL, not HIP (#1555) Tools/CMake/AMReXOptions.cmake commit 7cf969f1b656f9ab17975c90952360fb400dd712 Author: Axel Huebl Date: Wed Nov 18 09:14:25 2020 -0800 CMake: Fix Single-Precision Switch (#1550) * CMake: Fix Single-Precision Switch Since the latest refactoring, single-recision builds were not activated. This fixes it. * AMReX_Config.H: add AMREX_SINGLE_PRECISION_PARTICLES Src/Particle/CMakeLists.txt Tools/CMake/AMReX_Config.H.in Tools/CMake/AMReX_Defines.cmake commit 14c3003f8f4dda9ef9db0c04ba613e378677358a Author: Weiqun Zhang Date: Wed Nov 18 09:03:38 2020 -0800 Refactor FabArray's communication functions (#1553) Remove duplication and allow for more flexibility for extension. Src/AmrCore/AMReX_TagBox.cpp Src/Base/AMReX_FBI.H Src/Base/AMReX_FabArray.H Src/Base/AMReX_FabArrayCommI.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_MultiFabUtil.cpp Src/Base/AMReX_iMultiFab.cpp Src/Boundary/AMReX_YAFluxRegister.cpp commit 079f599f74057ea93d8c60b82760b8747b21849e Author: Weiqun Zhang Date: Tue Nov 17 18:40:51 2020 -0800 DPCPP: Fortran (#1554) Switch from gfortran to ifx that comes with oneapi in GNU make. Tools/GNUMake/comps/dpcpp.mak commit e38742e322f6fbdf0e0fb090271d229fc2c834b9 Author: mic84 Date: Tue Nov 17 13:26:24 2020 -0800 CMake: improvements to handling of buildInfo (#1552) Tools/CMake/AMReXBuildInfo.cmake commit 477df17dd96607c385fc9410c4fe7746f121b58d Author: Weiqun Zhang Date: Tue Nov 17 11:19:19 2020 -0800 Fix BaseFab::maxabs (#1546) ## Summary For the host version, call host function LoopOnCpu instead of host device function Loop because the function we pass to it is a host function. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_BaseFab.H commit cd6f4d0bdbc389f672e3d6a3b8d286729caca3b6 Author: Andrew Myers Date: Tue Nov 17 13:33:00 2020 -0500 Fix bug in the 'intarray' method of StructOfArrays (#1548) Thanks to @rporcu for finding this. The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Particle/AMReX_StructOfArrays.H commit b329cfe0bc4036aae7b69dc94b7bb34f8ab040c9 Author: Axel Huebl Date: Mon Nov 16 21:07:35 2020 -0800 CMake: generate_buildinfo target (#1545) This commit generates a static library `buildInfo::` for the passed user target. This target is then automatically linked. Also, the generated `.cpp` file is now scoped properly, avoiding collisions. This solves the following corner cases: - workflows with OBJECT libraries, that cannot be passed as targets since they have no `PRE_BUILD` custom command support - workflows with multiple targets that are not linked together (those call `generate_buildinfo` multiple times) - workflows with multiple targets that are linked together (those call `generate_buildinfo` once and re-use the target) Tools/CMake/AMReXBuildInfo.cmake commit 5076d33c583d88e0fb78214470715c2d09c083cd Author: Shreyas Ananthan Date: Mon Nov 16 10:00:02 2020 -0700 fcompare: fix logic error when comparing both rel/abs tolerances (#1542) PR #1537 introduced the option to allow tolerance checks on both absolute and relative tolerances. However, the check used `and` instead of `or` to allow tests to pass when either absolute or relative error was below user-specified tolerance. ## Checklist The proposed changes: - [X] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/Plotfile/fcompare.cpp commit 95b97799f2e611d19a4b685f725bfa3e5e12dc7f Author: Weiqun Zhang Date: Sun Nov 15 17:11:12 2020 -0800 nvcc --Werror cross-execution-space-call (#1540) * nvcc --Werror cross-execution-space-call * GNU make: Turn on the flag by default. Use `GPU_ERROR_CROSS_EXECUTION_SPACE_CALL=FALSE` to turn it off. * CMake: Add AMReX_ERROR_CROSS_EXECUTION_SPACE_CALL, which if off by default (just like AMReX_ERROR_CAPTURE_THIS). * update cmake documentation Docs/sphinx_documentation/source/GPU.rst Tools/CMake/AMReX_SetupCUDA.cmake Tools/GNUMake/Make.defs Tools/GNUMake/comps/nvcc.mak commit aef49304278572eb3c4a5db75462c8c213d35221 Author: Max Katz Date: Sun Nov 15 11:22:29 2020 -0800 Don't throw C++ std error when not config checking (#1543) Tools/GNUMake/comps/gnu.mak Tools/GNUMake/comps/nvcc.mak commit c80007df2f90e373f3d947fa7f621ab902378f3c Author: Shreyas Ananthan Date: Thu Nov 12 17:36:06 2020 -0700 fcompare: Add option for an absolute tolerance check (#1537) This PR adds a command line option to `fcompare` that takes in a user-specified tolerance for absolute error and adds logic to compare both absolute and relative errors against user-specified tolerances. Since `-a` is already used as a short-form for `--allow_diff_grids`, I have only used `--abs_tol` as the option without a short version when parsing command line options. Tools/Plotfile/fcompare.cpp commit cdb236ee435635f8007d2be97c0f6f1df13ecba0 Author: Weiqun Zhang Date: Thu Nov 12 08:50:35 2020 -0800 Remove call to cudaDeviceSetCacheConfig (#1536) ## Summary No longer set CUDA device cache configuration to prefer L1 cache. This does not appear to affect any kernels in a negative way. With more shared memory available, reduction functions are faster in some cases. ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_GpuDevice.cpp commit f55a950e0977a4b062bba6ff9787ba84e9f7da92 Author: Weiqun Zhang Date: Thu Nov 12 08:11:02 2020 -0800 Fix multi-component nodal solver tutorial (#1535) ## Summary `MultiFab::setVal` should not be called inside `MFIter`. ## Additional background This was caught by #1530. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tutorials/LinearSolvers/MultiComponent/MCNodalLinOp.cpp commit a01697fda4a2f38a1e7e3c849bc52baca2c2999d Author: Axel Huebl Date: Tue Nov 10 18:18:03 2020 -0800 Ascent: Check Bounds Particle Reals (#1515) Access of particle variable names with a bound-check, so users definitely pass the right amount of names. Src/Extern/Conduit/AMReX_Conduit_Blueprint_ParticlesI.H commit d9f60b4cb934852cb87653fbc84df22c98cfe0e0 Author: Andrew Myers Date: Tue Nov 10 09:02:30 2020 -0800 Allow for particle containers constructed with the same AmrCore... (#1532) Src/AmrCore/AMReX_AmrParGDB.H Src/Particle/AMReX_ParGDB.H Src/Particle/AMReX_Particles.H commit 10c2a5e5949213d607305f771edab433285166d7 Author: asalmgren Date: Mon Nov 9 17:58:38 2020 -0800 Update the docs to include hypre.adjust_singular_matrix (#1534) ## Summary adds documentation of hypre.adjust_singular_matrix ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [X] are described in the proposed changes to the AMReX documentation, if appropriate Docs/sphinx_documentation/source/LinearSolvers.rst commit 59b8bf1cc1ebf6dac76602a24a4ce696080b070c Author: Weiqun Zhang Date: Mon Nov 9 15:59:56 2020 -0800 DPC++: fix scan (#1533) * Use atomics instead of volatile to read status written to global memory by other blocks * Workaround `Random()` bug in the scan test * Limit the memory usage in the scan test Src/Base/AMReX_Scan.H Tutorials/GPU/ParallelScan/main.cpp commit 15ab2368225b81777e2810e0e02b1293364bb575 Author: Weiqun Zhang Date: Mon Nov 9 14:02:21 2020 -0800 Fix #1530 (#1531) `FillPatchIterator` is derived from `MFIter`. We need to reset `MFIter::depth` so that the ctor of `FillPatchIterator` can start `MFIter`. Src/Amr/AMReX_AmrLevel.cpp commit b94e1d2fca2bffd2b0e826eebfca7d63290efd40 Author: Weiqun Zhang Date: Mon Nov 9 10:30:06 2020 -0800 Assert no nested MFIters (#1530) ## Summary Add assertion to catch nested MFIters (e.g., MultiFab functions are called inside MFIter). ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_MFIter.H Src/Base/AMReX_MFIter.cpp commit ad7c4eac3475fb7ddfc2abe8c74400d0274f8398 Author: Weiqun Zhang Date: Mon Nov 9 10:29:06 2020 -0800 Add anonymous namespace back to parameters in AMReX_Amr.cpp (#1529) Src/Amr/AMReX_Amr.H Src/Amr/AMReX_Amr.cpp commit eb29a4f6b40e6458323db681795c1a12e1403c1c Author: Weiqun Zhang Date: Mon Nov 9 07:51:46 2020 -0800 Free MPI_Datatype defined by AMReX (#1527) ## Summary This fixes a one-time memory leak in MPI_Datatypes defined by AMReX as reported in #1525. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_ParallelDescriptor.cpp commit 0fd351b118b695d9c439c33a0f1ff9c6f3f00aeb Author: Weiqun Zhang Date: Sun Nov 8 10:27:36 2020 -0800 Update OLCF makefile (#1528) ## Summary * Remove summitdev and peak. * Environment variable OLCF_MODULEPATH_ROOT is used in addition to host name to detect OLCF machines. ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/GNUMake/Make.machines Tools/GNUMake/sites/Make.olcf commit 6cc5104e57548e575717051b04361ffed0fee9c2 Author: Weiqun Zhang Date: Sun Nov 8 10:26:56 2020 -0800 Remove getGridSize that is no longer used. (#1524) Src/Base/AMReX_GpuLaunch.H Src/Base/AMReX_GpuLaunch.cpp Src/Base/CMakeLists.txt Src/Base/Make.package commit 86abbe0221c1c3c53e69d740cdc24190ce1376c5 Author: Weiqun Zhang Date: Sat Nov 7 14:57:59 2020 -0800 Support CUDA arch < 6.0 (#1512) ## Summary * Implement atomicAdd for CUDA arch < 6.0. * Remove the CUDA arch check in CMake. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_BaseFab.H Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_GpuDevice.cpp Tools/CMake/AMReX_SetupCUDA.cmake commit ec4424eb66366c5b798639a100b30c6c01938f79 Author: Axel Huebl Date: Sat Nov 7 14:56:34 2020 -0800 Change Spelling: Env AMREX_CUDA_ARCH (#1522) ## Summary Make the _environment variable_ that sets a default CUDA architecture all-caps, as this is way more common in Unix. Typical Values: `7.0` or `Volta` (i.e. for V100) ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Docs/sphinx_documentation/source/GPU.rst Tools/CMake/AMReX_SetupCUDA.cmake commit 1b00f34ef1e170d4e2b398045e9dd45b4ba8ec34 Author: Weiqun Zhang Date: Fri Nov 6 18:18:27 2020 -0800 make clean & make cleanconfig (#1520) ## Summary `make clean` now does what `make realclean` does. `make realclean` is kept. `make cleanconfig` is introduced to do what `make clean` used to do. For most users, `make clean` is the one that should be used. Also add some tests in makefiles to avoid error messages in case when `make clean` is run on a makefile with the default not suitable for the system. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/GNUMake/Make.defs Tools/GNUMake/Make.rules Tools/GNUMake/README.md Tools/GNUMake/comps/hip.mak Tools/GNUMake/comps/nvcc.mak Tools/GNUMake/sites/Make.unknown commit d9ac9a6c844a5055cdd2b332d101096da3ae5d72 Author: mic84 Date: Fri Nov 6 13:47:58 2020 -0800 CMake: fix options name in Test directory (#1514) Tests/CMakeLists.txt Tests/HDF5Benchmark/CMakeLists.txt commit 6dc26f704a386a03120f71395f00186049499264 Author: mic84 Date: Fri Nov 6 13:47:15 2020 -0800 CMake: add component check for Fortran (#1511) Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/CMake/AMReXConfig.cmake.in commit 9f2822bdcdd5604eea85a6206413684fb0137ea4 Author: mic84 Date: Fri Nov 6 13:44:41 2020 -0800 CMake: fix some quirks with options output (#1516) Tools/CMake/AMReXOptions.cmake commit 79d991ca8518943155e312a9a7566380e563bc93 Author: Shreyas Ananthan Date: Fri Nov 6 12:41:36 2020 -0700 hypre: Adjust singular matrices before hypre solves (#1519) Src/Extern/HYPRE/AMReX_Hypre.H Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/HYPRE/AMReX_HypreIJIface.H Src/Extern/HYPRE/AMReX_HypreIJIface.cpp Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp commit fe6058b946fb4334b74c42fedde919836f82066d Author: Shreyas Ananthan Date: Fri Nov 6 12:04:55 2020 -0700 hypre: Update nodal Laplacian interface to accept custom options (#1518) HypreNodeLap class creates the hypre IJ instance in its constructor. Therefore, the custom option namespace specified by user must be part of the constructor arguments. Src/Extern/HYPRE/AMReX_HypreNodeLap.H Src/Extern/HYPRE/AMReX_HypreNodeLap.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLMG.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.cpp commit 7b7c1c8e6addc152fb9c0059577799bfe8dc6d93 Author: Weiqun Zhang Date: Thu Nov 5 16:27:28 2020 -0800 DPC++: sync after htod_memcpy on null stream (#1517) ## Summary DPC++ does not have the concept of null stream. Adding a sync after htod_memcpy on the "null" stream will help eliminate potential bugs (e.g., htod_memcpy is called before MFIter that uses non-null stream). ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_GpuDevice.H Src/Base/AMReX_Reduce.H commit 8035b01d86a7f8f4542bec6aa167dc864c8b3209 Author: Shreyas Ananthan Date: Thu Nov 5 15:06:36 2020 -0700 CMake: Fix ENABLE_HYPRE to AMReX_HYPRE (#1513) Fix the variable being checked to enable HYPRE. ## Checklist The proposed changes: - [X] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/CMake/AMReXThirdPartyLibraries.cmake commit 9aa75b1844e4ecc6c860a2d5d37b57bccedf73ed Author: Max Katz Date: Thu Nov 5 08:59:20 2020 -0800 Fix some build issues at LLNL (#1510) Tools/GNUMake/sites/Make.llnl commit 7ca532e5b299df1d7e9137e61f9271e4341c6189 Author: Andrew Myers Date: Thu Nov 5 08:54:57 2020 -0800 Add timer to neighbor list construction (#1509) Src/Particle/AMReX_NeighborList.H commit 86c6b3d00ae8d8e232b345b7060d9188fc838136 Author: mic84 Date: Wed Nov 4 19:39:39 2020 -0800 CMake: overhaul options (#1490) * Doc: move ENABLE_CUDA_FASTMATH to the GPU options section * CMake: change all options to include AMReX_ namespace * CMake: remove ENABLE_ACC option * CMake: make AMReX_SPACEDIM a multi-valued string * Doc: add option AMReX_ENABLE_TESTS * CMake: rename AMReX_Options.cmake to AMReXOptions.cmake * CMake: refactor GPU-related options * CMake: MPI must be turned off if DPCPP is enabled * CMake: set via precision via selection of value * CMake: forgot to update Particles/CMakeLists.txt * CMake: append prefix to TL_PROFILE * Update Docs/sphinx_documentation/source/AMReX_Profiling_Tools.rst Co-authored-by: Axel Huebl * Update Docs/sphinx_documentation/source/AMReX_Profiling_Tools.rst Co-authored-by: Axel Huebl * Update Docs/sphinx_documentation/source/BuildingAMReX.rst Co-authored-by: Axel Huebl * CMake: components in Config file shall not have namespace prefix * Revert "CMake: components in Config file shall not have namespace prefix" This reverts commit 924cf15f2a926467f3924a76f80595c0d1e53a7a. * Update Tools/CMake/AMReX_SetupCUDA.cmake Co-authored-by: Axel Huebl * CMake: update Doc * Update Tools/CMake/AMReX_SetupCUDA.cmake Co-authored-by: Axel Huebl Co-authored-by: Axel Huebl .github/workflows/linux.yml .github/workflows/macos.yml .github/workflows/windows.yml CHANGES CMakeLists.txt Docs/sphinx_documentation/source/AMReX_Profiling_Tools.rst Docs/sphinx_documentation/source/Basics.rst Docs/sphinx_documentation/source/BuildingAMReX.rst Docs/sphinx_documentation/source/GPU.rst Docs/sphinx_documentation/source/Visualization.rst Src/Amr/CMakeLists.txt Src/AmrCore/CMakeLists.txt Src/Base/CMakeLists.txt Src/Boundary/CMakeLists.txt Src/CMakeLists.txt Src/EB/CMakeLists.txt Src/Extern/HYPRE/CMakeLists.txt Src/Extern/PETSc/CMakeLists.txt Src/Extern/ProfParser/CMakeLists.txt Src/Extern/amrdata/CMakeLists.txt Src/F_Interfaces/CMakeLists.txt Src/LinearSolvers/CMakeLists.txt Src/Particle/CMakeLists.txt Tests/CMakeLists.txt Tools/CMake/AMReXConfig.cmake.in Tools/CMake/AMReXInstallHelpers.cmake Tools/CMake/AMReXOptions.cmake Tools/CMake/AMReXParallelBackends.cmake Tools/CMake/AMReXSYCL.cmake Tools/CMake/AMReXThirdPartyLibraries.cmake Tools/CMake/AMReX_Config.cmake Tools/CMake/AMReX_Defines.cmake Tools/CMake/AMReX_Options.cmake Tools/CMake/AMReX_SetupCUDA.cmake Tools/Plotfile/CMakeLists.txt Tutorials/Amr/Advection_AmrCore/CMakeLists.txt Tutorials/Amr/Advection_AmrLevel/CMakeLists.txt Tutorials/Basic/HeatEquation_EX1_C/CMakeLists.txt Tutorials/Basic/HeatEquation_EX1_CF/CMakeLists.txt Tutorials/Basic/HeatEquation_EX1_F/CMakeLists.txt Tutorials/Basic/HeatEquation_EX2_CF/CMakeLists.txt Tutorials/Basic/HeatEquation_EX3_C/CMakeLists.txt Tutorials/Basic/HelloWorld_F/CMakeLists.txt Tutorials/Basic/main_C/CMakeLists.txt Tutorials/Basic/main_F/CMakeLists.txt Tutorials/Blueprint/AssignMultiLevelDensity/CMakeLists.txt Tutorials/Blueprint/CellSortedParticles/CMakeLists.txt Tutorials/Blueprint/HeatEquation_EX1_C/CMakeLists.txt Tutorials/CMakeLists.txt Tutorials/EB/CNS/CMakeLists.txt Tutorials/EB/GeometryGeneration/CMakeLists.txt Tutorials/ForkJoin/MLMG/CMakeLists.txt Tutorials/ForkJoin/Simple/MyTest.cpp Tutorials/FortranInterface/Advection_F/CMakeLists.txt Tutorials/FortranInterface/Advection_octree_F/CMakeLists.txt Tutorials/FortranInterface/Advection_octree_F2/CMakeLists.txt Tutorials/GPU/CNS/CMakeLists.txt Tutorials/GPU/EBCNS/CMakeLists.txt Tutorials/GPU/Launch/CMakeLists.txt Tutorials/LinearSolvers/ABecLaplacian_C/CMakeLists.txt Tutorials/LinearSolvers/ABecLaplacian_F/CMakeLists.txt Tutorials/LinearSolvers/MAC_Projection_EB/CMakeLists.txt Tutorials/LinearSolvers/Nodal_Projection_EB/CMakeLists.txt Tutorials/Particles/CellSortedParticles/CMakeLists.txt Tutorials/Particles/ElectromagneticPIC/CMakeLists.txt Tutorials/Particles/ElectrostaticPIC/CMakeLists.txt Tutorials/Particles/NeighborList/CMakeLists.txt Tutorials/SUNDIALS/EX1_C/CMakeLists.txt commit 22e6f93f2f66d24e364f35226b3f0d990182e316 Author: Weiqun Zhang Date: Wed Nov 4 17:00:42 2020 -0800 New Reduce::Sum function taking lambda function (#1508) ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Reduce.H commit ae9d0c696dad43b4477517728bf0f2ef978165db Author: Weiqun Zhang Date: Wed Nov 4 14:50:18 2020 -0800 GNU Make: AMREX_CUDA_ARCH (#1507) ## Summary Option to use AMREX_CUDA_ARCH in addition to CUDA_ARCH to set CUDA arch. If AMREX_CUDA is set, it will be used over CUDA_ARCH. ## Additional background The CMake system is also planning to support AMREX_CUDA_ARCH. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [x] are described in the proposed changes to the AMReX documentation, if appropriate Docs/sphinx_documentation/source/BuildingAMReX.rst Tools/GNUMake/Make.defs commit 1ab6d82e749b039b0417fa6c539c04d464ba95ac Author: Andrew Myers Date: Wed Nov 4 10:54:39 2020 -0800 Fix bug in new assertion (#1506) Src/Particle/AMReX_ParticleContainerI.H commit db41d3d75449b162ebe4113656da49536149e165 Author: Max Katz Date: Wed Nov 4 09:23:41 2020 -0800 Add option for device sync at beginning and end of TinyProfiler region (#1505) ## Summary Now when setting `tiny_profiler.device_synchronize_on_region_end = 1` in the inputs file, we will synchronize before calling nvtxRangePop() and nvtxRangePush(), which means that TINY_PROFILE regions will include the full kernel time, rather than just the kernel launch time. ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_TinyProfiler.H Src/Base/AMReX_TinyProfiler.cpp commit 81fdb9a6254fee22daf3c32843c619e08d527bbc Author: Max Katz Date: Wed Nov 4 07:50:41 2020 -0800 Guard MFIter reduction macros against USE_GPU_PRAGMA (#1504) Src/Base/AMReX_BLFort.H commit 461ca5b42dfeb3ac24ab0aa344d3fc00129cc8b4 Author: Weiqun Zhang Date: Tue Nov 3 15:39:57 2020 -0800 GNU make: adjust multiple target rules (#1500) ## Summary Remove `/` from the object file name so that it works for multiple targets in different directories. This should not affect its current usage. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/GNUMake/Make.rules commit 3ed4b614a91e8b49d3e6f9b434493cf8f1c9464a Author: mic84 Date: Tue Nov 3 14:09:34 2020 -0800 CMake; bump up hypre minimi required version (#1503) Tools/CMake/AMReXThirdPartyLibraries.cmake commit f4c041f181dd1bb5743483e4b86f19cbb6e40fb2 Author: Andrew Myers Date: Tue Nov 3 13:54:38 2020 -0800 assert if aos.size() != soa.size() (#1502) Src/Particle/AMReX_ParticleContainerI.H commit b5a506997ce5db41bc46d6d64a238053fa2ea1ea Author: Weiqun Zhang Date: Tue Nov 3 13:53:23 2020 -0800 Fix warnings in AMReX_Machine.cpp (#1499) Src/Base/AMReX_GpuDevice.cpp Src/Base/AMReX_Machine.H Src/Base/AMReX_Machine.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp commit e61266427dc11ffdfd2bad6577ac204e19ad5e88 Author: Andrew Myers Date: Tue Nov 3 13:52:40 2020 -0800 Use Long type for pindex to avoid potential overflow. (#1501) Src/Particle/AMReX_ParticleContainerI.H commit 7117a8f9006ad2d647c3e6412aee3ec3047d6def Author: Weiqun Zhang Date: Tue Nov 3 09:01:25 2020 -0800 Update GNU make for Cori GPU node (#1497) ## Summary Set CUDA_ARCH to 80 for A100 on Cori. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/GNUMake/Make.machines Tools/GNUMake/sites/Make.nersc commit 24cacefc388e9811f52c1ebbceeb06cca90ba9b6 Author: Axel Huebl Date: Tue Nov 3 08:07:50 2020 -0800 Lin. Solvers (MLMG): Include Order (#1496) ## Summary Include own headers before stdlib headers to catch missing includes early. This avoid problems with less common compilers and environments. Carved out of #1485 ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/LinearSolvers/MLMG/AMReX_MLCGSolver.H Src/LinearSolvers/MLMG/AMReX_MLCGSolver.cpp Src/LinearSolvers/MLMG/AMReX_MLLinOp.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLaplacian.cpp commit ee3076a8306269bf6d18ffee3f4d4944c660241d Author: Max Katz Date: Tue Nov 3 08:06:58 2020 -0800 GNUMake: Merge Ascent and Peak with Summit (#1493) ## Summary These are all systems with the same architecture so we can share the build setup. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/GNUMake/sites/Make.olcf commit 43f112fe4b9deda918f3d3961d2979e124d0645d Author: Weiqun Zhang Date: Mon Nov 2 13:00:42 2020 -0800 Update CHANGES for AMReX 20.11 (#1494) CHANGES commit db23e834e9cdc24884b689e0bdbd285ba6124562 Author: Shreyas Ananthan Date: Mon Nov 2 11:00:19 2020 -0700 HYPRE: Fix issues with hypre IJ interface (#1495) This commit fixes two issues with the hypre IJ interface - Fixes a memory leak in HypreIJIface as it was calling parse_inputs in constructor - Sets default number of iterations and tolerance for BoomerAMG when used as a preconditioner Src/Extern/HYPRE/AMReX_HypreIJIface.cpp commit f29a0c9d1b8e6356f8007a0bcb9cfc1000f2a0ac Author: Weiqun Zhang Date: Fri Oct 30 15:45:23 2020 -0700 Change default hypre interface to ij for non-EB (#1492) Src/LinearSolvers/MLMG/AMReX_MLMG.H commit 93ffe6391b8a18c749610cabe05c785b6325378d Author: Weiqun Zhang Date: Fri Oct 30 10:39:39 2020 -0700 Fix DEBUG build for DPC++ (#1491) This fixes a bug introduced in #1489. Src/Base/AMReX.cpp commit 82c7a83ff6dd43be71d5960ad422f4db88d929e7 Author: Weiqun Zhang Date: Fri Oct 30 08:45:24 2020 -0700 Changes to make hipcc --save-temps work. (#1489) ## Summary It does not like AMREX_GPU_DEVICE_EXTERNAL. Also changed is the device version of `Error`, `Abort` and `Warning`. If `NDEBUG` is defined, they do not do anything. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX.H Src/Base/AMReX.cpp Tools/GNUMake/Make.defs commit 69c496f8e8e65f7fe4e9cd5afd4e9e89fa4ad9ba Author: Weiqun Zhang Date: Thu Oct 29 12:32:24 2020 -0700 Fix GPU race condition for EB (#1487) A GPU race condition was introduced in #1451 trying to fix faces and cells next to covered cells that were converted from small cut cells. It is fixed by launching a separate kernel for each direction. Src/EB/AMReX_EB2_2D_C.cpp Src/EB/AMReX_EB2_3D_C.cpp commit 147149885e7ca02dcfdc88f1f4953bf26dc10dc8 Author: Axel Huebl Date: Thu Oct 29 11:30:41 2020 -0700 Ascent: Use Default Name for Mesh Ghost (#1488) ## Summary Use the default name for ghost cells in Ascent. The current name is non-default and causes issues at domain-boundaries during volume rendering (unless one overwrites the `ghost_field_name` keyword in the options). Thanks to @mclarsen and @cyrush for debugging and hinting this with me. ## Additional background Seen with WarpX. Pre-PR: ![replay_000400(9)](https://user-images.githubusercontent.com/1353258/97613195-9eed1c80-19d5-11eb-9934-cafeffc51458.png) Post-PR: ![replay_000400(10)](https://user-images.githubusercontent.com/1353258/97614230-eb852780-19d6-11eb-8dcd-4ca4d86d2e21.png) ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp commit 32052a90a11da0dec54c353e0d6e29758e52b9a1 Author: Axel Huebl Date: Wed Oct 28 16:08:03 2020 -0700 CI: Check More Warnings (#1486) ## Summary Checks for more warnings that are likely to uncover nifty bugs: `-Wshadow -Woverloaded-virtual -Wunreachable-code` Also checks for superfluous `;` with `-Wextra-semi` (only a flag in Clang). ## Additional background Does not yet add `-Wall -Wextra -Wpedantic` because there are quite a few places where we need to add casts to avoid signed-unsigned-comparisons, etc. WarpX' CI covers currently: * GNU: `-Wall -Wextra -Wpedantic -Wshadow -Woverloaded-virtual -Wunreachable-code` * Clang: `-Wall -Wextra -Wpedantic -Wshadow -Woverloaded-virtual -Wextra-semi -Wunreachable-code` Even with super-builds, we current include AMReX with `-isystem` and build AMReX with default compiler flags; this enables us to suppress most AMReX build warnings besides the ones that drop-through from macros defined in headers, which atm. are all fixed for the code sections we use. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate .github/workflows/linux.yml .github/workflows/macos.yml Src/Particle/AMReX_NeighborParticles.H Tutorials/Basic/PrefixSum_MultiFab/main.cpp Tutorials/Particles/CellSortedParticles/cell_sorted_F.H Tutorials/Particles/ElectromagneticPIC/Exec/OpenMP/em_pic_F.H Tutorials/Particles/ElectrostaticPIC/ElectrostaticParticleContainer.cpp Tutorials/Particles/NeighborList/MDParticleContainer.cpp commit da1063cc5c16dcf519e74d630f898e7b23711c76 Author: Weiqun Zhang Date: Wed Oct 28 14:03:39 2020 -0700 HIP GNU Make: need to override COMP for regression tests (#1467) Tools/GNUMake/Make.defs commit ee020a50235341408297ff7018c536486eb30754 Author: Axel Huebl Date: Wed Oct 28 13:38:34 2020 -0700 CI: -Werror for host compilers (#1484) ## Summary Add more warning coverage in CI for CMake builds. ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate .github/workflows/linux.yml .github/workflows/macos.yml commit 0186e2f103f800280d8edfc5dffcd9280f132f42 Author: Weiqun Zhang Date: Mon Oct 26 11:18:29 2020 -0700 Fix warning in FabSet for single precision (#1481) * Fix warning in FabSet for single precision * Constify constants Co-authored-by: Axel Huebl Src/Boundary/AMReX_FabSet.cpp commit 6f07fe4e84a4dd57a0e89c2d2cb84c9953fda85d Author: Weiqun Zhang Date: Fri Oct 23 16:23:17 2020 -0700 Workaround for Mac CI (#1482) .github/workflows/dependencies/dependencies_mac.sh commit 61ef93db305fccfe81404686d6878f1f805fee02 Author: Axel Huebl Date: Fri Oct 23 13:29:11 2020 -0700 CMake 3.18+: CUDA Arch Policy (OLD) (#1480) Keep the old policy for now to avoid setting the code generation flags twice. We should be able to transition this well, but I need to first find out how device LTO generation flags are handled here, if at all, and how to restore the default-detection of the local GPU architecture, e.g. on Summits head-nodes. CMakeLists.txt commit f4d73901d209caa80951b4842f0e46848104a98f Author: asalmgren Date: Fri Oct 23 10:13:49 2020 -0700 More fixes to documenting hypre (previous text got lost). (#1477) Docs/sphinx_documentation/source/LinearSolvers.rst commit f42ab5658751c51d6792d7dd7f1999c4c3310143 Author: Luca Fedeli Date: Thu Oct 22 17:23:11 2020 +0200 Add fill method to GpuArray (#1472) ## Summary I would like to propose to add a `fill` method to `GpuArray`. This PR implements just that. ## Additional background I am wondering if it would be better to replace `i < N` with `i < amrex::max(N,std::size_t{1})`, in order to deal with the case `N=0`. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [X] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Array.H commit 0cfd7d349279040c1d9b9283f970b333296594d9 Author: asalmgren Date: Thu Oct 22 08:21:53 2020 -0700 Update the docs to reflect the new hypre options (#1473) Docs/sphinx_documentation/source/LinearSolvers.rst commit 6071cf447613beba8753fb0a5409dbe4e299111a Author: Weiqun Zhang Date: Tue Oct 20 18:10:01 2020 -0700 Gpu::Atomic::AddNoRet (#1469) Add Gpu::Atomic::AddNoRet to use HIP's atomicAddNoRet for float, which is much faster than atomicAdd for float that is currently implemented with CAS. Src/Base/AMReX_FBI.H Src/Base/AMReX_GpuAtomic.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_MultiFab.cpp Src/Base/AMReX_TypeTraits.H Src/EB/AMReX_EB_utils.cpp Src/Particle/AMReX_DenseBins.H Src/Particle/AMReX_Particle_mod_K.H Tests/Particles/NeighborParticles/MDParticleContainer.cpp Tests/Particles/ParticleMesh/main.cpp Tutorials/Particles/ElectromagneticPIC/Exec/CUDA/em_pic_K.H commit a004707203dcbd3e28c5e637c623d569301c64cd Author: Andy Nonaka Date: Tue Oct 20 18:01:29 2020 -0700 Enforce singular solvable (#1471) ## Summary Option to make the code NOT enforce solvability for singular problems. This is accomplished by calling linop.setEnforceSingularSolvable(false) The default behavior is true, so no results will change in existing codes ## Additional background ## Checklist The proposed changes: - [x] add new capabilities to AMReX Src/LinearSolvers/MLMG/AMReX_MLLinOp.H Src/LinearSolvers/MLMG/AMReX_MLMG.cpp commit 8350a1641044aa222d922e85bd082be5734e2a6a Author: mic84 Date: Tue Oct 20 17:15:07 2020 -0700 CMake: improve HDF5 support (#1468) * CMake: look only for the C component of HDF5 * CMake: use only C-related variables from FindHDF5 * CMake: add Tests/HDF5Benchmark to CTest * CMake: fix typo * Update Tools/CMake/AMReXThirdPartyLibraries.cmake * CMake: use hdf5 imported target if available Co-authored-by: Axel Huebl Tests/CMakeLists.txt Tests/HDF5Benchmark/CMakeLists.txt Tools/CMake/AMReXThirdPartyLibraries.cmake commit c0710a2bc4725aa8e7cab57690b27c73285436ae Author: Shreyas Ananthan Date: Sun Oct 18 14:16:56 2020 -0600 Use rocrand from HIP APT repository (#1466) .github/workflows/dependencies/dependencies_hip.sh commit 4f0052023f6c68ca2898ed99552e641a2e4dcc8f Author: Shreyas Ananthan Date: Sun Oct 18 14:16:03 2020 -0600 Address warnings in CUDA builds (#1465) Src/Base/AMReX.cpp Src/Base/AMReX_FBI.H Tools/Plotfile/fsnapshot.cpp commit aef02b9a4745005856a45417632b964a760dadb0 Author: mic84 Date: Sun Oct 18 11:03:49 2020 -0700 CMake: FindPETSc must not overwrite CMAKE_Fortran_FLAGS (#1464) ## Summary FindPETSc used to set ```CMAKE_Fortran_FLAGS``` to the list of PETSc include directories. This caused problems with the compilation of Fortran files (see Issue #1463 ). This PR fixes this. ## Additional background ## Checklist The proposed changes: - [X] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Tools/CMake/FindPETSc.cmake commit f64fb79d0f8a04a785e7c21f49dd81330ce55692 Author: Axel Huebl Date: Fri Oct 16 18:56:55 2020 -0700 CI: rocRAND v1.8.2 (#1462) Building the development branch is a bit unstable. .github/workflows/dependencies/dependencies_hip.sh commit 0cea78bf2f4f24568a1d71d20e8b2614fe19012c Author: mic84 Date: Fri Oct 16 17:33:42 2020 -0700 CMake: some cleanup (#1460) * CMake: provide function to retrieve AMReX version * CMake: remove unused helper functions CMakeLists.txt Tools/CMake/AMReX_Utils.cmake commit 931cf5993b8ad2d5fda1a853ebabb36dbe2ae5c2 Author: Weiqun Zhang Date: Fri Oct 16 17:30:50 2020 -0700 DPCPP: Scan (#1461) ## Summary Need to use global address space when doing atomic inc on virtual block id. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Scan.H commit 31ca23418f488fe0b309ef3fc3e99c522ced4921 Author: mic84 Date: Fri Oct 16 14:53:58 2020 -0700 [WIP] CMake: add support for dpc++ beta10 (#1459) * CMake: retrieve dpcpp version * CMake: modify dpc++ flags to account for beta10 version Tools/CMake/AMReXSYCL.cmake commit d036a6df1c6f8f7bb458c7e5253a91f4b6f96b83 Author: Max Katz Date: Fri Oct 16 14:21:28 2020 -0700 Fix potential out of bounds access in #1441 (#1458) ## Summary The user may have given us a vector that has too many values, so we just ignore the remainder. ## Additional background ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/AmrCore/AMReX_ErrorList.H commit 18b293dad9199deb329efecddd56ccce709371f5 Author: Weiqun Zhang Date: Fri Oct 16 10:12:19 2020 -0700 DPCPP beta10 (#1456) ## Summary * Update for DPCPP beta10 * Fix some warnings * Remove some device versions of Random() for HIP and DPC++ ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Base/AMReX_Extension.H Src/Base/AMReX_FBI.H Src/Base/AMReX_GpuPrint.H Src/Base/AMReX_GpuReduce.H Src/Base/AMReX_Random.H Src/Base/AMReX_Random.cpp Src/Base/AMReX_RandomEngine.H Src/Base/AMReX_Scan.H Src/Particle/AMReX_BinIterator.H Src/Particle/AMReX_ParticleCommunication.H Src/Particle/AMReX_ParticleContainerI.H Tools/GNUMake/comps/dpcpp.mak commit 15643f4b0ab28dc6b3277c5c44c18ac1f80e1345 Author: mic84 Date: Thu Oct 15 16:21:13 2020 -0700 CMake: prevent in-source builds (#1453) * CMake: prevent in-source builds * CMake: make error message more detailed CMakeLists.txt commit c53b1bbe88bed6b731bae4e11fd9ed0883a6b814 Author: Andy Nonaka Date: Thu Oct 15 14:29:41 2020 -0700 change char * to char const* so SWFFT code compiles with USE_CUDA=TRUE (#1454) Src/Extern/SWFFT/distribution.c commit 48b0a20210b9825c1c403e1e0b8a741ba913dffa Author: Shreyas Ananthan Date: Wed Oct 14 15:00:12 2020 -0600 Make HypreABecLap3::getSolution public for CUDA builds (#1452) Src/Extern/HYPRE/AMReX_HypreABecLap3.H commit 5843b838c1a86ea7a7978d0b6edd9f5e0c03d24a Author: asalmgren Date: Wed Oct 14 09:10:35 2020 -0700 Fix when small_volfrac is used -- we need to adjust the data in cut c… (#1451) Fix when small_volfrac is used -- we need to adjust the data in cut cell neighbors as well as regular neighbors This addresses issue #1450 ## Checklist The proposed changes: - [X] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/EB/AMReX_EB2_2D_C.cpp Src/EB/AMReX_EB2_3D_C.cpp commit 79f3a94fd50b6e9bcac6f99d0b03ffb47e9a5a87 Author: Andrew Myers Date: Wed Oct 14 00:39:23 2020 -0700 Allow users to set a Geometry object for ParticleContainer independently of the AmrCore / AmrLevel object. (#1446) Src/AmrCore/AMReX_AmrParGDB.H Src/Particle/AMReX_ParGDB.H Src/Particle/AMReX_Particles.H commit ff17a9295b54a36fa78c4f3c2350c3b3b5dd1338 Author: Houjun Date: Tue Oct 13 12:14:50 2020 -0700 HDF5 plotfile write bug fix and performance improvement (#1448) * Add HDF5 optimizations * Fix an issue that causes incorrect HDF5 data write with Tutorials/Basic/HeatEquation_EX1_C example code Co-authored-by: Houjun Tang Src/Base/AMReX_PlotFileUtil.cpp Src/Particle/AMReX_ParticleHDF5.H Tests/HDF5Benchmark/GNUmakefile commit 6649d227425fdc8ecbed301c4383c8dc74800323 Author: mic84 Date: Tue Oct 13 11:52:09 2020 -0700 CMake: add HIP support (#1316) * CMake: first basic attempt at HIP support * Update Tools/CMake/AMReXParallelBackends.cmake Co-authored-by: Axel Huebl * CI: Add HIP-Clang * fix typo: GCC 9.3 on focal * Fix sources: set two lists * fix target properties: append two lists * CMake: some changes to debug issue with generated files * CI: only run HIP check for now * CMake: fix typo * CMake: let's try this * CMake: other small modifications * CMake: attemp #1 * CMake: attemp #2 * CI: build just the bare minimum for now * CMake: attempt #3 * CMake: missing defines for HIP builds * CMake: defines were still missing :-P * CMake: AMREX_HIP_PLATFORM is actually the HIP_COMPILER * CMake: let's see if this works * Install rocRAND * CMake: re-factor and update defines for HIP compilation * CMake: list against hiprand too * CMake: find and link to hiprand * CMake: add -DNDEBUG to any type of build when HIP is on * CMake: we must manually add rocrand too * CMake: let's see if it works with Fortran enabled * Revert "CMake: let's see if it works with Fortran enabled" This reverts commit 065071539fd166f3bee994c2621d86c773180e3d. * CMake: try this * CMake: this should work too * Revert "Revert "CMake: let's see if it works with Fortran enabled"" This reverts commit 6a92adcc4ddfb06b4130f10d29cc49b298a66e73. * CMake: no fortran for now * CI: turn on linear solvers in HIP check * CI: turn on particles in HIP checks * Revert "CI: turn on linear solvers in HIP check" This reverts commit c723560a7b401266d6259af340130b1065e6be59. The reason is that Linear Solvers tutorials are not HIP-aware yet. * CMake: hipify tutorials * CMake: temporary fix * Revert "CMake: temporary fix" This reverts commit bd6724ed73f7afc82a4dc60c60d4390c6a83808d. * CMake: fix HIP compilation for Tutorials * CMake: some cleanup * CMake: add HIP options to pass in architecture and extra flags * CMake: oops * CMake: try to enable linear solvers * CMake: update ROCm version number in CI check name * CMake: enable Fortran * CMake: disable tutorials for the time being * CMake: commit custom FindHIP.cmake to debug the issue * CMake: trying this * CMake: now it should use the local FindHIP.cmake * CMake: now custom FindHIP.cmake should be able to find helper files * CMake: let's see if we really need this * CMake: let's try this * CMake: temporary fix * Revert "CMake: disable tutorials for the time being" This reverts commit 885b9113b72f54bca9eaf74f4e5b6b90b17032ff. * CMake: Tutorials/Particles/CellSortedParticles do not work with HIP * CMake: remove local FindHIP.cmake. * CMake: HIP arch flags are now PUBLIC and inheritable * CMake: no use for setup_target_for_hip_compilation anymore * CMake: remove unnecessary options for HIP * CMake: check that HIP_COMPILER is the same as CMAKE_CXX_COMPILER * CMake: better this * CI: uncomment all checks * CMake: fix visibility of arch flags for HIP * Doc: add subsection on HIP+CMake * Update Docs/sphinx_documentation/source/GPU.rst Co-authored-by: Shreyas Ananthan * CMake: fix incorrect configuration option passed to cmake * CI: target 'tutorials' no longer exists Co-authored-by: Axel Huebl Co-authored-by: Shreyas Ananthan .github/workflows/dependencies/dependencies_hip.sh .github/workflows/linux.yml Docs/sphinx_documentation/source/GPU.rst Src/CMakeLists.txt Tools/CMake/AMReXParallelBackends.cmake Tools/CMake/AMReXTargetHelpers.cmake Tools/CMake/AMReX_Defines.cmake Tools/CMake/AMReX_Options.cmake Tutorials/CMakeLists.txt Tutorials/Particles/CellSortedParticles/CMakeLists.txt commit 6dc6ce8f55a5ef19f349c267b7bb9fc4bb616b07 Author: Candace Gilet Date: Tue Oct 13 09:48:50 2020 -0400 volume multifab must be defined before calling GetVolume(volume) (#1449) Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp commit 5e6d555d07d3862b5b88b6c38559a59918541194 Author: mic84 Date: Mon Oct 12 17:38:22 2020 -0700 NodalProjector: use volume-weighted average down (#1444) Src/LinearSolvers/Projections/AMReX_NodalProjector.cpp commit 95ddfdb2c1434b1705d84174047f06cd3004174d Author: Weiqun Zhang Date: Wed Oct 7 19:02:08 2020 -0700 Fix EB interpolation from cell centers to faces (#1443) ##Summary There were race conditions that made it fail on AMD GPUs. ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/EB/AMReX_EBMultiFabUtil.cpp Src/EB/AMReX_EBMultiFabUtil_2D_C.H Src/EB/AMReX_EBMultiFabUtil_3D_C.H commit 2c7b5004fb98d92bf185ecc73b6429dfd7ed8b6f Author: Weiqun Zhang Date: Wed Oct 7 15:16:51 2020 -0700 Fix bug in hypre overset solver (#1442) ## Summary ## Additional background ## Checklist The proposed changes: - [x] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/Extern/HYPRE/AMReX_Habec_2D_K.H Src/Extern/HYPRE/AMReX_Habec_3D_K.H Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/PETSc/AMReX_PETSc.cpp commit 3972c6d582f024c4a4d108ff02e66d117e6db1e6 Author: Max Katz Date: Wed Oct 7 15:33:35 2020 -0400 Allow tagging value to vary by level (#1441) ## Summary The new error tagging scheme from #1166 is modified to allow the threshold value to vary by level. Using the example there, if we did: ``` amr.refinement_indicators = flame_tracer lo_temp amr.refine.flame_tracer.max_level = 3 amr.refine.flame_tracer.value_greater = 1.e-6 1.e-5 amr.refine.flame_tracer.field_name = Y(H) amr.refine.lo_temp.max_level = 2 amr.refine.lo_temp.value_less = 1000. amr.refine.lo_temp.field_name = temp ``` Then level 0 would be tagged for refinement for Y(H) >= 1.e-6, while level 1 and all higher levels would be tagged for refinement for Y(H) >= 1.e-5. (If a value for every level is not provided, we assume the last value holds for all remaining levels, similar to how we treat quantities like amr.n_error_buf.) ## Additional background This will help for cases where a field varies substantially on the domain and you want to have better control over refinement so that you don't have too many zones. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] are described in the proposed changes to the AMReX documentation, if appropriate Src/AmrCore/AMReX_ErrorList.H Src/AmrCore/AMReX_ErrorList.cpp commit 92399587f20875a98655ccbd06a083965cfc9c5c Author: Weiqun Zhang Date: Tue Oct 6 13:30:52 2020 -0700 Hypre with overset (#1439) * Hypre with overset Add overset mask support for hypre. * Keep bottom_verbose restricted to amrex side. Hypre side's verbosity will be controlled by parmparse parameters. Src/Extern/HYPRE/AMReX_Habec_2D_K.H Src/Extern/HYPRE/AMReX_Habec_3D_K.H Src/Extern/HYPRE/AMReX_Hypre.H Src/Extern/HYPRE/AMReX_Hypre.cpp Src/Extern/HYPRE/AMReX_HypreABecLap.cpp Src/Extern/HYPRE/AMReX_HypreABecLap3.H Src/Extern/HYPRE/AMReX_HypreABecLap3.cpp Src/Extern/HYPRE/AMReX_HypreIJIface.cpp Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.H Src/LinearSolvers/MLMG/AMReX_MLCellABecLap.cpp Src/LinearSolvers/MLMG/AMReX_MLNodeLinOp.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_2D_K.H Src/LinearSolvers/MLMG/AMReX_MLNodeTensorLap_3D_K.H Tests/LinearSolvers/CellOverset/MyTest.H Tests/LinearSolvers/CellOverset/MyTest.cpp commit 6a20c352cf54ed9a1393676b1bf19c57941db2af Author: Axel Huebl Date: Mon Oct 5 16:17:43 2020 -0700 GNUmake: -Titan (OLCF) (#1440) R.I.P. Titan. Btw, only found because a colleague of mine could not compile WarpX. Their hostname is called -titan. Tools/GNUMake/Make.machines Tools/GNUMake/sites/Make.olcf commit 9992d844099c3df81d9499519ffeb7f93a201dc6 Author: Shreyas Ananthan Date: Sun Oct 4 18:19:24 2020 -0600 Hypre IJ interface: Enable access to additional solvers and preconditioners available in Hypre (#1437) This PR extends the AMReX _hypre_ IJ interface to allow access to additional solvers and preconditioners available in _hypre_. - Refactored `HypreABecLap3` and `HypreNodeLap` (IJ matrix/vector ParCSR interfaces) and created a new IJ interface class that provides a unified way to access Hypre ParCSR solvers. - In addition to BoomerAMG, adds support for 7 other [ParCSR solvers](https://hypre.readthedocs.io/en/latest/ch-solvers.html) (GMRES (4 variants), PCG, BiCGSTAB, and Hybrid). Also adds support for BoomerAMG and Euclid as preconditioners for solvers that can use a preconditioner. - Adds support to parse user options to configure hypre library via AMReX ParmParse interface and input file, using a custom namespace (default is `hypre`) - Adds `MLMG::setHypreOptionsNamespace` that allow applications to customize the namespace for different types of linear solvers (e.g., `mac_proj.hypre.