Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions doxygen/parallelism_support/threading_tbb.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,20 @@ In addition to making `stan-math` thread safe this also turns on parallel execut

The Intel TBB library is used in stan-math since version 2.21.0. The Intel TBB library uses a threadpool internally and distributes work through a task-based approach. The tasks are dispatched to the threadpool via a the Intel TBB work-stealing scheduler. For example, whenever threading is enabled via `STAN_THREADS` the `map_rect` function in stan-math will use the `tbb::parallel_for` of the TBB. This will execute the work chunks given to `map_rect` with scheduling and thus load-balance CPU core utilization.

By default stan-math builds only the main `tbb` library by defining the `makefile` variable
The Intel TBB provides in addition to the main library memory allocators which are specifically designed to speedup threaded programs. On Linux and Windows stan-math builds and links the main `tbb` library together with these allocators by defining the `makefile` variable

```
TBB_LIBRARIES=tbb
TBB_LIBRARIES=tbb tbbmalloc tbbmalloc_proxy
```

The Intel TBB provides in addition to the main library memory allocators which are specifically designed to speedup threaded programs. These speedups have so far only been observed on MacOS systems for Stan programs such that on MacOS the default is set to
On MacOS the default is to build only the main `tbb` library

```
TBB_LIBRARIES=tbb tbbmalloc tbbmalloc_proxy
TBB_LIBRARIES=tbb
```

because on MacOS `tbbmalloc_proxy` registers a process-wide malloc zone which can crash host processes (such as Julia, Python, or R) that load a Stan model as a shared library. See [issue #3425](https://github.com/stan-dev/math/issues/3425) for details.

Users may override the default choices by defining `TBB_LIBRARIES` in the `make/local` file manually. Please refer to the [pull request](https://github.com/stan-dev/math/pull/1376) which merged the Intel TBB for further details on the performance evaluations.

# Requirements
Expand Down
17 changes: 11 additions & 6 deletions make/compiler_flags
Original file line number Diff line number Diff line change
Expand Up @@ -272,20 +272,25 @@ endif
# Sets up TBB CXXFLAGS_TBB and LDFLAGS_TBB to compile and link to TBB
#
# The tbbmalloc and tbbmalloc_proxy libraries are optionally included
# as targets. By default these are included on MacOS only. This behavior
# can be altered by explicitly setting the TBB_LIBRARIES variable which
# should contain "tbb" or "tbb tbbmalloc tbbmalloc_proxy". Setting the
# TBB_LIBRARIES variable overrides the default.
# as targets. By default these are included on Linux and Windows but not
# on MacOS. This behavior can be altered by explicitly setting the
# TBB_LIBRARIES variable which should contain "tbb" or
# "tbb tbbmalloc tbbmalloc_proxy". Setting the TBB_LIBRARIES variable
# overrides the default.

TBB_BIN ?= $(MATH)lib/tbb
TBB_RELATIVE_PATH ?= ../$(notdir $(TBB))
TBB_BIN_ABSOLUTE_PATH = $(abspath $(TBB_BIN))
TBB_ABSOLUTE_PATH = $(abspath $(TBB))

# On MacOS tbbmalloc_proxy registers a process-wide malloc zone that can
# crash host processes (Julia, Python, R) which load Stan as a shared library
# and free memory from the system allocator.
# See https://github.com/stan-dev/math/issues/3425
ifeq ($(OS),Darwin)
TBB_LIBRARIES ?= tbb tbbmalloc tbbmalloc_proxy
else
TBB_LIBRARIES ?= tbb
else
TBB_LIBRARIES ?= tbb tbbmalloc tbbmalloc_proxy
Comment on lines +292 to +293

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were previously only linking these on macos, so adding them to other platforms seems wrong

endif

ifdef TBB_LIB
Expand Down
Loading