Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/maximmasiutin/FastMM4-AVX

FastMM4 memory manager for Delphi and FreePascal (free pascal/Lazarus). A fork with improved synchronization between the threads that gives performance benefits on thread-heavy applications. Proper synchronization techniques are used depending on context and availability, i.e. umonitor/umwait, spin-wait loops, SwitchToThread, critical sections, etc
https://github.com/maximmasiutin/FastMM4-AVX

Last synced: about 1 month ago
JSON representation

Host: GitHub
URL: https://github.com/maximmasiutin/FastMM4-AVX
Owner: maximmasiutin
License: gpl-3.0
Fork: true (pleriche/FastMM4)
Created: 2017-06-16T22:04:41.000Z (almost 7 years ago)
Default Branch: master
Last Pushed: 2024-01-11T01:27:14.000Z (5 months ago)
Last Synced: 2024-01-11T04:26:15.905Z (5 months ago)
Language: Pascal
Homepage:
Size: 10.2 MB
Stars: 128
Watchers: 24
Forks: 16
Open Issues: 1
Metadata Files:
- Readme: README.md
- License: license_gpl.txt

Lists

awesome-pascal - FastMM4-AVX - threaded enhancements (faster locking) (Memory managers)

README

# FastMM4-AVX

FastMM4-AVX (efficient synchronization and AVX1/AVX2/AVX512/ERMS/FSRM support for FastMM4)
- Copyright (C) 2017-2020 Ritlabs, SRL. All rights reserved.
- Copyright (C) 2020-2023 Maxim Masiutin. All rights reserved.

Written by Maxim Masiutin

Version 1.0.7

This is a fork of the "Fast Memory Manager" (FastMM) v4.993 by Pierre le Riche
(see below for the original FastMM4 description)

What was added to FastMM4-AVX in comparison to the original FastMM4:

- Efficient synchronization
- improved synchronization between the threads; proper synchronization
techniques are used depending on context and availability, i.e., pause-
based spin-wait loops, umonitor/umwait (WaitPKG), SwitchToThread,
critical sections, etc.;
- used the "test, test-and-set" technique for the spin-wait loops; this
technique is recommended by Intel (see Section 11.4.3 "Optimization with
Spin-Locks" of the Intel 64 and IA-32 Architectures Optimization Reference
Manual) to determine the availability of the synchronization variable;
according to this technique, the first "test" is done via the normal
(non-locking) memory load to prevent excessive bus locking on each
iteration of the spin-wait loop; if the variable is available upon
the normal memory load of the first step ("test"), proceed to the
second step ("test-and-set") which is done via the bus-locking atomic
"xchg" instruction; however, this two-steps approach of using "test" before
"test-and-set" can increase the cost for the un-contended case comparing
to just single-step "test-and-set", this may explain why the speed benefits
of the FastMM4-AVX are more pronounced when the memory manager is called
from multiple threads in parallel, while in single-threaded use scenario
there may be no benefit compared to the original FastMM4;
- the number of iterations of "pause"-based spin-wait loops is 5000,
before relinquishing to SwitchToThread();
- see https://stackoverflow.com/a/44916975 for more details on the
implementation of the "pause"-based spin-wait loops;
- using normal memory store to release a lock:
FastMM4-AVX uses normal memory store, i.e., the "mov" instruction, rather
then the bus-locking "xchg" instruction to write into the synchronization
variable (LockByte) to "release a lock" on a data structure,
see https://stackoverflow.com/a/44959764
for discussion on releasing a lock;
you man define "InterlockedRelease" to get the old behavior of the original
FastMM4.
- implemented dedicated lock and unlock procedures that operate with
synchronization variables (LockByte);
before that, locking operations were scattered throughout the code;
now the locking functions have meaningful names:
AcquireLockByte and ReleaseLockByte;
the values of the lock byte are now checked for validity when
FullDebugMode or DEBUG is defined, to detect cases when the same lock is
released twice, and other improper use of the lock bytes;
- added compile-time options "SmallBlocksLockedCriticalSection",
"MediumBlocksLockedCriticalSection" and "LargeBlocksLockedCriticalSection"
which are set by default (inside the FastMM4Options.inc file) as
conditional defines. If you undefine these options, you will get the
old locking mechanism of the original FastMM4 based on loops of Sleep() or
SwitchToThread().

- AVX, AVX2 or AVX512 instructions for faster memory copy
- if the CPU supports AVX or AVX2, use the 32-byte YMM registers
for faster memory copy, and if the CPU supports AVX-512,
use the 64-byte ZMM registers for even faster memory copy;
- please note that the effect of using AVX instruction in speed improvement is
negligible, compared to the effect brought by efficient synchronization;
sometimes AVX instructions can even slow down the program because of AVX-SSE
transition penalties and reduced CPU frequency caused by AVX-512
instructions in some processors; use DisableAVX to turn AVX off completely
or use DisableAVX1/DisableAVX2/DisableAVX512 to disable separately certain
AVX-related instruction set from being compiled);
- if EnableAVX is defined, all memory blocks are aligned by 32 bytes, but
you can also use Align32Bytes define without AVX; please note that the memory
overhead is higher when the blocks are aligned by 32 bytes, because some
memory is lost by padding; however, if your CPU supports
"Fast Short REP MOVSB" (Ice Lake or newer), you can disable AVX, and align
by just 8 bytes, and this may even be faster because less memory is wasted
on alignment;
- with AVX, memory copy is secure - all XMM/YMM/ZMM registers used to copy
memory are cleared by vxorps/vpxor, so the leftovers of the copied memory
are not exposed in the XMM/YMM/ZMM registers;
- the code attempts to properly handle AVX-SSE transitions to not incur the
transition penalties, only call vzeroupper under AVX1, but not under AVX2
since it slows down subsequent SSE code under Skylake / Kaby Lake;
- on AVX-512, writing to xmm16-xmm31 registers will not affect the turbo
clocks, and will not impose AVX-SSE transition penalties; therefore, when we
have AVX-512, we now only use x(y/z)mm16-31 registers.

- Speed improvements due to code optimization and proper techniques
- if the CPU supports Enhanced REP MOVSB/STOSB (ERMS), use this feature
for faster memory copy (under 32 bit or 64-bit) (see the EnableERMS define,
on by default, use DisableERMS to turn it off);
- if the CPU supports Fast Short REP MOVSB (FSRM), uses this feature instead
of AVX;
- branch target alignment in assembly routines is only used when
EnableAsmCodeAlign is defined; Delphi incorrectly encodes conditional
jumps, i.e., use long, 6-byte instructions instead of just short, 2-byte,
and this may affect branch prediction, so the benefits of branch target
alignment may not outweigh the disadvantage of affected branch prediction,
see https://stackoverflow.com/q/45112065
- compare instructions + conditional jump instructions are put together
to allow macro-op fusion (which happens since Core2 processors, when
the first instruction is a CMP or TEST instruction and the second
instruction is a conditional jump instruction);
- multiplication and division by a constant, which is a power of 2
replaced to shl/shr, because Delphi64 compiler doesn't replace such
multiplications and divisions to shl/shr processor instructions,
and, according to the Intel Optimization Reference Manual, shl/shr is
faster than imul/idiv, at least for some processors.

- Safer, cleaner code with stricter type adherence and better compatibility
- names assigned to some constants that used to be "magic constants",
i.e., unnamed numerical constants - plenty of them were present
throughout the whole code;
- removed some typecasts; the code is stricter to let the compiler
do the job, check everything and mitigate probable error. You can
even compile the code with "integer overflow checking" and
"range checking", as well as with "typed @ operator" - for safer
code. Also added round bracket in the places where the typed @ operator
was used, to better emphasize on who's address is taken;
- the compiler environment is more flexible now: you can now compile FastMM4
with, for example, typed "@" operator or any other option. Almost all
externally-set compiler directives are honored by FastMM except a few
(currently just one) - look for the "Compiler options for FastMM4" section
below to see what options cannot be externally set and are always
redefined by FastMM4 for itself - even if you set up these compiler options
differently outside FastMM4, they will be silently
redefined, and the new values will be used for FastMM4 only;
- the type of one-byte synchronization variables (accessed via "lock cmpxchg"
or "lock xchg") replaced from Boolean to Byte for stricter type checking;
- those fixed-block-size memory move procedures that are not needed
(under the current bitness and alignment combinations) are
explicitly excluded from compiling, to not rely on the compiler
that is supposed to remove these function after compilation;
- added length parameter to what were the dangerous null-terminated string
operations via PAnsiChar, to prevent potential stack buffer overruns
(or maybe even stack-based exploitation?), and there some Pascal functions
also left, the argument is not yet checked. See the "todo" comments
to figure out where the length is not yet checked. Anyway, since these
memory functions are only used in Debug mode, i.e., in development
environment, not in Release (production), the impact of this
"vulnerability" is minimal (albeit this is a questionable statement);
- removed all non-US-ASCII characters, to avoid using UTF-8 BOM, for
better compatibility with very early versions of Delphi (e.g., Delphi 5),
thanks to Valts Silaputnins;
- support for Lazarus 1.6.4 with FreePascal (the original FastMM4 4.992
requires modifications, it doesn't work under Lazarus 1.6.4 with FreePascal
out-of-the-box, also tested under Lazarus 1.8.2 / FPC 3.0.4 with Win32
target; later versions should be also supported.

Here are the comparison of the Original FastMM4 version 4.992, with default
options compiled for Win64 by Delphi 10.2 Tokyo (Release with Optimization),
and the current FastMM4-AVX branch ("AVX-br."). Under some multi-threading
scenarios, the FastMM4-AVX branch is more than twice as fast compared to the
Original FastMM4. The tests have been run on two different computers: one
under Xeon E5-2543v2 with 2 CPU sockets, each has 6 physical cores
(12 logical threads) - with only 5 physical core per socket enabled for the
test application. Another test was done under an i7-7700K CPU.

Used the "Multi-threaded allocate, use and free" and "NexusDB"
test cases from the FastCode Challenge Memory Manager test suite,
modified to run under 64-bit.

Xeon E5-2543v2 2*CPU i7-7700K CPU
(allocated 20 logical (8 logical threads,
threads, 10 physical 4 physical cores),
cores, NUMA), AVX-1 AVX-2

Orig. AVX-br. Ratio Orig. AVX-br. Ratio
------ ----- ------ ----- ----- ------
02-threads realloc 96552 59951 62.09% 65213 49471 75.86%
04-threads realloc 97998 39494 40.30% 64402 47714 74.09%
08-threads realloc 98325 33743 34.32% 64796 58754 90.68%
16-threads realloc 116273 45161 38.84% 70722 60293 85.25%
31-threads realloc 122528 53616 43.76% 70939 62962 88.76%
64-threads realloc 137661 54330 39.47% 73696 64824 87.96%
NexusDB 02 threads 122846 90380 73.72% 79479 66153 83.23%
NexusDB 04 threads 122131 53103 43.77% 69183 43001 62.16%
NexusDB 08 threads 124419 40914 32.88% 64977 33609 51.72%
NexusDB 12 threads 181239 55818 30.80% 83983 44658 53.18%
NexusDB 16 threads 135211 62044 43.61% 59917 32463 54.18%
NexusDB 31 threads 134815 48132 33.46% 54686 31184 57.02%
NexusDB 64 threads 187094 57672 30.25% 63089 41955 66.50%

The above tests have been run on 14-Jul-2017.

Here are some more test results (Compiled by Delphi 10.2 Update 3):

Xeon E5-2667v4 2*CPU i9-7900X CPU
(allocated 32 logical (20 logical threads,
threads, 16 physical 10 physical cores),
cores, NUMA), AVX-2 AVX-512

Orig. AVX-br. Ratio Orig. AVX-br. Ratio
------ ----- ------ ----- ----- ------
02-threads realloc 80544 60025 74.52% 66100 55854 84.50%
04-threads realloc 80751 47743 59.12% 64772 40213 62.08%
08-threads realloc 82645 32691 39.56% 62246 27056 43.47%
12-threads realloc 89951 43270 48.10% 65456 25853 39.50%
16-threads realloc 95729 56571 59.10% 67513 27058 40.08%
31-threads realloc 109099 97290 89.18% 63180 28408 44.96%
64-threads realloc 118589 104230 87.89% 57974 28951 49.94%
NexusDB 01 thread 160100 121961 76.18% 93341 95807 102.64%
NexusDB 02 threads 115447 78339 67.86% 77034 70056 90.94%
NexusDB 04 threads 107851 49403 45.81% 73162 50039 68.39%
NexusDB 08 threads 111490 36675 32.90% 70672 42116 59.59%
NexusDB 12 threads 148148 46608 31.46% 92693 53900 58.15%
NexusDB 16 threads 111041 38461 34.64% 66549 37317 56.07%
NexusDB 31 threads 123496 44232 35.82% 62552 34150 54.60%
NexusDB 64 threads 179924 62414 34.69% 83914 42915 51.14%

The above tests (on Xeon E5-2667v4 and i9) have been done on 03-May-2018.

Here is the single-threading performance comparison in some selected
scenarios between FastMM v5.03 dated May 12, 2021 and FastMM4-AVX v1.05
dated May 20, 2021. FastMM4-AVX is compiled with default optinos. This
test is run on May 20, 2021, under Intel Core i7-1065G7 CPU, Ice Lake
microarchitecture, base frequency: 1.3 GHz, max turbo frequencey: 3.90 GHz,
4 cores, 8 threads. Compiled under Delphi 10.3 Update 3, 64-bit target.
Please note that these are the selected scenarios where FastMM4-AVX is
faster then FastMM5. In other scenarios, especially in multi-threaded
with heavy contention, FastMM5 is faster.

FastMM5 AVX-br. Ratio
------ ------ ------
ReallocMem Small (1-555b) benchmark 1425 1135 79.65%
ReallocMem Medium (1-4039b) benchmark 3834 3309 86.31%
Block downsize 12079 10305 85.31%
Address space creep benchmark 13283 12571 94.64%
Address space creep (larger blocks) 16066 13879 86.39%
Single-threaded reallocate and use 4395 3960 90.10%
Single-threaded tiny reallocate and use 8766 7097 80.96%
Single-threaded allocate, use and free 13912 13248 95.23%

You can find the program, used to generate the benchmark data,
at https://github.com/maximmasiutin/FastCodeBenchmark

FastMM4-AVX is released under a dual license, and you may choose to use it
under either the Mozilla Public License 2.0 (MPL 2.1, available from
https://www.mozilla.org/en-US/MPL/2.0/) or the GNU Lesser General Public
License Version 3, dated 29 June 2007 (LGPL 3, available from
https://www.gnu.org/licenses/lgpl.html).

FastMM4-AVX is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

FastMM4-AVX is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License
along with FastMM4-AVX (see license_lgpl.txt and license_gpl.txt)
If not, see .

FastMM4-AVX Version History:

- 1.0.7 (22 March 2023) - implemented the optional use of user mode wait
(WaitPKG) umonitor/umwait instructions to wait for a synchronization
variable; it is disabled by default; define the "EnableWaitPKG" conditional
define to enable this feature; however it may not be as efficient
as the pause-based loop, so only use this feature it if your tests
show clear benefit in your scenarios.

- 1.0.6 (25 August 2021) - it can now be compiled with any alignment (8, 16, 32)
regardless of the target (x86, x64) and whether inline assembly is used
or not; the "PurePascal" conditional define to disable inline assembly at
all, however, in this case, efficient locking would not work since it
uses inline assembly; FreePascal now uses the original FreePascal compiler
mode, rather than the Delphi compatibility mode as before; resolved many
FreePascal compiler warnings; supported branch target alignment
in FreePascal inline assembly; small block types now always have
block sizes of 1024 and 2048 bytes, while in previous versions
instead of 1024-byte blocks there were 1056-byte blocks,
and instead of 2048-byte blocks were 2176-byte blocks;
fixed Delphi compiler hints for 64-bit Release mode; Win32 and Win64
versions compiled under Delphi and FreePascal passed the all the FastCode
validation suites.

- 1.05 (20 May 2021) - improved speed of releasing memory blocks on higher thread
contention. It is also possible to compile FastMM4-AVX without a single
inline assembly code. Renamed some conditional defines to be self-explaining.
Rewritten some comments to be meaningful. Made it compile under FreePascal
for Linux 64-bit and 32-bit. Also made it compile under FreePascal for
Windows 32-bit and 64-bit. Memory move functions for 152, 184 and 216 bytes
were incorrect Linux. Move216AVX1 and Move216AVX2 Linux implementation had
invalid opcodes. Added support for the GetFPCHeapStatus(). Optimizations on
single-threaded performance. If you define DisablePauseAndSwitchToThread,
it will use EnterCriticalSection/LeaveCriticalSectin. An attempt to free a
memory block twice was not caught under 32-bit Delphi. Added SSE fixed block
copy routines for 32-bit targets. Added support for the "Fast Short REP MOVSB"
CPU feature. Removed redundant SSE code from 64-bit targets.
- 1.04 (O6 October 2020) - improved use of AVX-512 instructions to avoid turbo
clock reduction and SSE/AVX transition penalty; made explicit order of
parameters for GetCPUID to avoid calling convention ambiguity that could
lead to incorrect use of registers and finally crashes, i.e., under Linux;
improved explanations and comments, i.e., about the use of the
synchronization techniques.
- 1.03 (04 May 2018) - minor fixes for the debug mode, FPC compatibility
and code readability cosmetic fixes.
- 1.02 (07 November 2017) - added and tested support for the AVX-512
instruction set.
- 1.01 (10 October 2017) - made the source code compile under Delphi5,
thanks to Valts Silaputnins.
- 1.00 (27 July 2017) - initial revision.

The original FastMM4 description follows:

# FastMM4
Fast Memory Manager

Description:
A fast replacement memory manager for Embarcadero Delphi applications
that scales well under multi-threaded usage, is not prone to memory
fragmentation, and supports shared memory without the use of external .DLL
files.

Homepage:
https://github.com/pleriche/FastMM4

Advantages:
- Fast
- Low overhead. FastMM is designed for an average of 5% and maximum of 10%
overhead per block.
- Supports up to 3GB of user mode address space under Windows 32-bit and 4GB
under Windows 64-bit. Add the "$SetPEFlags $20" option (in curly braces)
to your .dpr to enable this.
- Highly aligned memory blocks. Can be configured for either 8-byte or 16-byte
alignment.
- Good scaling under multi-threaded applications
- Intelligent reallocations. Avoids slow memory move operations through
not performing unneccesary downsizes and by having a minimum percentage
block size growth factor when an in-place block upsize is not possible.
- Resistant to address space fragmentation
- No external DLL required when sharing memory between the application and
external libraries (provided both use this memory manager)
- Optionally reports memory leaks on program shutdown. (This check can be set
to be performed only if Delphi is currently running on the machine, so end
users won't be bothered by the error message.)
- Supports Delphi 4 (or later), C++ Builder 4 (or later), Kylix 3.

Usage:
Delphi:
Place this unit as the very first unit under the "uses" section in your
project's .dpr file. When sharing memory between an application and a DLL
(e.g. when passing a long string or dynamic array to a DLL function), both the
main application and the DLL must be compiled using this memory manager (with
the required conditional defines set). There are some conditional defines
(inside FastMM4Options.inc) that may be used to tweak the memory manager. To
enable support for a user mode address space greater than 2GB you will have to
use the EditBin* tool to set the LARGE_ADDRESS_AWARE flag in the EXE header.
This informs Windows x64 or Windows 32-bit (with the /3GB option set) that the
application supports an address space larger than 2GB (up to 4GB). In Delphi 6
and later you can also specify this flag through the compiler directive
{$SetPEFlags $20}
*The EditBin tool ships with the MS Visual C compiler.
C++ Builder:
Refer to the instructions inside FastMM4BCB.cpp.

# FastMM4
Fast Memory Manager
![FastMM-Title.jpg with title only](images/FastMM-Title.jpg "FastMM-Title.jpg with title only")

## Description:
A fast replacement memory manager for Embarcadero Delphi applications
that scales well under multi-threaded usage, is not prone to memory
fragmentation, and supports shared memory without the use of external .DLL
files.

## Homepage:
https://github.com/pleriche/FastMM4

## Advantages:
* Fast
* Low overhead. FastMM is designed for an average of 5% and maximum of 10%
overhead per block.
* Supports up to 3GB of user mode address space under Windows 32-bit and 4GB
under Windows 64-bit. Add the "$SetPEFlags $20" option (in curly braces)
to your .dpr to enable this.
* Highly aligned memory blocks. Can be configured for either 8-byte or 16-byte
alignment.
* Good scaling under multi-threaded applications
* Intelligent reallocations. Avoids slow memory move operations through
not performing unneccesary downsizes and by having a minimum percentage
block size growth factor when an in-place block upsize is not possible.
* Resistant to address space fragmentation
* No external DLL required when sharing memory between the application and
external libraries (provided both use this memory manager)
* Optionally reports memory leaks on program shutdown. (This check can be set
to be performed only if Delphi is currently running on the machine, so end
users won't be bothered by the error message.)
* Supports Delphi 4 (or later), C++ Builder 4 (or later), Kylix 3.

## Usage:
### Delphi:
Place this unit as the very first unit under the "uses" section in your
project's .dpr file. When sharing memory between an application and a DLL
(e.g. when passing a long string or dynamic array to a DLL function), both the
main application and the DLL must be compiled using this memory manager (with
the required conditional defines set).

There are some conditional defines
(inside `FastMM4Options.inc`) that may be used to tweak the memory manager. To
enable support for a user mode address space greater than 2GB you will have to
use the EditBin* tool to set the `LARGE_ADDRESS_AWARE` flag in the EXE header.
This informs Windows x64 or Windows 32-bit (with the /3GB option set) that the
application supports an address space larger than 2GB (up to 4GB). In Delphi 6
and later you can also specify this flag through the compiler directive
`{$SetPEFlags $20}`

*The EditBin tool ships with the MS Visual C compiler.
### C++ Builder:
Refer to the instructions inside `FastMM4BCB.cpp`.