I get an error when compiling large installers

NSIS can theoretically compile installers as large as 2GB. This is first limited by the amount of free disk space on the drive containing the system temporary directory. To get the best results, this drive must be formatted as NTFS and must have enough free space to hold the temporary file, which can get up to 2GB depending on the packed files. This means you can't compile very large installers on Windows 9x because it doesn't support NTFS. The limit is about 700MB on a FAT partition.

More RAM and virtual memory will only help speed up the process but won't increase the limit.

When distributing very large data, it's usually better not to store everything in the executable file, but store the data separately and use CopyFiles or extraction plug-ins to install.