Windows Vista restricts non-Win32 apps to 32 MB of memory

Thomas R. Nicely

http://www.trnicely.net
Current e-mail address

Freeware copyright © 2017 Thomas R. Nicely. Released into the public domain by the author, who disclaims any legal liability arising from its use.

Last updated 0400 GMT 28 November 2017. Original posting 0600 GMT 08 March 2007.

THE NATURE OF THE PROBLEM

Executable images created for the DOS/Wintel environment, using the GNU GCC compilers and language standards (but not linking to the Win32 API), are subject to failure (or performance degradation) when executed in Microsoft Windows Vista, because Vista arbitrarily restricts the memory space for the GCC executable to 32 MB (33,554,432 bytes). Attempts to allocate more memory than this using the malloc(...) function (or related functions, such as calloc(...)) will fail. This limitation applies whether the application is executed with the Run command, within a Command Prompt box (DOS box), or with the Start command. This limitation does not appear in Windows XP, Windows 98SE, or standalone DOS; the exact same executable, running under Windows XP SP2 or Win98SE, is capable of allocating several hundred megabytes of physical memory (if present on the machine). The limitation appears to apply to any compiler, linker, or executable not conforming to Microsoft's proprietary Win32 API.

The operating environment exhibiting this incompatibility in Vista includes GNU GCC C 4.12, DJ Delorie's DJGPP 2.04 beta (11/30/2003), and 32-bit Vista Home Basic 6.0.6000 running on a Dell E520 with a Pentium D 915 2.8GHz dual core processor, 1 GB of 533 MHz DDR memory, 800 MHz front side bus, and SATA drives. The limitation is also observed in Vista SP1 (service pack 1), as released in March, 2008. Note that none of these phenomena has been verified in 64-bit Vista (the author does not have access to any system running 64-bit Vista).

The limitation appears with or without the use of executable compression, and with or without the use of the DJGPP DPMI servers CWSDPMI (Charles W. Sandmann), CWSDPR0, CWSDSTUB, and PMODE/DJ (which are apparently ignored in any case, in favor of the native Windows DPMI server). The system is operated as a standalone, single-user machine, logged on as Administrator (Command Prompt run as Administrator), with no Internet connection. Nearly all security features are disabled, including User Account Controls, Windows Firewall, Windows Defender, and (except for a warning message) Security Center.

Essentially the same limitation applies to source compiled with the g77 Fortran compiler; one difference is that the code will usually fail immediately with the error message "Load error: no DPMI memory" if total memory requests exceed 32MB. I do not know if the problem appears in other environments (e.g., other editions of Vista or codes cross-compiled for DOS/Wintel with other versions of GCC). Proprietary compilers which inherently convert code to the Win32 API (such as Visual C++ or even Borland C++ 4.52 with PowerPak) do not exhibit this limitation; they convert calls to the standard "C" malloc(...) to calls to Win32 API functions such as GlobalAllocate(...) or VirtualAllocate(...), which are then linked in at run time from Microsoft system DLLs. I suspect the real problem may be that Vista is treating any application that does not call the proprietary Win32 API (either directly, or through compiler or linker translation) as a "16-bit" application, and is then applying the 32MB limitation for "security" reasons. The presence of what appears to a 16-bit stub loader at the beginning of the GCC executables may also be an issue for Vista, although it is no problem for XP or Win98SE.

Since the problem does not appear when the same executable image is run under Windows XP or Win98SE, the problem clearly lies with Vista. There appears to a fundamental difference between the NTVDM modules of Vista and those of XP and 98SE, particularly in the manner in which they handle DPMI calls.

A simple test code is provided in the file vista1.zip (39K), which includes the source code vista1.c and a DOS/Wintel executable vista1.exe compiled with GCC C 4.12 and DJGPP 4.12 under DJGPP 2.04 (no compression or DPMI stub). The test code attempts to allocate a large chunk of memory (40,000,000 bytes), using malloc(...); if it fails, it repeats the attempt, decreasing the amount by 1,000,000 bytes each time. The code also writes to the memory, then reads it back to verify the integrity (one compiler generated code in which malloc reported success, but the memory could not in fact be accessed). Results are reported to the console. Run it from a Vista Command Prompt (DOS box), with the Run command, or with the "Start" command; the results will be the same---failure until the request is decreased to 33,000,000 bytes (or possibly less). However, if the same code is run in a Command Prompt session in Windows XP or Win98SE, or in a standalone DOS session under Win98SE, the allocation will be limited only by the available physical memory. The code can also be run with a command-line parameter, e.g., "vista1 100", which will begin the allocation attempts at 100 million bytes.

For many applications, the problem may not be significant. However, much of my work requires access to very large amounts of memory (arrays of several hundred megabytes in some cases), and I feel sure this is true for other developers as well. Even if all your work is normally done on non-Windows platforms, it would still be unfortunate to lose as a distribution base all of the Windows-based systems on the planet---currently about 90 % of all systems, and more than likely nearly all of that base will eventually be converted to Vista (or its successor).

SOLUTIONS

Known solutions fall into three classes: those which work only with Vista SP1, those which will also work with earlier releases (including the original release, 6.0.6000), and those which bypass the problem entirely by conforming the application to the Win32 API.

Registry Solution for Vista SP1

Rugxulo reports (3 June 2008) that the following workaround appears to eliminate the problem in Vista with Service Pack 1 (SP1). Use a registry editor (such as REGEDIT) to create the new (REG_DWORD) registry value DpmiLimit under the key 

Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\WOW

Now edit the value to reflect the maximum amount of memory (in bytes) that you wish to make available to console applications (DOS command boxes). Thus, a value of 128000000 (decimal) would make 128 million bytes available to DOS boxes, as opposed to the default value of 32MB; a hex value of 0x7A12000 would have the same effect. It appears that 32MB is the minimum allowed; the maximum has not been determined, but is almost certainly 2GB or less. You will probably need administrator privileges to implement these steps, and you may have to reboot before the new setting takes effect.

This trick does not work in Vista releases prior to SP1 (such as my own version, 6.0.6000 from February, 2007); adding the new value to the registry has no effect, either before or after rebooting.

TSR/DLL Solution for all Vistas

The Japheth TSR/DLL solution for this difficulty is, unfortunately, no longer available.

Converting Applications to the Win32 API

The ultimate solution is to port the application source code to a compiler and development environment which links to the Win32 API. The recompiled application will then run as a native Windows Vista application. Of course, this is only a solution if you are the developer, or else have access to the source code and the necessary expertise. Rewriting the source code to conform to the Win32 API may be a major undertaking.

There are a number of (relatively expensive) commercial compilers and development environments which conform to the Win32 API, Microsoft's Visual Studio C++ with its SDK being the best known. However, there are also several C compilers and development environments which conform to the Win32 API, and are available for download without charge.

After months of experimentation, I have settled on MinGW/MSYS as the best development environment for my purposes. Its advantages include the following: Disadvantages of the MinGW/MSYS environment include the following:

ADDITIONAL NOTES