www.digitalmars.com         C & C++   DMDScript  

c++.dos.32-bits - Accessing > 64MB of RAM; using DOS 32-bit extender

reply Denzien <notforhumanconsumption gmail.com> writes:
Hi everyone, I'm working on a simple application to research the
true volatility of RAM in my Digital Forensics class and I've run
into a small roadblock.

I've read that I can (possibly) address up to 3/4 of the available
system memory using the x32 DOS extender, but my application can
only malloc up to about 64MB.

The idea of the program is simple: after booting into a very small
footprint OS (DOS), allocate all available memory and fill that
memory with identifiable "needles".  After rebooting the machine,
again run the application to re-allocate the memory and then
search for the needles in memory (in lieu of performing a memory
dump and using a hard drive).  This is designed to be run from a
CD for fast results on a variety of machines (Laptop, Desktop,
Workstation, different brand names, etc)

The file is named Needler.cpp, and I from its directory I compile
using the following command:
"c:\dm\bin\dmc Needler.cpp -Ae -ml -mx x32.lib"

The following is my simple and un-optimized code that seems to
work great in the first 64MB:

/********************************/
#include <stdio.h>
#include <stdlib.h>
#include <iostream.h>
#include <malloc.h>
#include <x32.h>
#include <dos.h>

#define KB  1024
#define MB (1024 * KB)

void processArguments(int argc, char* argv[]);
void writeMenu();
void hideNeedles(char * buffer);
int FindNeedles();
int ByteToInt(char *data);
void PrintMenu();
char* doMalloc();
int SearchMemory(int offset);

void *_x386_zero_base_ptr;

int plantedNeedles = 0;
int needleLength = 53;
char needle[53]
= "ABCDEFGHIJKLMNOPQRSTUVWXYZzyxwvutsrqponmlkjihgfedcba";

bool verbose = false;
bool read = false;
bool write = false;

int main(int argc, char* argv[])
{
	int result;// = SearchMemory();
	processArguments(argc, argv);

	if (read || write)
	{
		printf("\nPerforming Operations...\n\n");
		result = FindNeedles();
	}
	else
	{
		writeMenu();
		return 0;
	}
	if (read) printf("\n\nFound %i needles in the haystack.",
result);
	return 0;
}

void processArguments(int argc, char* argv[])
{
	char verboseSwitch[3] = "-v";
	char readSwitch[3] = "-r";
	char writeSwitch[3] = "-w";
	for(int i = 0; i < argc; i++)
	{
		if(argv[i][1] == verboseSwitch[1]) verbose = true;
		else if(argv[i][1] == readSwitch[1]) read = true;
		else if(argv[i][1] == writeSwitch[1]) write = true;
		else if(argv[i][1] == 'o')
		{
			int x = ByteToInt(argv[++i]);
			printf("\nOffset = %i\n\n", x);
			//x = SearchMemory(x);
		}
	}
}

int ByteToInt(char *data)
{
	int result = 0;
	int i = 0;
	while(data[i] != 0)
	{
	/*for (int i = 0; i < 20; i++)
	{*/
		if(((int)data[i] >= 0x30) && ((int)data[i] <=
0x39))
		{
			result *= 10;

			switch(data[i])
			{
				case '1':
					result += 1;
					break;
				case '2':
					result += 2;
					break;
				case '3':
					result += 3;
					break;
				case '4':
					result += 4;
					break;
				case '5':
					result += 5;
					break;
				case '6':
					result += 6;
					break;
				case '7':
					result += 7;
					break;
				case '8':
					result += 8;
					break;
				case '9':
					result += 9;
					break;
				default:
					break;
			}
		}
		/*else if(data[i] == 0x0)
			return result;*/
		i++;
	}

	return result;
}

void writeMenu()
{
	printf("\n\nUsage:\n");
	printf("  -w  Write needles into memory\n");
	printf("  -r  Read needles from memory\n");
	printf("  -v  Verbose output (makes processing very slow)
\n   *Must also choose read or write.\n\n");
	printf(" If both read and write are specified,\n  Needler
will write needles first, then read.\n\n");
}

char* doMalloc()
{
	unsigned long Kilobyte = 1024;
	char * buffer = (char*) malloc (Kilobyte * 64);
	//if (buffer == NULL) buffer = (char*) malloc (KB);
	if ((write) && (buffer != NULL)) hideNeedles(buffer);
	return buffer;
}

void hideNeedles(char * buffer)
{
	char *ptr = &buffer[0];
	int index = 0;
	int iterations = KB / (needleLength + 1);

	for (int i = 0; i < iterations; i++)
	{
		memcpy(ptr, needle, needleLength);
		ptr += (needleLength + 1);
		plantedNeedles++;
	}
}

int FindNeedles()
{
	char *buffer = doMalloc();
	char *ptr = &buffer[0];
	char *MAX = &buffer[sizeof(buffer)];

	printf("Allocating Memory...");
	if (verbose) printf("\n");
	while (buffer != NULL)
	{
		if (&buffer[0] < ptr) ptr = &buffer[0];
		if (&buffer[sizeof(buffer)] > MAX) MAX = &buffer
[sizeof(buffer)];
		//MAX = &buffer[sizeof(buffer)];
		if (verbose) printf("\rMALLOC: &%p ", &buffer[0]);
		buffer = doMalloc();
	}
	if (verbose) printf("\n");
	printf("Done.  ");
	if (write) printf("%i needles planted.\n", plantedNeedles);
	if (!read) return 0;
	printf("\nFinding needles...");
	if (verbose) printf("\nNeedles found:\n0");
	int found = 0;

	while(ptr < MAX)
	{
		try
		{
			if(*ptr == needle[0])
			{
				if (memcmp(ptr, needle,
needleLength) == 0)
				{
					found++;
					if (verbose) printf("\r%
i", found);
					ptr += needleLength;
				}
				else
				{
					ptr++;
				}
			}
		}
		catch(...)
		{
			printf("\nError detected -- aborting.\n");
			break;
		}

		ptr++;
	}

	return found;
}

int SearchMemory(int offset)
{
	int count = offset;
	char *ptr = (char*)_x386_zero_base_ptr + count;
	for(int i = 0; i < 100000000; i++)
	{
		ptr = (char*)_x386_zero_base_ptr + count;
		printf("\r%i - (%c)", count, *ptr);
		count++;
	}
	return count;
}
/********************************/


Also, I can't seem to find any information that describes how to
use a DOS 32-bit extender to my liking, so I'm still unsure if I'm
going about this in the correct way.  Any input would be great!
Mar 30 2007
next sibling parent "Kevin G. Rhoads" <kgrhoads alum.mit.edu> writes:
You may need to allocate in several chunks to reach the limit due to limitations
in the extender or underlying services (e.g., DPMI, VCPI) that it may be using.

Also, the allocation limit may have nothing to do with the actual true installed
RAM available, even when the DOS extender does not provide virtual memory,
other OS services might.  So you needs be careful about just what gets loaded
or starts running, even on a CD-ROM boot.  

So, is X32 running on bare metal?  Or does the CD-ROM boot disk load other
things
that may have a finger in the pie?  And just what is the DOS?  MS-DOS, DR-DOS,
OpenDOS, FreeDOS or something else?  And what version?

I don't have experience with large core footprint issues in DOS extenders, so
I have no specific help to give. Sorry.  But, IIRC, there were issues similar
to those seen in 16 bit segmented addressing.
Apr 02 2007
prev sibling parent Heinz Saathoff <newshsaat arcor.de> writes:
Hello,

Denzien wrote...
 
 I've read that I can (possibly) address up to 3/4 of the available
 system memory using the x32 DOS extender, but my application can
 only malloc up to about 64MB.
The DOSX extender is rather old and seems to have a 64MB limit. In my DOS app I used the function _x32_allcoreleft() from X32.h to show me the available memory. I get about 62MB free memory when my app allocates about 2MB. It doesn't matter if the PC has 128MB, 256MB or even more! It doesn't make a difference if HIMEM is or isn't used. - Heinz
Apr 04 2007