Contents
1. About the library
1.1 So who are Daybo Logic?
1.2 Contacting the author
1.3 FAQ
2.0 What are the flavors of the library
2.1 How much does it cost?
2.2 What platforms is the library available
for?
3.0 Concepts and structures used by the library
3.1 Getting started
3.2 Portability issues
3.3 8086 builds
3.4 Precompiled headers
4.0 History
4.1 Plans for next version
5.0 Function listings
5.1 Descriptions of all functions and their
use
Q - Error DPCRTLMM.H 53: Declaration missing ;
A - This stumped me for a while too, this can only happen if you have
the source and selected to have explict far data for the 16-bit tiny, small
or medium memory models. The answer is one of these solutions I hope,
firstly check the language is set to non-ANSI compliance, if your compiler
only supports ANSI you cannot support explicit far pointers so give up!
The other problem is on line 112 (ish) of BUILD.H #define DPCRTLMM_FARDATA
_ _ far, change the _ _ far bit to just far and try again. Look for
what the 'far' keyword equivilant is in your language. If you still
have problems contact me.
Q - No prototype for function 'farmalloc' or 'farcalloc' or 'farfree'
or 'farrealloc'
A - The explicit fars are non-ANSI, unfortunately the functions differ
from one compiler to the next when one strays from ANSI. In BUILD.H
there are a list of functions here, change them to point to your compiler's
equivilant functions if you can discover what they are. If you don't
have any you will have to give up and turn off this option, make sure you
don't have strict ANSI C compliance on or anything first but if all else
fails just compile for the compact/large or huge model.
Q - I want to compile for 16-bit x86 processors in small memory models
A - The library only functions correctly with far data, edit BUILD.H
and define the macro DPCRTLMM_WANTFARDATA
Q - My program is crashing unexpectedly when I try to free blocks of
memory (or earlier while using the memory manager)
A - The most common problem is forgetting to start the library, if
this is not the case and you are compiling for a 16-bit processor it may
be that the data is near. The easiest way to avoid this is to rebuild
in a memory model which has far data. There is another way in this
version of the library, open BUILD.H and find the commented out macro called
DPCRTLMM_WANTFARDATA and remove the comments. If this does not solve
the problem please send me all details possible.
Standard version - The standard version is the version which most people will probally have, it is freeware and has limited functionallity. It comes with builds for DOS and Win32 and the header to use the library. The version offers all the functionallity excluding what is offered in the others.
Professional version - Adds advanced debug hook support for tracing calls for allocations / deallocations etc. so that the programmer's own code can view this information before allowing it to be passed on to another hook, or stopping it.
Absolutely mental version - Everything mentioned above plus laerger
size hook chains and double sized maximum safety list. All source
and debug builds supplied so these items could be modified even further,
far past madness, it's mental.
These block descriptors contain this simple information: The block's base address, size in bytes and a number of flags with special meanings.
The other structure you will encounter is the block descriptor array. I recommend that every C file maintains it's own array. When a block is allocated it's decriptor is always inserted into one of these arrays and nearly every library call will want to know the address of one of these arrays. This only causes a problem if the program passes a descriptor to another module the program which tries to get size information on the block using the library functions while passing it's own array, then the program will be terminated because the library will assume that the address is invalid. I recommend using NULL as a placeholder for arrays, for blocks which are designed to be shared. Normal sharing is fine, as long as the library is not used on that block from another module.
The final structure is stored internally and the program should never need to access, this structure is called the safety list. It is a large fixed size array of all the addresses of the block descriptor arrays used throughout the program. The interface to this list is kept internal to the library, if you want to access it you could always add it to DPCRTLMM.H if you have the source code although there should be no reason to do this.
The only time you will encounter the safety list is if you run out of space. By default the standard library comes with 64 entries which is about normal for most programs. The professional version comes with 512 entries which is massive. Purchasing the source code will man you may change the size to anything and / or expose the hidden interface to the safety list.
So what is this safety list for if I can't access it under normal circumstances?
Well it's used to trap any call to the library which specifies an invalid
pointer to a block descriptor array. It is also processed when the
library is shut down to ensure all blocks have been released, it is therefore
is the primary structure for producing a leak report.
If you are writing a new program use the library from the start, add a block descriptor array structure to the main module at file level, then in main call the library startup routine, create the array then allocate blocks using it. When main() is about to exit free all blocks, destroy the array with the dedicated function and shutdown the library. These destroy and shutdown routines are the main ones which check if anything was not freed from the array etc.
I have supplied an example with the library to help you see this in
action, it is called example.c, actually as you will see, it intentionally
fails to free hald of the blocks.
By default the library source is portable, it can be setup to use some
non-ANSI features:
far pointers and precompiled headers. Most instructions for changes
are in build.h, however allowing PCH pragmas has to be done with a predefinition.
See section 3.4
3rd May 2000 - Overlord:
Totally got rid of SAFELIST.C/.H and decided to start again.
The
declaration of the safety list object is in INTDATA.C and the size
is
determined in the configuration file BUILD.H
The new SAFELIST.C will contain simple functions to save having to
write
the list processing several times, code to count the number of items,
code to add a new one at the first blank entry and remove items.
There is no need for a linked list or shift up code because it's a
fixed
size determined at compile time therefore anything which is not a valid
entry will just be NULL.
4th May 2000 - Overlord:
Compiled in the new SAFELIST.C and tried to build the library muliple
times
until all references to other Daybo Logic stuff were stripped out.
BOOL had
to change to unsigned int, however the user of the library may still
treat
these values as BOOL/TRUE/FALSE even though they are used as unsigned
ints
inside the library. The user can only do this if they have their
own
compatible versions of BOOL/TRUE/FALSE which most developers have by
now, or
can easily make them if they do now have them.
Also changed defintion of _safeList and all people who used it had
to be
altered to be aware of the new definition.
Wrote a new page on the web site offering it for sale.
5h May 2000 - Overlord:
Wrote documentation for library and released it on the Daybo Logic
web site
http://www.daybologic.co.uk
10th May 2000 - Overlord:
There was a bug in the new safety list which stopped the whole library
working, a single missing '!' (logical not). Republishing the
library.
14th June 2000 - Overlord:
In 16-bit memory models on x86 processors, if the data segment is near
the pointers do not mean anything to the hosting program when given to
it, the near pointers are for us with DPCRTLMM's own assumed data segment,
therefore explicit far data is needed, added a macro to build.h for doing
so and severely modified this documentation.
18th July 2000 - Overlord:
OK this isn't a technical note, it's just that today I set myself the
deadline of August 1st for the version 1.0 release.
19th July 2000 - Overlord:
Coverted all the filenames to lower case and all the #includes to lower
case for compatibility with UNIX systems.
Tried and failed miserably to create a working makefile and the Borland
C++ IDE auto-generated one came up with warnings about some of my symbols
not being in the library, works ok from the IDE and I don't ever get warnings
for it so I gave up on this for now. The library is saimple enough
to build I feel.
Merged BBATRAP and BBPTRAP, well, put them in the
same module with another function call _VerifyPtrs() which calls both,
this stops having to do both calls in some functions which looks untidy.
Both functions were always called at once anyway, the new module is called
vptrap.c
In realloc.c the string "Realloc()" was used twice,
optimized out second instance.
Noticed quite a few modules called the old BBA trap
and did not have a pointer to verify also, so I changed _VerifyPtrs() slightly
so argument three could be NULL and the second trap call would be skipped.
Well mustn't grumble, it still feels cleaner this way and I got rid of
a whole module in effect!
Free() used the string "Free()" twice, made it share a variable, only
has an effect on non-optimizing compilers which don't support duplicate
strings merged.
Removed all #pragma hdrstop PCH support unless DPCRTLMM_HDRSTOP
is defined, to save annoying warnings to people who's compiler does not
understand the pragma
Added support for NULL as the default built-in block array,
define DPCRTLMM_NONULL_BLOCKDESCARRAY in build.h to restore old behaviour,
old behaviour was to say that NULL was not a valid block desc array and
crash the program.
Added a hack table for shorter function names, this
was suggested by Jeremy
and will also shorten the structure names. Developers already using
the library can totally ignore this.
20th July 2000 - Overlord:
In adding the support for default block arrays I had to split up the
trap routine called TrapUnFreedArrays() in strtstop.c into a new routine
also TrapUnFreedBlocks(). I realized that not all data which is unfreed
(garbage) is just arrays now due to the default array which is never destroyed
but can contain blocks. I renamed the define DPCRTLMM_TRAP_UNFREED_ARRAYS
to DPCRTLMM_TRAP_UNFREED_DATA. If you used this (which I doubt) at
this stage just rename the macro in your code.
21st July 2000 - Overlord:
Corrected an assertion failure due to the new NULL block array.
In the betas, when DPCRTLMM_LOG was not defined the messages were not output
to a log file but the messages were still taking space in the resulting
program, got rid of most of them, some are used as traps too so I couldn't
get rid of them. To try to stop users making the mistake of including
build.h in their own programs I made build.h look for a macro I define
in my source only.
Loads of warnings about variables that were never
used when I tried to define DPCRTLMM_LOG for an experiment, filtered them
all out, all the #ifdef DPCRTLMM_LOG #endif /*DPCRTLMM_LOG*/ stuff everywhere
looks a right mess though.
24th July 2000 - Overlord:
Limited block descriptor arrays to 64 entries in standard builds
26th July 2000 - Overlord:
Added a new module getstats.c, countains these functions: dpcrtlmm_GetBlockCount(
) & dpcrtlmm_GetStats( ), GetStats( ) is
very simple and it uses _blockCount which I added to intdata.c. Alloc()/Free()
don't update the counter yet
Limited library to 50 blocks for standard builds, first set up a define
in build.h so I could modify it easily and then put all the statistics
(except flag counting) into internal vars (intdata.c). Flag counting
is done by an internal routine called CountFlagsInUse( ), I put that in
getstats.c
Added dpcrtlmm_AreTrapsEnabled(
)
There is now an extern in town called dpcrtlmm__EnableTraps,
it used to be an internal variable called dpcrtlmm_int__enableTraps, now
the modifiers dpcrtlmm_EnableTraps( ), dpcrtlmm_DisableTraps(
) & dpcrtlmm_AreTrapsEnabled( )
are just macros to use it.
Making this new extern meant the simple module enbltrap.c
was redundant, deleted it.
Limited standard version of the library to 50 blocks,
not sure it's turned off in payed for versions, better test that or I could
be in trouble!
27th July 2000 - Overlord:
Been doing other stuff mainly, we ran out of coffee, arghhh!
Fixed a link problem and Mike agreed to test the library.
Got Alloc()/Free() to update statistics, got code to work to nag user
and terminate the program if more than DPCRTLMM_STDBLOCKLIMIT blocks are
used in the program, oh and no worries it doesn't nag registered users.
28th July 2000 - Overlord:
Renamed getstats.c stats.c, added support for the default array into
CountFlagsInUse( ). It seems Alloc( ) is correctly allocating blocks
in the default array because of the NULL array resolver macro, must be
Free( ) that is not so aware. I have noticed that GrowBlockArray(
) is relying on the run-time library realloc( ) to accept NULL and pass
it on to alloc( ), I'm not sure that this is always supported, I
may have to change it. Had to make some changes to isbad.c both IsBad...(
) calls did not resolve array pointers and dpclrtmm_IsBadArrayPtr(
) was so somple it was just asking the safety list. Some functions
may try dereferencing NULL pointers without resolving them first so I'll
have to find all this instances of this behaviour, all functions which
are exposed in the library so the user can call them will probally be affected.
Whoops! vptrap.c is a bit odd, looking for
bad blocks is simple, it called IsBadBlockPtr( ), seems logical, and yet
the trap on a bad array pointer (the trap above) decides to call the safety
list itself, hmm, it might be more advisable to call IsBadArrayPtr( ) of
course, changing code again....
29th July 2000 - Overlord:
The trap unfreed arrays routine was always seeing one less block then
it should, traced the problem down to a loop which looked at PArr->Count
but that count was being changed by dpcrtlmm_Free( ) which the routine
was calling as automatic garbage collection on every unreleased block it
found. I changed it to while ( PArr->Count ) and took off element
0
30th July 2000 - Overlord:
The change to the loop has (or perhaps not but I suspect it's this)
has caused the machine to halt running as a 16-bit program under DOS.
I have been trying to debug it all day but it's very hard to see what goes
wrong, might have to re-write this loop in the next release, it's too close
to release day now, I'll have to say that the library is currently limited
to flat builds ie. 32-bit.
Wrote dpcrtlmm_Ver( ), which uses DPCRTLMM_VER as
in build.h
31st July 2000 - Overlord:
Hmm, this morning the program no longer halts, if anybody out there
decides to read this revision history I'd really like to know what was
going on! Not a lot of programming done today, too hot. Only
7 hours until it's midnight in the USA so I should have the demos built
by then.
1st August 2000 - Overlord:
Altered the way the version number is reported, it now puts data into
a small structure, S_DPCRTLMM_VERSION, removed the define from build.h
and made the structure in dpcrtlmm.h I don't think the debug hooks
are properly set up I noticed dpcrtlmm_CreateBlockArray( ) at no time lead
to the hook chain being processed. Must fix this in all functions
I suspect.
Added dpcrtlmm_IsDefaultBlockArray(
)
Whoops saw dpcrtlmm_Realloc( ) forgot to resolve
NULL arrays, fixed.
Invalid page faults were being caused at the end, changed raise(SIGABRT)
to abort( ), then found this wasn't the real cause, the real cause was
the library was designed for the static run time library and then example
was built for the DLL version. I'll leave that as abort( ) anyway.
Swapping to disk should be implemented with an LRU algortihm, based on the flag in block descriptor (unswappable, bit 1) and the block descriptor may have to be extended with more fields to support this.
The library is a bit slow due to an excessive amount of calls to _VerifyPtrs(
), which an internal routine called to check the user's pointers are valid,
however it's also called when functions call other functions, even though
they already know the pointers are safe, to avoid this I may make an inner
shell, where the user calls safety wrappers around the real library and
all functions call the inner shell functions and not the entrypoints the
users sees.
dpcrtlmm_InstallDebugHook( )
dpcrtlmm_GetDebugHookChainCount(
)
dpcrtlmm_GetDebugHookMatrixCount(
)
dpcrtlmm_UninstallDebugHook( )
dpcrtlmm_Alloc( )
dpcrtlmm_Free( )
dpcrtlmm_CreateBlockArray( )
dpcrtlmm_DestroyBlockArray( )
dpcrtlmm_IsDefaultBlockArray( )
dpcrtlmm_Startup( )
dpcrtlmm_Shutdown( )
dpcrtlmm_GetBlockSize( )
dpcrtlmm_IsBadBlockPtr( )
dpcrtlmm_IsBadArrayPtr( )
dpcrtlmm_Realloc( )
dpcrtlmm_Calloc( )
dpcrtlmm_InstallTrapCallback( )
dpcrtlmm_RemoveTrapCallback( )
dpcrtlmm_GetTrapCallbackInfo( )
dpcrtlmm_ModifyDescriptorFlags( )
dpcrtlmm_SetBlockLockingFlag( )
dpcrtlmm_IsBlockLocked( )
dpcrtlmm_LockBlock( ) & dpcrtlmm_UnlockBlock(
)
dpcrtlmm_ToggleBlockLockingStatus(
)
dpcrtlmm_EnableTraps( )
dpcrtlmm_DisableTraps( )
dpcrtlmm_AreTrapsEnabled( )
dpcrtlmm_GetStats( )
dpcrtlmm_GetBlockCount( )
dpcrtlmm_Ver( )
External variables
dpcrtlmm__EnableTraps
Pass pointer to new flags for the block, old flags are returned, to
get the flags without modifying them, pass NULL as the
pointer PNewFlags. It is feasible that the function could return
after a trap if traps are off or a user handler is called for the trap,
which returned. It is not possible to tell, if anything is suspected
on running the program, at redesign your code so that it does not use a
user trap (just comment out the line installing the handler, or don't turn
off trapping).
void dpcrtlmm_SetBlockLockingFlag(PS_DPCRTLMM_BLOCKDESCARRAY
PBlockArray, const void* Ptr, const unsigned int NewStatus);
Locks or unlocks a block of memory. When a block of memory is
locked, it means it cannot be freed or resized, if a trap is fired as a
result of the call, the function ModifyDescriptorFlags(
) is mentioned as the location of the trap, so don't get confused!
NewStatus is a boolean, nonzero = true, zero = false.
unsigned int dpcrtlmm_IsBlockLocked(PS_DPCRTLMM_BLOCKDESCARRAY
PBlockArray, const void* Ptr);
Simply returns the boolean status 1 if the block specified has been
locked or 0 if it has not. When a block is locked, it cannot be freed
or resized.
#define dpcrtlmm_LockBlock(pArr, pBlock) dpcrtlmm_SetBlockLockingFlag(pArr,
pBlock, (1U));
#define dpcrtlmm_UnlockBlock(pArr, pBlock) dpcrtlmm_SetBlockLockingFlag(pArr,
pBlock, (0U));
Both of these functions are simple macros to lock and unlock blocks
of memory see dpcrtlmm_SetBlockLockingFlag(
)
void dpcrtlmm_ToggleBlockLockingStatus(PS_DPCRTLMM_BLOCKDESCARRAY
PBlockArray, const void* Ptr);
If the said block is locked, it will be unlocked and vice versa, not
sure if anyone will bother to use this function really
void dpcrtlmm_EnableTraps(void);
See dpcrtlmm_DisableTraps( ), normally
only that will be used in a program for the reasons described. The
only reason one might want to use this function is for temporary things
say like, disable traps, perform actions, re-enable traps. Implemented
as a macro, see dpcrtlmm__EnableTraps
void dpcrtlmm_DisableTraps(void);
Normally when the library starts traps are enabled, however let's say
that the program is massive, a deadline is coming and it must be released.
Although there are a few leaks that can't be tracked down it's stable and
users would have nothing to complain about. They won't be too happy
if DPCRTLMM keeps outputting all the mistakes at the end. Removing
the library may not be feasible if the program is designed intimately around
it so the answer is to disable the traps, just call this function once
after starting the library to avoid traps. Implemented as a macro,
see dpcrtlmm__EnableTraps
unsigned char dpcrtlmm_AreTrapsEnabled(void);
Returns whether traps are enabled or not, 1U means enabled 0U if not.
Implemented as a macro, see dpcrtlmm__EnableTraps
void dpcrtlmm_GetStats(PS_DPCRTLMM_STATS PReadStats);
Returns memory statistics for the library, such as, number of allocated
blocks, amount of memory used (charge) and peaks, pass a pointer to the
structure to read the data, forgetting to pass the pointer will do absolutely
nothing. Most of the information can be returned directly from internal
counts, most statistics are modified at the time they are changed, for
example allocating a block would increment the count of blocks and the
amount you added would be added to the total allocation charge. However,
because I made it possible to modify the flags of a descriptor I cannot
tell how many locks etc. there are, for these I had to implement a nested
loop to check the flags of all descriptors, shouldn't reduce performance
too much but remember that the function should not be called too often.
unsigned long dpcrtlmm_GetBlockCount(void);
Returns the number of blocks allocated, don't worry I'm not completely
mad, the count is stored internally it is not worked out by this function,
I don't go mad, looping through all the arrays looking for blocks!
PS_DPCRTLMM_VERSION dpcrtlmm_Ver(PS_DPCRTLMM_VERSION
PVerStruct);
Returns library version info, the caller supplies the structure and
I return them the same structure, passing NULL does nothing.
extern unsigned char dpcrtlmm__EnableTraps;
This can be used to disable/enable traps, while it is off no trap will
be executed and if the program makes a mistake it could crash, if can either
be modified directly or with the macros dpcrtlmm_EnableTraps(
), dpcrtlmm_DisableTraps( ) and dpcrtlmm_AreTrapsEnabled(
). Generally one will want to switch it off when releasing a
program, simple start the program with a call to dpcrtlmm_Startup(
) and then set this variable to 0U, the two valid states are 0U = no
traps, 1U = traps will be used. 1U is the default. If you have
the absoluetely version (source) this variable is store in dpcrtlmm.c
If you want to make sales enquires please contact sales@daybologic.co.uk, for lastest news check the web page and update your documentation & library regularly, remember updates cost nothing. If you have an equiry of a more technical nature you can contact me directly at Overlord@DayboLogic.co.uk
(C)Copyright 2000 Overlord David Duncan Ross Palmer, Daybo Logic. This copyright will be defended to th maximum extent of applicable law. The source is only released for rebuilding or user modifcation purposes for necersary extensions to functionaility. Modified version resale by purchasers or people who have obtained a registered copy through other means is a copyright violation if if we discover such activity do not assume we will not sue.