Calling an external DLL using the System.dll plugin

Author: djc (talk, contrib)

Description

Some install processes are required to call functions contained inside third party DLL files. A prime example of this is when installing a Palm(TM) conduit.

Some background about System.dll

The System.dll plug-in (by Brainsucker) enables calling of external DLL files by providing the 'Call' function. There are a number of other functions provided by System.dll, but they will not be covered here. For more details about the other functions, lock the doors, take the phone off the hook, screw your head on *real* tight and head on over to the Contrib/System directory and read the documentation there.

Data Types

System.dll recognises the following data types:

• v - void (generally for return)
• i - int (includes char, byte, short, handles, pointers and so on)
• l - long & large integer (know as int64)
• t - text, string (LPCSTR, pointer to first character)
• b - boolean (needs/returns 'true':'false') - by the fact this type is senseless -> usual integer can be used ('0':'1')
• k - callback. See Callback section in system.txt.
• * - pointer specifier -> the proc needs the pointer to type, affects next char (parameter) [ex: '*i' - pointer to int]

Mapping System.dll variables to NSIS script variables

There's not much point in being able to call an external function if you can't get any data back. System.dll maps function variables to NSIS script variables in the following way:

   NSIS $0..$9 become System.dll r0..r9
   NSIS $R0..$R9 become System.dll r10..r19

Each parameter is specified by type, input and output. To skip input or output use a dot. Examples:

String (pointer to a characters array), input is 'happy calling': t 'happy calling'

String (pointer to a characters array), input is taken from $5 and changes to the array made by the call are saved into $R8: t r5r18

Pointer to an integer, value taken from $1 and put into $2: *i r1r2

Pointer to a 64-bit integer, output pushed on stack, no input: *l .s

Using System.dll::Call

To call a function in a third party DLL, the Call function is used like this:

System::Call 'YourDllName::YourDllFunction(i, *i, t) i(.r0, r1, .r2) .r3'

The

'(.r0, r1, .r2) .r3'

section at the end are the parameters that are passed between your DLL and your NSIS script. As can be seen in this parameters list type and input/output can be separated. Each block of "(parms list) return value" overrides and/or adds to the last one. In this case, the first block specifies the types and the second specifies input and output.

Before starting to code the NSIS script

Before you start to code any NSIS code, you need to know the full prototype of the function you are going to call. For the purposes of this example, we will use the 'CmGetHotSyncExecPath' function from the Palm 'CondMgr.dll'. This function is used to return the full path of 'HotSync.exe'.

Function Definition

int CmGetHotSyncExecPath(TCHAR *pPath, int *piSize);

where

• *pPath is a pointer to a character buffer. Upon return, this is the path & file name of the installed HotSync manager.
• *piSize is a pointer to an integer that specifies the size (in TCHAR's), of the buffer referenced by the pPath parameter.

Return values:

• 0: No error
• -1: A non-specific error occurred
• ERR_REGISTRY_ACCESS(-1006):Unable to access the Palm configuration entries
• ERR_BUFFER_TOO_SMALL(-1010): The buffer is too small to hold the requested information
• ERR_INVALID_POINTER(-1013):The specified pointer is not a valid pointer

Also, if the buffer is too small the value in *int is the size (in TCHARs) that the buffer should be.

This function definition maps to the following System.dll definition:

CmGetHotSyncExecPath(t, *i) i

i.e. It takes a text variable, a pointer to int, and returns an int value.

Using the external dll function

Now that we've sorted out what the function does, and how it maps to the System.dll format, we can use the function in a NSIS script.

First, it is recommended to turn 'PluginUnload' off before making multiple calls to System.dll. According to Brainsucker (and others), this will speed up execution of the installer package.

Second, you have to change the output directory to that where the DLL you want to use is. It may also work if the DLL is on the system path, but this hasn't been tested.

The following code fragment will install 'condmgr.dll' to a temporary directory, execute the CmGetHotSyncExecPath function, display returned data and finally unload the System.dll plug-in.

; **** snip ****
SetPluginUnload  alwaysoff
 
Function loadDll
 
  SetOutPath $TEMP\eInspect             ; create temp directory
  File bin\CondMgr.dll                  ; copy dll there
  StrCpy $1 ${NSIS_MAX_STRLEN}          ; assign memory to $0
  System::Call 'CondMgr::CmGetHotSyncExecPath(t, *i) i(.r0, r1).r2'
  DetailPrint 'Path: "$0"'
  DetailPrint "Path length: $1"
  DetailPrint "Return value: $2"
 
; last plugin call must not have /NOUNLOAD so NSIS will be able to delete 
; the temporary DLL
 
  SetPluginUnload manual
; do nothing (but let the installer unload the System dll)
  System::Free 0
FunctionEnd
; **** snip ****

and this function produces the following output in the 'details' page:

Output folder: c:\windows\TEMP\eInspect
Extract: CondMgr.dll
Path: "C:\Dave\palm\Hotsync.exe"
Path length: 24
Return value: 0

Acknowledgements & Thanks

Lots of thanks go to kichik and Sunjammer for spending a lot of time assisting in solving this problem. Also to brainsucker for creating the System.dll plug-in in the first place. Good Luck!