Check whether your application is running

Author: Wizou (talk, contrib)

There are many methods to detect whether your application is running or not.

It is much better to implement a method that involve some kind of cooperation from your application so that you are sure that you will detect your application specifically. However many of these methods also work without specific cooperation from the application.

Choose your method below, and place the adequate code in your installer script. If you want to check at the very start, use the .onInit (installer) or un.onInit (uninstaller) function. Or you may want to do it later, in a section or a page callback function.

using a DDE server

If the application implements a DDE server, your (un)installer can use the nsisDDE plug-in to contact the DDE server to query its state, bring the application to the foreground or ask the application to exit.

If your application is written in MFC, it is very easy to add a DDE server in it.. (don't forget to use GuidGen tool to generate a unique service identifier specific to your application)

Many applications already implement such a DDE server (typically used by Explorer to open/print documents). You can search for "ddeexec" keys under HKEY_CLASSES_ROOT to find the name of applications' DDE server.

Examples:

nsisDDE::Execute "MyApp-{5A60A807-D40C-4554-935A-E570B818DF75}" "[Quit]"
Pop $0
IntCmp $0 0 notRunning
    Sleep 2000 ; give 2 seconds for the application to finish exiting
notRunning:

nsisDDE::Execute "MyApp-{5A60A807-D40C-4554-935A-E570B818DF75}" "[SetForeground]"
Pop $0
IntCmp $0 0 notRunning
    MessageBox MB_OK|MB_ICONEXCLAMATION "MyApp is running (see foreground window). Please close it first" /SD IDOK
    Abort
notRunning:

using a Win32 Synchronization Object

Many applications typically create a named mutex to prevent multiple instance of the application. Your (un)installer could call the Win32 API "OpenMutex" to detect if this named mutex is active.

If you can, make sure you choose a unique name for the application mutex, typically generated with the GuidGen tool. ex: "MyApp-{5A60A807-D40C-4554-935A-E570B818DF75}"

If you don't have access to the application source code, you can use tools like ProcessExplorer to find the synchronization objects owned by an application.

Example:

System::Call 'kernel32::OpenMutex(i 0x100000, b 0, t "MyAppMutex") i .R0'
IntCmp $R0 0 notRunning
    System::Call 'kernel32::CloseHandle(i $R0)'
    MessageBox MB_OK|MB_ICONEXCLAMATION "MyApp is running. Please close it first" /SD IDOK
    Abort
notRunning:

Advanced applications may use other Win32 named synchronization objects like an Event or a Pipe to send a signal or a command to the application.

using the name of the process

You can detect if your application is running by using the FindProcDLL plug-in to check if a process having the name of your application executable is running.

Make sure your application executable has an easily identifiable name, or you risk detecting another unrelated application having the same name as yours.

Example:

FindProcDLL::FindProc "MyApp.exe"
IntCmp $R0 1 0 notRunning
    MessageBox MB_OK|MB_ICONEXCLAMATION "MyApp is running. Please close it first" /SD IDOK
    Abort
notRunning:

Additionally you may decide (or offer the user the possibility) to kill the running process forcibly, by using the KillProc plug-in. Note: This can be dangerous as the state of the application and edited files might not be saved correctly.

using the name of the process (with Windows's tasklist command)

Note: tasklist tool is only present in Windows XP professional or more recent Windows

An alternative to the FindProc plugin above is to check the output of Windows's 'tasklist' command to see if it contains the process name. (FindProc didn't seem to work for the author of this subsection). This could be done with the ExecWait command, but that would flash a command prompt window. Instead, the ExecCmd_plug-in can be used to run the command in a hidden window.

Again, make sure your application executable has an easily identifiable name, or you risk detecting another unrelated application having the same name as yours.

Example:

!define FindProc_NOT_FOUND 1
!define FindProc_FOUND 0
!macro FindProc result processName
    ExecCmd::exec "tasklist /NH /FI $\"IMAGENAME eq ${processName}$\" | find /I $\"${processName}$\"" 
    Pop $0 ; The handle for the process
    ExecCmd::wait $0
    Pop ${result} ; The exit code
!macroend
 
Var /GLOBAL processFound
!insertmacro FindProc $processFound "MyApp.exe"

Additionally you may decide (or offer the user the possibility) to kill the running process forcibly, by using the KillProc plug-in. Note: This can be dangerous as the state of the application and edited files might not be saved correctly.

using a window class

If your application creates a (main) window with a specific classname or title, you can use it to detect whether the application is running.

Unless you're sure of the application behaviour, be aware that many applications have dynamically generated window classname (different each time), and localized window title depending on the user language.

You can use the Visual Studio tool Spy++ or a freeware tool like Winlister or WinSpy++ to find the window class of your application.

Examples:

FindWindow $0 "MyAppWindowClass"
StrCmp $0 0 notRunning
    MessageBox MB_OK|MB_ICONEXCLAMATION "MyApp is running. Please close it first" /SD IDOK
    Abort
notRunning:

or

FindWindow $0 "" "MyApp Window Title"
StrCmp $0 0 notRunning
    MessageBox MB_OK|MB_ICONEXCLAMATION "MyApp is running. Please close it first" /SD IDOK
    Abort
notRunning:

Once you obtained the window handle with FindWindow, you can also use SendMessage to send WM_CLOSE to close the application. Note that this may not work gracefully with all applications, for example applications with multiple top-level windows.

using a registry entry

Your application could, for example, write 1 on startup in a registry entry, and write 0 when the application is closed. You can then test this registry in your (un)installer.

However this solution is not clean because the registry would not be reset to 0 if your application crashes or is forcibly killed.

Example:

ReadRegDWORD $R0 HKLM "SOFTWARE\MyCompany\MyApp" "IsRunning"
IntCmp $R0 0 notRunning
    MessageBox MB_OK|MB_ICONEXCLAMATION "MyApp is running. Please close it first" /SD IDOK
    Abort
notRunning: