Previously, we learned we needed to put our callback function in a separate DLL and then load that DLL in another program. So you did that you create a separate dll, you put the callback there you call the SetWindowsHookEx function to set up the callback, But the callback doesn’t get called at all. So what’s wrong?
In order to figure this out, you need to notice that the SetWindowsHookEx function is actually a DLL injector. What does that mean? It means it first loads or injects your DLL inside the target application, And then it will hook your callback to the Windows messages queue or any other callback that you specified.
So for this to work, there are several conditions that need to be met otherwise several things can go wrong. And in my case, all of them went wrong. I’m going to list them here. So maybe it will help you. By any chance if you bump into any other issue and you finally got to solve it. Then please mention that in the comments, so other people and I can learn from your mistakes as well. Alright, let’s get to the problems.
The first problem that I had was the thread ID. So there are several ways to get the thread ID. First, don’t use the GetWindowThreadProcessId function
the best and most legitimate way to get the thread ID is by using the CreateToolhelp32Snapshot function which goes through all threads and checks if they are the main thread for an application. Actually, I found that the CreateToolhelp32Snapshot function is returning a different value than the GetWindowThreadProcessId, and it was actually the reason why my DLL doesn’t get floats. So that’s first.
Second, you have to be aware that your callback is going to inject your DLL into another program. So you have to make sure that the DLL that you wrote is not dependent on any other libraries other than the one that is already installed in the “System Path”.
so in my case, I was using the Qt library and the SetWindowsHookEx function was installing the hook successfully, But in reality, the dll never gets injected into the target program.
The last dilemma was, don’t use the printf function without taking extra care. So in my case, I was using this
To know why you got to understand that when you’re calling this function from your application and then you call AllocConsole function, it will open up a console that is hooked up to your application not to the DLL that is injected into the program so if you printf something in your callback, you won’t see anything because it’s running on a thread on your main application. If you want to see that you need to call the AllocConsole function inside your DLL main function.
Otherwise when you use the printf function that will write to the output of the console of the target processor instead of the console that you had opened
