AutoHotKey Debugger for IntelliJ Platform
IntelliJ platform allows the plugins to implement debuggers for various technologies. In this post, we will explore how to implement a debugger for AutoHotKey (via DBGP common debugging protocol) using the XDebugger API of IntelliJ Platform.
For this tutorial, I was looking for something that isn't yet available in IntelliJ, while having a reasonable level of complexity to demonstrate the debugging API. So I chose the DBGP protocol (the one originally invented by PHP but later used by some other parties), and in particular — ability to debug AutoHotKey programs using it. As it turns out, AutoHotKey supports debugging via DBGP for a long time already; here's the list of compatible debuggers that understand the protocol and support AutoHotKey.
So, in this post I'll explain some core concepts that are required for understanding how debuggers work in IntelliJ, then explain my approach to DBGP, and then show some examples of the code in the debugger. All the sources are available in a separate repository: intellij-autohotkey-debugger-sample.
To start a new plugin, use the IntelliJ Platform Plugin Template repository: intellij-platform-plugin-template.
Core Concepts
Run Configuration
The first, obvious entry point for the user to debug a program in IntelliJ is a corresponding run configuration: the user clicks the 🐞 Debug button on the main toolbar, and the debugger gets started.
The run configuration subsystem in IntelliJ is a bit complex and has a lot of moving parts; it's worth reading the documentation to understand the basics.
Let's imagine we have implemented a run configuration already; how to make it debuggable in IntelliJ then? The main specific is that there should be a ProgramRunner that returns true from its canRun when passed DefaultDebugExecutor.EXECUTOR_ID and your run configuration. This design allows you to even implement debug runners for others' run configurations when necessary.
Your ProgramRunner is then supposed to call com.intellij.xdebugger.XDebuggerManager::startSession method: this will initialize the debugging session, create the required UI elements etc.
Debugger API
There are two main points that control the debugger in IntelliJ:
- You are supposed to extend
XDebugProcessclass — it will receive various signals from user, be it actions like Pause or Resume, or various more complex user interactions, like controlling the breakpoints, or evaluating expressions. - You will use a platform-provided implementation of
XDebugSessionthat processes information in a different direction — any events happening in the debugger should be passed to this class.
How do you provide the platform with your instance of XDebugProcess? And, while we are at it, how do you obtain an instance of XDebugSession? There's no special registration for XDebugProcess, but it is supposed to be created by your XDebugProcessStarter. The starter will also provide you with the session instance. For example:
val debuggerManager = XDebuggerManager.getInstance(environment.project)
debuggerManager.startSession(environment, object : XDebugProcessStarter() {
override fun start(session: XDebugSession): XDebugProcess {
return AutoHotKeyDebugProcess(session, …)
}
})
XDebugProcess
XDebugProcess has several groups of members that should be implemented by the user:
Debug action handlers, e.g.:
startStepOver,startStepInto,startStepOut,stopAsync,resume,runToPosition,sessionInitialized
(there are others).
These are called in response to direct user actions. You are supposed to handle them as signals to your debuggee, and they should lead to some changes in the debug session, perhaps asynchronously.
Various state managers:
doGetProcessHandler,getEditorsProvider,getBreakpointHandlers,createConsole.
Not all of these are required, but you should implement at least all the abstract members of
XDebugProcess.Various IDE subsystems will use these managers to interact with your debug process, e.g. manage the breakpoints, watched variables, and expression evaluator (
EditorsProvideris involved in this).
Editor Provider
An implementation of XDebuggerEditorsProvider should handle expression evaluation and completion in the evaluation editor. You see, whenever the user edits a watch expression or uses the Evaluate action, IntelliJ will create an editor for this purpose, and each editor has a document underneath. To correctly provide completion in this document, it will ask the editor provider implementation to handle it.
Breakpoint Handler
An implementation of XBreakpointHandler should handle breakpoint-related actions, such as adding and removing them, so it has two main methods: registerBreakpoint and unregisterBreakpoint.
There are related concepts of verified and invalid breakpoints.
A verified breakpoint is a breakpoint that is currently valid and can be hit by the debugger — usually it's marked as a green check mark in the debugger UI. Sometimes, breakpoints can be unverified for some time, e.g. while the corresponding file is not yet loaded into the interpteter under debug, so a breakpoint might be correctly set but not yet resolved. If your debugger supports this distinction, it might signal breakpoint's resolve state via XDebugSession::setBreakpointVerified.
An invalid breakpoint is a breakpoint that will never be hit — for example, if the process under debug has loaded a different version of the current file, or some other error happened while attempting to set the breakpoint. You can mark a breakpoint as invalid via xDebugSession.setBreakpointInvalid(breakpoint, reason).
Most of IntelliJ breakpoints inherit from XLineBreakpoint, meaning it's always possible to determine which line a breakpoint belongs to (it's also possible to create more precise breakpoints if case of several statements per line).
XDebugSession
You should trigger corresponding methods on XDebugSession object when events happen in the debuggee. Most useful examples:
- when a debugger has stopped on a breakpoint, call
session.breakpointReached; - when a debugger has stopped in some other place (e.g. on the explicit Pause action), call
session.positionReached.
Both of these methods require an instance of XSuspendContext, another important class.
XSuspendContext
You should create a custom inheritor from XSuspendContext, fill it with information and pass into the XDebugSession's methods when you signal that the debuggee is paused. This class is the main entry point into the current "program state" of sorts — it knows how to obtain stacks of all threads, local and global variables in each thread.
The top stack frame is supposed to be immediately available, while all the other information can be calculated asynchronously. So, methods like AutoHotKeyExecutionStack::computeStackFrames or XStackFrame::computeChildren can schedule background calculations that will resolve at later point of time. Since the code base of XDebugger was designed a while ago, these methods don't make any use of coroutines or modern concurrency APIs like channels; nevertheless, they provide means for the implementation to report errors, provide items one by one or in groups, and signal of computation termination. The general form might be demonstrated by this example:
class AutoHotKeyExecutionStack(…) : XExecutionStack(…) {
// val coroutineScope: CoroutineScope
// val client: DbgpClient (all calls are suspending)
override fun computeStackFrames(firstFrameIndex: Int, container: XStackFrameContainer) {
coroutineScope.launch {
try {
for (d in firstFrameIndex until maxDepth) {
val info = dbgpClient.getStackInfo(d)
val isLast = d == maxDepth - 1
container.addStackFrames(
listOf(AutoHotKeyStackFrame(coroutineScope, dbgpClient, info, d)),
isLast
)
}
} catch (e: Throwable) {
if (e is ControlFlowException || e is CancellationException) throw e
container.errorOccurred(…)
}
}
}
}
Here you can see how the debugger implementation spawns an asynchronous activity using a coroutineScope, calculates items via a suspending API, and then feeds the items as soo as they are calculated to the passed container. Any error except the cancellation is passed into the container.errorOccurred method that will report failure to the user. The last invocation of addStackFrames will report the fact that it's the last one by passing true as the second parameter.
Note that the coroutineScope is not passed from the debugger; you are supposed to manage its lifetime yourself. It is recommended that you terminate/cancel previous coroutine scope when entering a new suspending context — i.e. when the debugger has entered a new stage, any calculations related to the previous context might be cancelled. This is a responsibility of the party that provides this context, in most cases you, not the platform.
XStackFrame
A stack frame, as far as IntelliJ is concerned, mostly plays the role of a container for the variables. Variables might be nested into each other (e.g. an object variable nests its child properties), and they are stored in groups. XStackFrame inherits from XValueContainer which has a single method, computeChildren, which follows the same general async computation rules as computeStackFrames from the previous section.
The children of XStackFrame are a bit more complex, though. They can be separated into multiple groups, and when adding children to the XCompositeNode, you can choose to either add them directly or group into XValueGroup.
For example, here we have an implementation that collects the children from default context directly, and groups values from other contexts into groups (which might be static variables, global variables, or anything else debugger-specific) — note how list.add is different from list.addBottomGroup:
override fun computeChildren(node: XCompositeNode) {
scope.launch {
try {
val list = XValueChildrenList()
val contexts = dbgp.getAllContexts()
for (context in contexts) {
val properties = dbgp.getProperties(depth, context.id)
val isDefault = context.id == 0
if (isDefault) {
for (property in properties) {
list.add(AutoHotKeyValue(scope, dbgp, depth, property))
}
} else {
val group = AutoHotKeyValueGroup(scope, dbgp, depth, context.name, properties)
list.addBottomGroup(group)
}
}
val isLast = true
node.addChildren(list, isLast)
} catch (e: Exception) {
if (e is ControlFlowException || e is CancellationException) throw e
logger.error(e)
node.setErrorMessage((e.localizedMessage ?: e.message)?.nullize() ?: DebuggerBundle.message("general.unknown-error"))
}
}
}
XValue
Any "variable" shown in the debugger tree is an XValue. They have the same property of their children being dynamically computable via computeChildren. Another important feature is XValue::getModifier which can provide an object that will be invoked to modify the value of a variable.
XValueModifier::setValue is also an asynchornous method, the implementation is supposed to call either callback.valueModified() or callback.errorOccurred() on finishing.
Event Ordering Notes
Most of debuggers will need to correctly order their asynchronous events. One particularly interesting place is the process startup. Imagine the user starts a process: in this case, IntelliJ will want to restore breakpoints from a previous session (if any), and then proceed with debugging. So, it will call your XBreakpointHandler::registerBreakpoint, immediately followed by XDebugProcess::sessionInitialized.
It is the implementation's responsibility to make sure these events are ordered correctly, and e.g. the debuggee doesn't resume its execution before all the breakpoints are processed.
If your debugger processes events asynchronously, you should be extra careful to not allow them to be reordered by any party; in particular, you should be very careful about the coroutine calls in Kotlin — mind that any coroutineScope.launch or coroutineScope.async might launch the computation out of order.
One possible solution for this problem would be to design your own debugger class with plain synchronous API, each method call being a launcher for an asynchronous activity that will be ordered correctly. For example, imagine that IntelliJ wishes to call the following sequence of actions in our debugger abstraction:
registerBreakpoint()
registerBreakpoint()
resumeExecution() // triggered by sessionInitialized()
We might design a CustomDebugger type with the following methods:
interface CustomDebugger {
// `launch` prefix suggests that these methods behave similar to Kotlin's
// coroutine `launch`: they start an async computation.
fun launchRegisterBreakpoint()
fun launchResumeExecution()
}
and then implement them in a manner like this:
class CustomDebuggerImpl(private val parentScope: CoroutineScope) : CustomDebugger {
// note `suspend`
private suspend fun doRegisterBreakpoint()
private suspend fun doResumeExecution()
private val scope = parentScope.childScope("Custom Debugger")
private val singleAccess = Mutex()
private fun launchInOrder(block: suspend CoroutineScope.() -> Unit) {
try {
scope.launch(start = CoroutineStart.UNDISPATCHED) {
singleAccess.withLock {
block()
}
}
} catch (e: CancellationException) {
throw e
} catch (e: Throwable) {
if (e is ControlFlowException) throw e
logger.error(e)
}
}
override fun launchRegisterBreakpoint() {
launchInOrder { doRegisterBreakpoint() }
}
override fun launchResumeExecution() {
launchInOrder { doResumeExecution() }
}
}
This construct guarantees:
- That only one action of the ones scheduled under
launchInOrderis executed simultaneously, even if they suspend. - That the first suspension point each action hits will be
singleAccess.withLock— meaning the actions will always be executed in the order theirlaunch*methods were called (that is — if they were called in any order at all, and there's no race condition in the IntelliJ platform).
DBGP Protocol
In this section, we'll discuss my implementation of handler for the DBGP protocol. First, let's see the interface of our custom debug client:
sealed interface DbgpClientEvent
data class BreakExecution(val breakpoint: XLineBreakpoint<*>?) : DbgpClientEvent
interface DbgpClient {
suspend fun setBreakpoint(breakpoint: XLineBreakpoint<*>): Boolean
suspend fun removeBreakpoint(breakpoint: XLineBreakpoint<*>)
suspend fun run()
suspend fun getStackDepth(): Int
suspend fun getStackInfo(depth: Int): DbgpStackInfo
suspend fun getAllContexts(): List<DbgpContextInfo>
suspend fun getProperties(depth: Int, contextId: Int): List<DbgpPropertyInfo>
suspend fun getProperty(property: DbgpPropertyInfo, stackDepth: Int): DbgpPropertyInfo
suspend fun setProperty(property: DbgpPropertyInfo, stackDepth: Int, value: String)
val sessionInitialized: Deferred<Unit>
val events: Channel<DbgpClientEvent>
}
You can see that it implements several execution-related commands (set/removeBreakpoint, run), some information retrieval commands (getStackDepth, getStackInfo, getAllContexts, getProperties), and property manipulation commands (getProperty, setProperty).
Additionally, it has a signal (Deferred) about its initialization, and a channel with all the events (the only kind of supported event being BreakExecution).
Let's see how it's implemented; here I only show one example of a run command since it's quite good for illustrational purposes.
class DbgpClientImpl(scope: CoroutineScope, private val socket: AsynchronousSocketChannel) : DbgpClient {
private val pendingResponses = concurrentMapOf<Int, CompletableDeferred<DbgpResponse>>()
override suspend fun run() {
val response = command("run")
handleContinuationResponse(response)
}
private val writeMutex = Mutex()
private var lastTransactionId = 0
private suspend fun command(command: String, vararg args: String): DbgpResponse =
writeMutex.withLock {
val transactionId = lastTransactionId++
val response = CompletableDeferred<DbgpResponse>()
pendingResponses[transactionId] = response
val commandList = listOf(command) + listOf("-i", transactionId.toString()) + args
val fullCommand = commandList.joinToString(" ") + "\u0000" // TODO: escape the arguments as needed
logger.trace { "Sending command: $fullCommand" }
socket.writeSuspending(fullCommand.toByteArray(Charsets.UTF_8))
logger.trace { "Awaiting for response with transaction ID $transactionId." }
response.await()
}
private suspend fun handleContinuationResponse(response: DbgpResponse) {
logger.trace { "Processing response: $response" }
if (response.command == "run" && response.status == "break") {
logger.trace("We hit a breakpoint. Getting the stack trace.")
val top = getStackInfo(0)
val breakpoint = activeBreakpoints.keys.firstOrNull { bp ->
val sp = bp.sourcePosition
sp != null &&
sp.file.toNioPath() == top.file &&
sp.line == top.oneBasedLineNumber - 1
}
logger.trace("Found breakpoint: $breakpoint.")
events.send(BreakExecution(breakpoint))
}
}
}
Here you can see that a command will write some data to the socket (check the protocol description for details what the command format is), and put a continuation token into the pendingResponses map.
Let's see the socket reader routines now:
class DbgpClientImpl(scope: CoroutineScope, private val socket: AsynchronousSocketChannel) : DbgpClient {
private val packets = Channel<DbgpPacket>(capacity = Channel.UNLIMITED)
init {
launchSocketReader(scope, socket)
launchPacketDispatcher(scope)
}
private fun launchSocketReader(scope: CoroutineScope, socket: AsynchronousSocketChannel) {
scope.launch(CoroutineName("DBGP socket reader")) {
while (true) {
val packet = readPacketBody(socket) ?: break
dispatchPacketBody(packet)
}
}
}
private suspend fun dispatchPacketBody(body: ByteArray) {
val xml = body.toString(Charsets.UTF_8)
try {
logger.trace { "Received packet:\n$xml" }
val packet = DbgpPacketParser.parse(xml)
packets.send(packet)
} catch (e: Throwable) {
if (e is ControlFlowException || e is CancellationException) throw e
logger.error("Failed to parse DBGP packet:\n$xml", e)
}
}
private fun launchPacketDispatcher(scope: CoroutineScope) {
scope.launch(CoroutineName("DBGP packet dispatcher")) {
while (true) {
val packet = packets.receive()
try {
when (packet) {
is DbgpInit -> initialized.complete(Unit)
is DbgpResponse -> {
val transactionId = packet.transactionId
pendingResponses[transactionId]?.let { deferred ->
deferred.complete(packet)
pendingResponses.remove(transactionId) // safe: transaction ids are never reused
} ?: logger.warn("Received a response with unknown transaction ID: ${packet.transactionId}.")
}
}
} catch (e: Throwable) {
if (e is ControlFlowException || e is CancellationException) throw e
logger.error("Unable to process the packet $packet.", e)
}
}
}
}
}
So, we have a coroutine that endlessly (unless cancelled) reads new packets from the socket, and then it writes the packets to a Channel. Another coroutine reads from said Channel and dispatches the results — might trigger the initialized flag, or resolve one of the pendingResponses.
This is supposed to be free of data races because there's only one reader per Channel, and if any overflow happens, the events will just be stored in channel's buffer.
Implementation Notes
Run Configuration
Run Mode
First of all, let's make sure the user is able to run the program — not debug yet.
To do this, we'll need to implement and register several things, mostly boilerplate.
An implementation of the
ConfigurationTypeinterface, also registered in theplugin.xml:class AutoHotKeyRunConfigurationType : ConfigurationTypeBase(…) { // … }Registering this type will allow the user to create a run configuration manually in the run configuration dialog.
An implementation of the
RunConfigurationinterface that will be produced by the factory:class AutoHotKeyRunConfiguration(…) : LocatableConfigurationBase<Element>(…) { var filePath: Path? = null override fun getState( executor: Executor, environment: ExecutionEnvironment ): RunProfileState = AutoHotKeyRunProfileState( environment, filePath ?: throw CantRunException("File path is not set.") ) // … }A factory class that will be producing the configuration (mostly used by the
ConfigurationTypeimplementation):class AutoHotKeyRunConfigurationFactory(…) : ConfigurationFactory(…)An implementation of the
RunProfileStateinterface that contains the code that actually runs the process in the PTY mode (this mode isn't important for AutoHotKey, but it's a good practice to use it for other languages):class AutoHotKeyRunProfileState( environment: ExecutionEnvironment, private val filePath: Path ) : CommandLineState(environment) { // … private fun startProcess(arguments: List<String>): ProcessHandler { val interpreter = findAutoHotKeyInterpreter() ?: throw CantRunException(DebuggerBundle.message("run-configuration.error.interpreter-not-found")) val commandLine = PtyCommandLine() .withConsoleMode(false) .withWorkingDirectory(filePath.parent) .withExePath(interpreter.pathString) .withParameters(arguments + filePath.pathString) return object : KillableColoredProcessHandler(commandLine) { override fun shouldKillProcessSoftly() = false } } override fun startProcess(): ProcessHandler = startProcess(emptyList()) }Here,
private fun startProcessis extracted only for the future use when we start implementing the debugger: it will use the same function, but add some arguments.One notable fact here is that we override the
shouldKillProcessSoftlyfunction for our implementation of theProcessHandler: this is necessary because AutoHotKey doesn't track the graceful termination attempts when run in PTY mode, so there's no sense in trying to terminate it gracefully. This isn't necessarily true for any other debuggers and run configurations, though — you can provide your own graceful termination routines if necessary.For the user's convenience, to be able to use the Run Current File function, and run
.ahkfiles from the context menu, implement and register an implementation of theRunConfigurationProducerinterface:class AutoHotKeyRunConfigurationProducer : LazyRunConfigurationProducer<AutoHotKeyRunConfiguration>() { // … }
Debug Mode
Having that sorted out, let's focus on the debugger. As explained above, to make your custom run configuration debuggable, you need a custom runner. Let's implement some boilerplate first:
class AutoHotKeyDebugProgramRunner(private val scope: CoroutineScope) : AsyncProgramRunner<RunnerSettings>() {
override fun getRunnerId(): @NonNls String = "AutoHotKeyDebugRunner"
override fun canRun(executorId: String, profile: RunProfile): Boolean =
executorId == DefaultDebugExecutor.EXECUTOR_ID && profile is AutoHotKeyRunConfiguration
}
and remember to register it in the plugin.xml.
Now, let's get to the actual implementation. The first thing we need is the execute method. In IntelliJ, it returns a Promise (an old type designated for asynchronous workflows), but we'll use coroutines and adapter to Promise:
@ExperimentalCoroutinesApi
override fun execute(
environment: ExecutionEnvironment,
state: RunProfileState
): Promise<RunContentDescriptor?> = scope.async {
saveAllDocuments()
val session = createDebugSession(environment, state as AutoHotKeyRunProfileState)
session.runContentDescriptor
}.toPromise()
Note that the state argument will be the state received from our run configuration via its AutoHotKeyRunConfiguration::getState method.
Now, createDebugSession that delegates most of the work to XDebuggerManager::startSession:
suspend fun createDebugSession(
environment: ExecutionEnvironment,
state: AutoHotKeyRunProfileState,
listener: XDebugSessionListener? = null
): XDebugSession {
val debuggerManager = XDebuggerManager.getInstance(environment.project)
val debugger = startDebugServer()
try {
val processHandler = state.startDebugProcess(debugger.port)
return withContext(Dispatchers.EDT) {
debuggerManager.startSession(environment, object : XDebugProcessStarter() {
override fun start(session: XDebugSession): XDebugProcess {
listener?.let { session.addSessionListener(it) }
return AutoHotKeyDebugProcess(session, processHandler, debugger)
}
})
}
} catch (e: Exception) {
Disposer.dispose(debugger)
throw e
}
}
So, here we call state.startDebugProcess, and then wrap it into an AutoHotKeyDebugProcess.
state.startDebugProcess will just call the aforementioned startProcess while adding a debug argument:
class AutoHotKeyRunProfileState(…) : … {
// …
suspend fun startDebugProcess(port: Int): ProcessHandler {
val command = "/Debug=127.0.0.1:$port"
logger.info("Will execute command in debuggee process: $command")
return withContext(Dispatchers.IO) { startProcess(listOf(command)) }
}
// …
}
Now, how is the startDebugServer implemented?
private suspend fun startDebugServer(): AutoHotKeyDebugger {
return withContext(Dispatchers.IO) {
val port = NetUtils.findFreePort(9000)
AutoHotKeyDebugger(port, scope)
}
}
AutoHotKeyDebugger is a part of DBGP implementation and has been described above.
Breakpoints
The first thing we need is to allow the user to put the breakpoints into .ahk files; by default, IDE will not allow this, as the files are recognized as the plain-text ones.
So, let's start by creating a class inheriting the XLineBreakpointType:
class AutoHotKeyBreakpointType : XLineBreakpointType<XBreakpointProperties<*>>("ahk", …) {
override fun canPutAt(
file: VirtualFile,
line: Int,
project: Project
): Boolean {
return file.extension == "ahk"
}
override fun createBreakpointProperties(
file: VirtualFile,
line: Int
): XBreakpointProperties<*>? = null
}
Here, the most important part is the canPutAt method, which checks if the file extension is ahk. In other languages, you might decide to check for file's language (since the user might mark other files as belonging to the language), but for this simple example check for file extension will be enough.
The Debugger
This is the class that will wrap our DbgpClient (see above) and provide services for the IDE:
interface DbgpDebugger {
fun launchResumeExecution()
fun launchSetBreakpoint(
breakpoint: XLineBreakpoint<*>,
successCallback: (Boolean) -> Unit,
errorCallback: (Throwable) -> Unit
)
fun launchRemoveBreakpoint(breakpoint: XLineBreakpoint<*>)
fun connectToSession(session: XDebugSession)
}
class AutoHotKeyDebugger(val port: Int, parentScope: CoroutineScope) : DbgpDebugger {
// The stuff related to call ordering has been already described above.
private val client: Deferred<DbgpClient>
init {
socketChannel.bind(InetSocketAddress(port))
client = scope.async(Dispatchers.IO) {
val channel = socketChannel.accept().await()
DbgpClientImpl(scope, channel)
}
}
override fun connectToSession(session: XDebugSession) {
scope.launch {
var currentSuspendScope: CoroutineScope? = null
logger.trace("Awaiting for client.")
val client = client.await()
logger.trace("Subscribing for events.")
client.events.consumeEach { event ->
logger.trace { "Received event: $event" }
when(event) {
is BreakExecution -> {
// We stop on a new breakpoint, terminate any calculations related to the previous one.
currentSuspendScope?.cancel()
currentSuspendScope = scope.childScope("AutoHotkeyDebugger: current execution scope")
val depth = client.getStackDepth()
val stack = AutoHotKeyExecutionStack(
currentSuspendScope,
client,
client.getStackInfo(0),
depth
)
val sc = AutoHotKeySuspendContext(stack)
val bp = event.breakpoint
if (bp != null) {
// Use breakpointReached to auto-focus the Threads tab when we know the exact breakpoint.
session.breakpointReached(bp, null, sc)
} else {
session.positionReached(sc)
}
}
}
}
}
}
}
Note how it implements asynchronous startup sequence (so that other activities might be performed after client.await() resolves), and how the suspend context lifetime management is implemented — the previous context's coroutineScope gets cancelled after the next one appears.
XDebugProcess
The AutoHotKeyDebugProcess will be the heart of our implementation. But first, let's see how its parts are implemented.
Debugger will need an editor provider for the user to edit the evaluation expressions:
class AutoHotKeyDebuggerEditorsProvider : XDebuggerEditorsProvider() {
override fun getFileType(): FileType = PlainTextFileType.INSTANCE
companion object {
private var documentId = AtomicInteger(0)
}
override fun createDocument(
project: Project,
expression: XExpression,
sourcePosition: XSourcePosition?,
mode: EvaluationMode
): Document {
val id = documentId.getAndIncrement()
val psiFile = PsiFileFactory.getInstance(project)
.createFileFromText(
"debugger$id.ahk",
PlainTextFileType.INSTANCE,
expression.expression,
LocalTimeCounter.currentTime(),
true
)
val document = PsiDocumentManager.getInstance(project).getDocument(psiFile)!!
return document
}
}
Then, we'll need a breakpoint handler, which will simplt delegate the heavy work to AutoHotKeyDebugger:
class AutoHotKeyBreakpointHandler(
private val session: XDebugSession,
private val debugger: AutoHotKeyDebugger
) : XBreakpointHandler<XLineBreakpoint<XBreakpointProperties<*>>>(AutoHotKeyBreakpointType::class.java) {
companion object {
private val logger = logger<AutoHotKeyBreakpointHandler>()
}
private fun validateBreakpoint(breakpoint: XLineBreakpoint<XBreakpointProperties<*>>): Boolean {
val sourcePosition = breakpoint.sourcePosition
if (sourcePosition == null || !sourcePosition.file.exists() || !sourcePosition.file.isValid) {
session.setBreakpointInvalid(breakpoint, DebuggerBundle.message("breakpoint.invalid.message"))
logger.warn("Invalid breakpoint: $breakpoint: file doesn't exist or is invalid")
return false
}
val lineNumber: Int = breakpoint.line
if (lineNumber < 0) {
session.setBreakpointInvalid(breakpoint, DebuggerBundle.message("breakpoint.invalid.message"))
logger.warn("Invalid breakpoint $breakpoint: line $lineNumber")
return false
}
return true
}
override fun registerBreakpoint(breakpoint: XLineBreakpoint<XBreakpointProperties<*>>) {
if (!validateBreakpoint(breakpoint)) return
debugger.launchSetBreakpoint(breakpoint, { success ->
if (success) {
session.setBreakpointVerified(breakpoint)
}
}, { error ->
val errorMessage = (error.localizedMessage ?: error.message).nullize(nullizeSpaces = true)
?: DebuggerBundle.message("general.unknown-error")
logger.warn("Failed to set breakpoint $breakpoint.", error)
session.setBreakpointInvalid(breakpoint, errorMessage)
})
}
override fun unregisterBreakpoint(
breakpoint: XLineBreakpoint<XBreakpointProperties<*>>,
temporary: Boolean
) {
debugger.launchRemoveBreakpoint(breakpoint)
}
}
And finally, the implementation of the AutoHotKeyDebugProcess:
class AutoHotKeyDebugProcess(
private val session: XDebugSession,
private val debuggeeHandler: ProcessHandler,
private val debugger: AutoHotKeyDebugger
) : XDebugProcess(session), Disposable.Default {
init {
Disposer.register(this, debugger)
debugger.connectToSession(session)
debuggeeHandler.addProcessListener(object : ProcessListener {
override fun processWillTerminate(event: ProcessEvent, willBeDestroyed: Boolean) {
logger.info("[$processHandler] Will terminate.")
Disposer.dispose(this@AutoHotKeyDebugProcess)
}
override fun onTextAvailable(event: ProcessEvent, outputType: Key<*>) {
logger.trace { "[$processHandler] $outputType: ${event.text}" }
}
override fun processNotStarted() {
logger.warn("[$processHandler] Not started.")
}
override fun processTerminated(event: ProcessEvent) {
logger.info("[$processHandler] Terminated, exit code: ${event.exitCode}.")
}
})
}
override fun stop() {
Disposer.dispose(this)
}
override fun resume(context: XSuspendContext?) {
debugger.launchResumeExecution()
}
private val editorsProvider by lazy { AutoHotKeyDebuggerEditorsProvider() }
private val breakpointHandlers by lazy { arrayOf(AutoHotKeyBreakpointHandler(session, debugger)) }
override fun doGetProcessHandler(): ProcessHandler = debuggeeHandler
override fun getEditorsProvider(): XDebuggerEditorsProvider = editorsProvider
override fun getBreakpointHandlers(): Array<out XBreakpointHandler<*>> = breakpointHandlers
override fun createConsole(): ExecutionConsole {
return TerminalExecutionConsole(session.project, processHandler).also {
processHandler.startNotify()
}
}
override fun sessionInitialized() {
logger.info("Debug session initialized.")
debugger.launchResumeExecution()
super.sessionInitialized()
}
companion object {
private val logger = logger<AutoHotKeyDebugProcess>()
}
}
A particularly interesting thing is two places where we call dispose — it's okay to terminate a bit earlier in case the process under debug will get terminated soon.
Testing
Any good IDE feature should be accompanied by an appropriate number of automated tests. Having tests is very important! Even while prepating this example, I found a bug in my threading implementation thanks to tests. And IntelliJ provides facilities to organize testing for plugins, the main helpful API being XDebuggerTestUtil.
Here I'll show only the core test body and omit some auxiliary APIs I implemented; see the other details in the full example.
@TestApplication
class DebuggerTest {
@Test
fun testDebuggerStopsAtBreakpointOnExpectedLine() {
val file = copyAndOpenFile("debugger/script.ahk")
val oneBasedLine = 11 // currentMessage := "Iteration number: " . index
val zeroBasedLine = oneBasedLine - 1
XDebuggerTestUtil.toggleBreakpoint(project, file, zeroBasedLine)
val listener = createSessionListener()
val debugSession = startDebugSession(file.toNioPath(), listener)
try {
Assertions.assertTrue(
XDebuggerTestUtil.waitFor(
listener.paused,
timeout.inWholeMilliseconds
),
"Pause should be triggered within ${timeout.inWholeSeconds} seconds."
)
val suspendContext = debugSession.suspendContext as AutoHotKeySuspendContext
Assertions.assertEquals(zeroBasedLine, suspendContext.activeExecutionStack.topFrame?.sourcePosition?.line)
} finally {
debugSession.stop()
}
}
}
This test will create a temporary copy of debugger/script.ahk script from its test data on disk, set a brakpoint, run the program, and verify that the breakpoint has been hit, and that the suspend context reports the correct location after that.
Conclusion
As you can see, implementing a debugger for IntelliJ is not as convoluted as it seems, provided that you split the implementation into digestable parts. If you have some client API to begin with, it's not too hard to wrap it into XDebugger.
Again, see the full implementation of this whole example is the repository.