Debugging Dataform with Node.js in VScode

Debugging Dataform JavaScript

If you happen to be a data engineer who is not familiar with the Node.js ecosystem, and end up having to debug complex JavaScript in a BigQuery Dataform code base might be feeling lost.

I will try to explain how I debug JavaScript and maybe you will find this helpful.

Run and Debug in VSCode

Let’s start by clicking on the Run and Debug tab in VScode and opening a JavaScirpt Debug Terminal.

When you click on this you will get a second terminal. This will be below any other terminals you may already have open.

Now, we can add debugger; statements within our JavaScript code for breakpoints.

In Dataform, we then need to run a compile within the Debug Terminal. We also need to pass the timeout option to a high time (otherwise it will error out before we drop into debug mode).

dataform compile --timeout 10m

Now we can inspect the state of variables, step through lines of code and use the Debug Console to test lines of JavaScript code.

The above screenshot of VSCode debugger shows the debugger actions at the top, highlights the current line of code, and the state of all variables in different context (global, local etc). Down the bottom we have a console where we can type in JavaScript commands to test them.

Try cloning this example repo and testing it out for yourself. I have left the debugger; statement in this SQLX file.

config {
  type: "table",
  name: "debug_example",
  schema: dataform.projectConfig.vars.ref,
}

js {
    // Place this where you need to debug
    debugger;
    const simple_sql = `select 'hello world' as test_column`
}

${ simple_sql }