Capturing Regions with Org Capture

28. February 2018

I’m an avid user of org-capture for recording TODOs, code snippets, and reference material without interrupting my train of thought. A quick C-c c t lets me rattle off a note-to-self while capturing a link back to the buffer I’m in for later reference.

I recently found myself frequently repeating the pattern of calling kill-ring-save on a region to yank it into my org-capture buffer. By taking advantage of org-capture-templates and a little elisp, this pattern can be reduced to simply highlighting the target region before invoking capture.

read more

Remote Debugging Spark Jobs

11. January 2018

As we move past kicking the tires of Spark, we’re finding quite a few areas where the documentation doesn’t quite cover the scenarios you’ll run into day-after-day in a production environment. Today’s exemplar? Debugging jobs running in our staging cluster.

While taking a functional approach to data transformations allows us to write easily testable and composable code, we don’t claim to be perfect, and being able to set breakpoints and inspect values at runtime are invaluable tools in a programmer’s arsenal when println just won’t do.

The what of debugging JVM application in production is simple, and among the most powerful (and impressive!) capabilities of the Java platform. The Java Debug Wire Protocol (JDWP) enables attaching to a remote server as if it were on your machine. Just launch the JVM for your Spark driver with debug flags and attach. Easy, right?

Not quite.

read more