If you need your Node.js Lambda function to have support for any given extension, add that extension locally to have your local npm download the dependencies into node_modules, then zip the folder into an archive, and create a Lambda function from that archive.
You can do it from a Linux VPS, or download the ready-to-use archive below.
cd /tmp
mkdir ./lambda-nodejs-mysql
cd ./lambda-nodejs-mysql
npm init && npm install --save mysql
zip -D -r lambda-nodejs-mysql.zip .Here's the lambda-nodejs-mysql.zip I got hosted on OneDrive.