Blog Posts

My Year With Yoga

On Friday, January 6th 2017  I walked into the first Yoga class of my life at YogaSol  as part of fulfilling a new years resolution.
I was in the best shape of my life. I was running, swimming and lifting weights multiple times a week. I weighed 165 pounds and was at 9% body fat. I was also really stressed at work, my blood pressure had moved into the hypertension range and I felt like my life was stuck in permanent decision paralysis.
Mark was my teacher that day for a “Gentle Flow” class that lasted 60 minutes and was one of the hardest things I have ever done. The flow made me feel childish (What hand is my left?) and weak (Downward Dog is a resting position?) but more importantly it showed me I couldn’t turn off my brain for 5 minutes for shavasana  ( inhale- ‘Did I send that email?’  Exhale- ‘I should submit a CFP for Defcon this year.’).
I could also tell it would be one of the most fulfilling things I had ever done and stuck with it even when I didn’t want to. I now feel at home on the mat and practice yoga 4 or 5 times a week.

Here are a few things that have stuck with me this year:

Yoga taught me that if I am do not control of my life it controls me.

“Jerry, You didn’t breathe during that sun salutation.”
“…or that one.”
“…Are you trying to hold your breath?”
I really thought my teacher was being hard one me for no reason. I was doing what she asked, when she asked and how she asked. I soon came to understand that if I couldn’t pay mindful attention to the one thing that keeps me alive and control it I am not running my life. My life is running me.

Yoga taught me that more strength I have the less power I want.

I work in security and with that comes the need for power and control. We let this run our lives to the point that we have Brazilian Jiu Jitsu meet-ups at our biggest conferences (which I love).
Yoga quickly taught me that all the power in the world doesn’t matter if you don’t have strength. On Sunday’s I take an afternoon class with Lisa and she loves to say when we are in Downward Dog  “OK Class, we will be resting here for 5 breaths”.
I can bench press 100%+ of my weight multiple times so I thought I should be able to stay in Downward Dog for 60 seconds with no problem. I couldn’t, the 60 year old woman next to me could… and probably could have for 15 minutes.
It took me a few months to start building that strength and at the same time letting go of some of that unneeded power,

Yoga has taught me to disconnect.

I get up at 0415 most mornings and start working and don’t really stop working until I go to bed. I answer emails on  vacation. I write code on the weekends. I.ALWAYS.HAVE.MY.PHONE.
…except at Yoga class. It is the only time of the day that I am not sleeping that I don’t check my phone for an hour plus and the world doesn’t end. I need to get a lot better at this.

Yoga has taught me to slow down.

When I first started yoga I loved the high energy workouts and leaving yoga with the feeling of being totally exhausted. Then one day I took a yin class with Megan and realized that taking the time to slow down really makes me feel complete both mentally and physically.

Yoga has taught me the only way to improve your balance is to practice.

When I first I started yoga I could not do the tree pose at my ankle for 10 seconds without falling down. As I practiced over the last year I have gotten better but not perfect. As with most things in life you have to be willing to fail to get better.

If you ever get to LA check out YogaAqua.

Yoga has taught me to find comfort in discomfort.

If you want to be successful in life you can not stop when things get uncomfortable.  If you can stop yourself from going into Balasana because your arms are tired when your teacher decides in the middle of the downward dog during the fifth sunsultion is the best time to give the complete oral history of a sanskrit word you didn’t hear you can also send that difficult email, ask for a raise and have that hard discussion.
I found Yoga at the perfect time in my life and I look forward to learning the lessons it will have for me in 2018.

Automating Digicert Certificates Into AWS ACM

Like most security professionals I am spending a large amount of time helping my company move securely to AWS.
Certificate management in AWS is done with AWS Certificate Manager  and while they do offer *free* certificates, ACM generated certs are outside your direct control. You don’t get the keys which, at least for some things, should probably be a non-starter (granted, for plenty of other things it’s likely  ¯\_(ツ)_/¯).
I also really like digicert and have been using them for TLS certificates for over 10 years but I could not find any automation already built for Digicert to AWS ACM so I spent some time this week and hacked a script together to do it.
Here is a link to the script  (also embedded at the bottom of the post). On the host running the script you will need AWS CLI  configured and a Digicert API Key.  You also need to configure the first 15 lines of the script with your information.

To Run The Script:

./awasacm.sh your.fdqn.com

Script Output:

Here is what the script looks like running:

Here is the cert uploaded to ACM:

The script also saves all of the commands, keys and certs on the host running the script for auditing and backup:

Full Script:

https://gist.github.com/jgamblin/f8bd03d3743ba4f08f710d5e11c177c7

Closing:

I will be making improvements to this script as we implement it in production and will likely move it to a full GitHub repo soon.   If you have any questions please reach out to me on twitter at @JGamblin. 
Update:  I have built a full Github repo here.

Network Monitoring With Slack Alerting

Last November I hacked together a script that continually monitored your network and sent a slack alert when something change.   It worked but I was never 100% happy with it so I spent some time this weekend and rewrote it so that is hopefully more user friendly and functional. Some changes in this version includes the ability to set timeouts between scans, better output on the machine running the script, better logging and the start of a framework to add new tools.
All you need to run this project for yourself is a Ubuntu install with NMap, PripsSlackCLI and a copy of the script.
Once running here are what a slack alert looks like:

Here is what the script looks like running:

Here is a copy of the script:

#!/bin/bash -u
#
# Requires NMAP, NDIFF, PRIPS and Slackcli
# https://candrholdings.github.io/slack-cli/
# NETOWORKS should be the list of networks you want to monitor.
# INTERVAL how many seconds to wait between scans
# SLACKTOKEN from here https://api.slack.com/web
#
NETWORKS="192.168.0.0/24"
TARGETS=$(for NETWORK in ${NETWORKS}; do prips $NETWORK; done)
INTERVAL="1800"
SLACKTOKEN="Get This From https://api.slack.com/web"
OPTIONS='-T4 --open --exclude-ports 25'
cd  ~/scan
LAST_RUN_FILE='.lastrun'
while true; do
    # If the last run file exists, we should only sleep for the time
    # specified minus the time that's already elapsed.
    if [ -e "${LAST_RUN_FILE}" ]; then
        LAST_RUN_TS=$(date -r ${LAST_RUN_FILE} +%s)
        NOW_TS=$(date +%s)
        LAST_RUN_SECS=$(expr ${NOW_TS} - ${LAST_RUN_TS})
        SLEEP=$(expr ${INTERVAL} - ${LAST_RUN_SECS})
        if [ ${SLEEP} -gt 0 ]; then
            UNTIL_SECS=$(expr ${NOW_TS} + ${SLEEP})
            echo $(date) "- sleeping until" $(date --date="@${UNTIL_SECS}") "(${SLEEP}) seconds"
            sleep ${SLEEP}
        fi
    fi
    START_TIME=$(date +%s)
    echo ''
    echo '=================='
    echo ''
    DATE=`date +%Y-%m-%d_%H-%M-%S`
    for TARGET in ${TARGETS}; do
        CUR_LOG=scan-${TARGET/\//-}-${DATE}
        PREV_LOG=scan-${TARGET/\//-}-prev
        DIFF_LOG=scan-${TARGET/\//-}-diff
	echo ''
	echo $(date) "- starting ${TARGET}"
        # Scan the target
        nmap ${OPTIONS} ${TARGET} -oX ${CUR_LOG} >/dev/null
        # If there's a previous log, diff it
        if [ -e ${PREV_LOG} ]; then
            # Exclude the Nmap version and current date - the date always changes
            ndiff ${PREV_LOG} ${CUR_LOG} | egrep -v '^(\+|-)N' > ${DIFF_LOG}
            if [ -s ${DIFF_LOG} ]; then
			printf "Changes Detected, Sending to Slack."
			nmap -sV ${TARGET} | grep open | grep -v "#" > openports.txt
			slackcli -t $SLACKTOKEN -h nmap -m "Changes were detected on ${TARGET}. The following ports are now open: "
			sleep 1
			cat openports.txt | slackcli -t $SLACKTOKEN -h nmap -c
			rm openports.txt
                # Set the current nmap log file to reflect the last date changed
                ln -sf ${CUR_LOG} ${PREV_LOG}
            else
                # No changes so remove our current log
		printf "No Changes Detected."
                rm ${CUR_LOG}
            fi
            rm ${DIFF_LOG}
        else
            # Create the previous scan log
            ln -sf ${CUR_LOG} ${PREV_LOG}
        fi
    done
    touch ${LAST_RUN_FILE}
    END_TIME=$(date +%s)
    echo
    echo $(date) "- finished all targets in" $(expr ${END_TIME} - ${START_TIME}) "second(s)"
done

Some Quick Notes:

  • You will want to run this in screen so that it runs continually.
  • I excluded port 25 because it was reporting as “filtered” every other scan causing false alerts.
  • NDIFF really needs to be updated.  Its output is ridiculously bad.
  • Let me know on twitter if you have any questions.

Disallow Million Most Common Passwords

I was working on a project recently and was asked if it was possible to stop users from setting common passwords.   Using the pam_cracklib module and @DanielMiessler  common passwords list it is as simple as these 3 commands:

sudo apt-get install libpam-cracklib -y
sudo wget https://raw.githubusercontent.com/danielmiessler/SecLists/master/Passwords/10_million_password_list_top_1000000.txt /usr/share/dict/ -O /usr/share/dict/million.txt
sudo create-cracklib-dict /usr/share/dict/million.txt


Seriously,  that’s it.
Here is what a user will see when they attempt to use a password from the list: 

Getting Started With Mod_Security

 
Mod_Security is the most widely known and used server based Web Application Firewall but I had not had a chance to play with it so I decided to take sometime this weekend to build a website (modsec.handsonhacking.org) to test it.   Here is a small walk through on how I did it.

Base Server Install:

I used AWS Lightsail to build a webserver using Ubuntu 16.04,  Apache2,  LetsEncrypt , and this HTML5 Template.
Install and configure the website with these commands:

sudo apt update && sudo apt upgrade -y
sudo apt install apache2 git -y
sudo rm /var/www/html/index.html
sudo git clone https://github.com/themefisher/Blue-Onepage-HTML5-Business-Template.git /var/www/html/
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install python-certbot-apache
sudo certbot

Mod_Security Install

Install Mod_Security with these commands:

sudo apt-get install libapache2-modsecurity
sudo cp /etc/modsecurity/modsecurity.conf-recommended /etc/modsecurity/modsecurity.conf

Move from logging to blocking move with these commands:

sudo nano /etc/modsecurity/modsecurity.conf
# Change SecRuleEngine DetectionOnly
SecRuleEngine On

It should look like this:Install the updated OWASP ModSecurity Core Rule Set:

sudo rm -rf /usr/share/modsecurity-crs
sudo git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git /usr/share/modsecurity-crs

Enable them in the apache config file:

sudo nano /etc/apache2/mods-enabled/security2.conf
Add:
     IncludeOptional /usr/share/modsecurity-crs/*.conf
     IncludeOptional /usr/share/modsecurity-crs/rules/*.conf

It should look like this:
Move the OWASP rules from logging to blocking:

cd /usr/share/modsecurity-crs
sudo cp crs-setup.conf.example crs-setup.conf
sudo nano crs-setup.conf
Comment Out:
#SecDefaultAction "phase:1,log,auditlog,pass"
#SecDefaultAction "phase:2,log,auditlog,pass"
Uncomment:
SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"

It should look like this:

Next restart apache to enable mod_security:

sudo systemctl restart apache2

Testing

To test I used burp suite to scan modsec.handsonhacking.org to generate plenty of “bad traffic”.

Run this to see what is being blocked in real time:

sudo tail -f /var/log/apache2/modsec_audit.log

Next Steps

Now that I have mod_security running I need to find a better logging solution.   So far I have quickly looked at waf-fle and auditconsole but they both look to be abandoned.  It looks like there are people who are doing a lot with ELK but I have not found anything solid yet.  I am really surprised there isn’t a ready made Dashboard but I will keep looking.

Warning:

I have spent all of four hours playing with this on non-production traffic.  Please do not just install this in front of your website and then blame me when things break.

Closing:

Overall with the help of @infosecdad  and @lojikil guiding me through some of the places where documentation is lacking it was fairly easy to get this setup and going.   If you have any questions please reach out to me on twitter at @JGamblin. 

MAC Address Randomization for MacOS

One of the things that even the new MacOS beta is missing is MAC Address Randomization on boot.  After spending a few hours working on it I put together this completely hack-y solution that uses Spoof and an automator Script saved as an application.
Here is how I configured it:

on run {input, parameters}
	delay 4
	tell application "Terminal"
		activate
	end tell
	tell application "System Events"
		delay 0.3
		keystroke "sudo spoof randomize en0"
		keystroke return
		delay 0.5
		keystroke "#SADLYYOURPASSWORDHERE"
		keystroke return
		delay 5
	end tell
	tell application "Terminal" to quit
	return input
end run
  • Change “#SADLYYOURPASSWORDHERE” to your local password.
  • Test & Save:

  • Add to System Preferences -> Users & Groups -> Login items

Overall this is a pretty simple solution.  I dont love it because you have to save your local password in the script and I am looking for a way to change that but it looks like to change the MAC address you have to be root.  I will update this post if I figure out a way to remove the password.

My Security Summer Camp Talk List

Security summer camp is about a week away so I spent some time this afternoon trying to figure out what talks and events I want to make sure I attend.
BSides Las Vegas:
A Day in the Life of a Product Security Incident Response Manager
From SOC to CSIRT
Hadoop Safari : Hunting For Vulnerabilities
Introduction to Reversing and Pwning
YARA-as-a-Service (YaaS): Real-Time Serverless Malware Detection
Abusing Webhooks for Command and Control
BSides Las Vegas Full Schedule

Blackhat:

Breaking Electronic Door Locks Like You’re On CSI: Cyber
Free-Fall: Hacking Tesla From Wireless To Can Bus
Blackhat Full Sechedule

Defcon 25:

Meet the Feds (who care about security research)
There’s no place like 127.0.0.1 – Achieving reliable DNS rebinding in modern browsers
Wiping Out CSRF
Real-time RFID Cloning in the Field
Exploiting 0ld Mag-stripe information with New technology
Secret Tools: Learning About Government Surveillance Software You Can’t Ever See
Next-Generation Tor Onion Services
Using GPS Spoofing to Control Time
Cisco Catalyst Exploitation
Defcon Full Schedule

Other Events:

IOACTIVE IOASIS
ShabbatCon
Defcon Parties List
 

Run SSH and HTTPS On The Same Port

I recently saw this SSH/HTTP(S) multiplexer on Github and tweeted that it looked amazing:


A couple of people responded that you should be able to do the samething with HAProxy or something similar but my experience with HAProxy has been that is temperamental so I didn’t want to mess with it.  After some more research I found a tool called SSLH that did what I wanted so I built a demo site at  sshttps.jgamblin.com that is running SSH and HTTPS on port 443.

How To Build It Yourself:

To demo this I used a $5 Ubuntu AWS lightsail instance with a valid DNS record (sshttps.jgamblin.com)

Base Out The System:

These commands will update the system, install SSLH and Apache, and install a valid TLS certificate from LetsEncrypt:

sudo apt update && sudo apt upgrade
sudo apt install sslh build-essential apache2
wget https://dl.eff.org/certbot-auto
chmod a+x ./certbot-auto
./certbot-auto

Configure SSHL:

You need to edit the config so that <ETH0 IP> is the local (not public) IP:

sudo nano /etc/default/sslh
DAEMON_OPTS="--user sslh --listen <ETH0 IP>:443 --ssh 127.0.0.1:22 --ssl 127.0.0.1:443 --pidfile /var/run/sslh/sslh.pid"

Configure Apache:

You just need to change Listen *:443 to Listen 127.0.0.1:443

sudo nano /etc/apache2/ports.conf
<IfModule ssl_module>
        Listen 127.0.0.1:443
</IfModule>
<IfModule mod_gnutls.c>
        Listen 127.0.0.1:443
</IfModule>

Reboot and Enjoy:

You can probably restart services but a  sudo reboot works here and you are good to go.  If you visit with a web browser you get the page:

…*but* you can now ssh into the box on port 443 using ssh [email protected] -p 443

Closing Thoughts:

NMap only knows it is SSH if you use -sV:
I am looking forward to using this method in the future to stack services.  Let me know on twitter @jgamblin if you have any thoughts.

Quickly Building A Cloud Virtual Lab

Often while doing research I need temporary access to a bunch of different virtual machines. While it is possible to do this on my Macbook using VMWare Fusion or Virtualbox the overhead seems unnecessary for something I will delete in under a week.
My goto solution is a virtualization stack of:
16GB DigitalOcean Droplet + Wok + Kimchi
Here is the shell script I use to build it:

#!/bin/bash
apt-get update &&  apt-get upgrade -y
apt-get -y install qemu qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils nginx python-cherrypy3 python-jsonschema python-m2crypto nginx python-ldap python-psutil fonts-font-awesome texlive-fonts-extra python-configobj python-parted sosreport python-imaging websockify novnc nfs-common python-ethtool open-iscsi python-guestfs libguestfs-tools spice-html5 python-paramiko
wget http://kimchi-project.github.io/kimchi/downloads/latest/kimchi.noarch.deb
wget http://kimchi-project.github.io/wok/downloads/latest/wok.noarch.deb
wget http://kimchi-project.github.io/gingerbase/downloads/latest/ginger-base.noarch.deb
dpkg -i wok.noarch.deb
apt-get install -f -y
dpkg -i ginger-base.noarch.deb
apt-get install -f -y
dpkg -i kimchi.noarch.deb
apt-get install -f -y
reboot
#You will need to know the root password for the web interface (passwd lets you reset it).

After the server is rebooted you can access the web interface at https://ip:8001:

The next step is to add the templates you want to build VMs for:

You can use these commands to grab newer isos (there is a feature request to automate this):

cd /var/lib/kimchi/isos
wget -c http://cdimage.kali.org/kali-2017.1/kali-linux-2017.1-amd64.iso
wget -c http://releases.ubuntu.com/17.04/ubuntu-17.04-desktop-amd64.iso
wget -c http://releases.ubuntu.com/17.04/ubuntu-17.04-server-amd64.iso
wget -c http://releases.ubuntu.com/16.04/ubuntu-16.04.2-desktop-amd64.iso
wget -c http://releases.ubuntu.com/16.04/ubuntu-16.04.2-server-amd64.iso
wget -c ftp://opensuse.mirrors.ovh.net/opensuse/distribution/13.2/iso/openSUSE-13.2-DVD-x86_64.iso
wget -c http://slackware.mirrors.ovh.net/ftp.slackware.com/slackware64-14.2-iso/slackware64-14.2-install-dvd.iso
wget -c http://archlinux.mirrors.ovh.net/archlinux/iso/2016.09.03/archlinux-2016.09.03-dual.iso
wget -c https://download.fedoraproject.org/pub/fedora/linux/releases/25/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-25-1.3.iso
wget -c https://az792536.vo.msecnd.net/vms/VMBuild_20150801/VirtualBox/MSEdge/Windows/Microsoft%20Edge.Win10.For.Windows.VirtualBox.zip

Once you are done with that is is amazingly easy to spin up VMs and manage them in the browser:

I use this virtualization stack a lot in my research and it is amazing.  If you have any questions feel free to reach out to me on twitter.

Reminder: Operational Security Is Hard

I love OWASP  (I wanted to get that out of the way) but they let their TLS certificate expire yesterday:


Should it have happened to an organization whose whole goal is to secure web applications?

No.

There are a million reasons why their TLS certificate could have expired and plenty of reasons it shouldn’t have  (OWASP uses letsencrypt for their TLS certificate which can automatically renew certificates and sends you email when they are close to expiring).
Is it forgivable?
Yes.
Expired certificates,  missing patches and unknown cloud services haunt every security organization. Some people look at these things as *easy* to fix and if you miss them you dont care about security… most of those people have usually never worked in operational security.
Why did it happen?
Operational Security Is Hard.

Being perfect is impossible.   Stephen Curry (Arguably the best shooter in the NBA) only makes 90% on his free throws.  So everyone is going to miss a patch, let a certificate expire and have unknown cloud services.  It.Is.Going.To.Happen.
What can we learn from this?
A lot. 
How would your organization have handled this on Saturday morning?  Would you have been able to update your certificate in an hour on a Saturday morning?    If you know the answer to those questions you can pick a tweet from @badthingsdaily and work through it with your team.
Let me know your thoughts on twitter.

Site Footer