Secure Software Development using CI

Part II - Web Application Scanning / DAST

Posted on July 17, 2018

Introduction

The [last post] (/blog/securecode/deps/en) showed how we can find and fix insecure dependencies in a project. In addition to the problem of dependencies issues to fix, which can be present in a large amount in a project, there is another way to harden your own code.

In a running application on the Internet, there is a “noise floor”. That means, any application that presents itself in public is exposed to certain attacks after some time. This can come from botnets, but also people who want to “test”, whether you can get something from the page through hacking.

In order to arm your own application against such attacks before the release, there are DAST systems, short for ‘Dynamic Application Security Testing’. Here, a tool tries to execute different attack patterns from the outside with different inputs and then evaluates the results for potential vulnerabilities. In principle, this is exactly what the later malicious attacker or botnet is trying to use against the application. This article differs from the last one in that it does not analyze code, but unleashes a “real” attacker and possible attack requests on the own application.

A popular DAST system for doing this is the open source tool OWASP ZAP.

The following describes how to integrate OWASP ZAP into Jenkins. Many descriptions are based on the great documentation Jenkins at your Service! Integrating ZAP in Continuous Delivery:

The individual sections are divided into:

The steps described here have been performed on a Mac OS X. This applies in particular to the chapter where OWASP ZAP is installed as a desktop application. You can also perform the steps on another operating system. However, for this you have to look up the corresponding command line equivalents in another OS.

Installing OWASP ZAP in Jenkins

Stopping old docker container

Those who follow from the old chapter: For the following explanations, we do not need the git server anymore. We can stop it with:

docker stop <containerid_gitserver>

Installation

Next we install OWASP ZAP in Jenkins.

Installation of necessary plugins

For installation, we use the Docker infrastructure, which we described in the last chapter.

Next we need:

  • the official ‘OWASP ZAP Plugin’, which we will use for the scans against our application
  • the ‘Custom Tool Plugin’ - for easy installation and
  • the ‘HTML Publisher Plugin’, to view the results in the builds.

To do this, follow these steps:

owaspzapinstallation1 owaspzapinstallation2 owaspzapinstallation3 owaspzapinstallation4 owaspzapinstallation5

Click on ‘Official OWASP ZAP’:

owaspzapinstallation7

Also choose ‘Custom Tool’. We can do that via the filter field in the top right corner. Afterwards we click on ‘Download now and install after restart’ and ‘Restart Jenkins when installation is complete and no jobs are running’:

customtoolsinstallation1 customtoolsinstallation2 customtoolsinstallation3 customtoolsinstallation4

We now go to the standard Jenkins page:

owaspzapinstallation9 owaspzapinstallation10

After some time, the login screen of Jenkins should reappear and you can log in.

loginafterrestart

Configuring Custom Tools

We are looking for the installation package ‘OWASP ZAP 2.7.0’ for Linux:

On the GitHub release 2.7.0 page you can find the link to the linux release: https://github.com/zaproxy/zaproxy/releases/download/2.7.0/ZAP_2.7.0_Linux.tar.gz

And set it as an installation source:

installowaspzapbycustomtools1 installowaspzapbycustomtools2 installowaspzapbycustomtools3 installowaspzapbycustomtools4

As input under ‘custom tool’ we use:

  • ‘custom tool’
  • name : ZAP_2.7.0
  • install automatically
  • download URL for binary archive: https://github.com/zaproxy/zaproxy/releases/download/2.7.0/ZAP_2.7.0_Linux.tar.gz
  • subdirectory of extracted archive: ZAP_2.7.0

installowaspzapbycustomtools5

and finally click on ‘Save’.

Configuring the port for ZAP

managejenkins configuresystem

Under ‘ZAP’:

We set the port to something rare, like 12123:

  • default host: localhost
  • default port: 12123

configowaspzapport

‘Save’

Configuring a ZAP job

Now we create our first ‘OWASP ZAP scan job’:

newitem

Enter ‘zap_scan_demo’ and click ‘Ok’.

zapscandemo1 zapscandemo2 zapscandemo3 zapscandemo4

Now for ‘Tool Selection’: ZAP_2.7.0.

zapscandemo6 zapscandemo7 zapscandemo8 zapscandemo9 zapscandemo10

We deliver where we want OWASP ZAP to be installed:

  • path: /var/jenkins_home/owaspzap
  • persist session: owasp_webgoat_zap_session

zapscandemo11

Configuring the target URL

Next, we need to specify the target URL, the URL we want to attack.

For this we need the following information:

  • session properties
  • include in context: which URLs are in the test frame? We do not want to scan URLs outside this and which may be liable to prosecution.
  • authentification: Here you can set how the scanner can log in to the application and how it notices this. Any application with a login that you want to test inside needs this setting.
  • attack mode
  • starting point: The address from which we want to start the scan.
  • spider scan: Here we can choose if the scanner tries to find further pages using existing links in the root page. This makes sense, as we don’t want to test just one page.
  • active scan: Here we can select the scan policy. As long as we haven’t created one yet, the default policy of OWASP ZAP will be taken.
  • finalize run: Here you can define how the report should be generated.

Using a small sample application that is vulnerable to XSS, I’ll first explain how we can configure each item. Then I’ll go into the topic of authentication in applications and how to set them in the scanner. With the shown it should be possible to configure a larger project at the end.

Configuring OWASP ZAP

Installation

To make the settings easier, we install the original application of OWASP ZAP locally.

To be able to understand everything better we use a fixed version : 2.7.0. Therefore we use a specific git hash in the URL to install specifically version 2.7.0:

brew cask install https://raw.githubusercontent.com/caskroom/homebrew-cask/645dbb8228ec2f1f217ed1431e188687aac13ca5/Casks/owasp-zap.rb

installowaspzapbrewcask

You can now launch the application from your Mac by typing OWASP ZAP into Spotlight:

startowaspzap

Now click ‘No, I do not want to persist this session at this moment in time’:

notpersistingsessionowaspzap

Then select ‘Start’:

notpersistingsessionowaspzap2

Configuring Proxy ZAP

We click on the little wheel:

preferencesowaspzap1

and set the following data for ‘Local Proxies’:

  • address: 127.0.0.1
  • port: 9000

and save with Ok.

Configuring FoxyProxy

In the browser we configure a proxy for localhost:9000.

For that, you can use the extension FoxyProxy in Firefox (under Add-ons) as well as in Chrome (under Extensions).

First we install the extension.

foxyproxyfirefox

Then we open it by clicking on the icon at the top right of the browser:

foxyproxyconfig1

And set this:

foxyproxyconfig2 foxyproxyconfig3

  • Proxy Type: HTTP
  • Title or Description (optional): OWASP ZAP 127.0.0.1:9000
  • IP address, DNS name, server name: 127.0.0.1
  • Port: 9000

foxyproxyconfig4

Save

foxyproxyconfig5

Now we can select the proxy:

foxyproxyconfig1 foxyproxyconfig6

The icon changes to:

foxyproxyconfig7

Now the traffic runs through the proxy.

Example application XSS

We launch an application that is vulnerable to XSS:

docker run -ti --rm -p 127.0.0.1:1185:1185 \
-d secf00tprint/victim_easy_xss_server

When we open the application we see text input fields for comments. Here you can inject code:

open http://127.0.0.1:1185

victim_easy_xss1

We determine the IP on the docker network:

docker inspect <containerid_victim_easy_xss_server>|grep "IPA"

Jenkins Session Properties

Then we enter under ‘Session Properties’:

  • Context Name: zap_scan_demo
  • Include in Context: http://&#x3C;determined_IP&#x3E;:1185/.*, e.g. http://172.17.0.4:1185/.*

sessionpropertiessimplexss_owaspzap

Under ‘Attack Mode’ we enter the root URL:

http://<determined_IP>:1185/ 

e.g.

http://172.17.0.4:1185/ 

sessionpropertiessimplexss_owaspzap2

‘Save’

Then we build the item for the first time.

‘Build Now’

buildnow

sessionpropertiessimplexss_owaspzap3

Then ZAP should be installed in the folder /var/jenkins_home/owaspzap.

Checking the vulnerability in the Example Application

If we start our proxy and call http://127.0.0.1:1185 in the browser,

foxyproxyconfig7 xss_easy_vuln1

this should appear in OWASP ZAP:

xss_easy_vuln2a xss_easy_vuln2b

We now go back to the browser and enter the following in the GET-field:

  • Comment (using GET): Test

‘Show’

Result:

The URL shows http://127.0.0.1:1185/?comment=Test&enter_comment=Show and the comment appears:

xss_easy_vuln3

If we click on ‘Back’, enter

<script>alert(1)</script>

x and click on ‘Save’, we see a pop-up:

xss_easy_vuln4

This means the application is vulnerable to XSS.

OWASP ZAP should find this. Let’s take a closer look now:

Configuring Scan Policy

We can use the icon with the mixer to set how we want to scan:

owaspzapscanpolicy1

First we click on ‘Add’

owaspzapscanpolicy2

and select the following values:

  • Scan policy
  • Policy: XSS
  • Default alert threshold: Medium
  • Default attack strength: Low
  • Information gathering: Threshold: OFF, Strength: Default
  • Server security: Threshold: OFF, Strength: Default

owaspzapscanpolicy3

Under ‘Injection’ we set everything to ‘Threshold’: OFF and Strength: Default, except entries with cross-site scripting. These we set to Low:

owaspzapscanpolicy4

‘Miscellaneous’, ‘External Redirect’, ‘Threshold’ we switch to OFF, Strength to Default and ‘Script Active Scan Rules’ to Low, Default:

owaspzapscanpolicy5

‘Ok’

Then ‘XSS’ and ‘Export’ and save the file as ‘XSS.policy’:

owaspzapscanpolicy6 owaspzapscanpolicy7 owaspzapscanpolicy8

‘Save’

If we look at the file, we see that it is an XML file that adheres to a certain format.

owaspzapscanpolicy8b

The individual entries can be looked up:

Policy abbreviations for active and passive scans

owaspzapscanpolicy9

We remember the location of the file where we put it, because we need it afterwards for Jenkins.

XSS Scan

We now look in the ‘History’ of OWASP ZAP for the request in which we entered ‘Test’:

xss_easy_vuln5

and select ‘Attack’, ‘Active Scan’:

xss_easy_vuln6

Under ‘Policy’ we select ‘XSS’:

xss_easy_vuln7

and click on ‘Start Scan’.

xss_easy_vuln8

If we now click on ‘Alerts’

xss_easy_vuln9

we can see that an XSS has been found:

xss_easy_vuln10

We also want to reproduce this finding in Jenkins.

Jenkins Attack Mode

First, we need to copy the policy in Jenkins. To do this, type the following command on the command line in the directory where XSS.policy was placed:

docker cp XSS.policy <containerid_jenkins>:/var/jenkins_home/owaspzap/policies/

If we now open the configuration page OWASP ZAP in Jenkins we can select the policy.

Attack Mode:

  • Starting point: http://&#x3C;determined_IP&#x3E;:1185/?comment=test&#x26;enter_comment=Show e.g. http://172.17.0.3:1185/?comment=test&enter_comment=Show
  • Spider Scan: True
  • Recurse: True
  • Subtree Only: Max Children to Crawl: 2
  • Active Scan
  • Policy: XSS wählen

owaspzapattackmode1

Jenkins Report

To generate a corresponding report,

we set ‘Finalize Run’:

  • Generate Reports: True
  • Clean Workspace Reports: True
  • Filename: JENKINS_ZAP_VULNERABILITY_REPORT_${BUILD_ID}
  • Generate Report: True
  • Format: choose xml and html

owaspzapreport1

Under ‘Add post-build action’, ‘Archive the artifacts’:

owaspzapreport2

  • Archive the artifacts
  • Files to archive: logs/*,reports/*

owaspzapreport3

and ‘Add post-build action’, ‘Publish HTML Reports’:

owaspzapreport4

‘Add’

owaspzapreport5

  • Publish HTML reports: Reports
  • HTML directory to archive: reports/
  • Index page[s]: JENKINS_ZAP_VULNERABILITY_REPORT_${BUILD_ID}
  • Report title: ZAP Scan Demo

owaspzapreport6

and finally we click on ‘Save’.

Final Scan

To start the final scan we select ‘Build now’:

buildnow

The performed scan should find the XSS:

owaspzapscandemofinal1 owaspzapscandemofinal2 owaspzapscandemofinal3

Authentication

In the following section I will explain how to set up authentication. To make it clearer I take 2 applications: OWASP WebGoat and OWASP Juice Shop:

OWASP Juice Shop

To start this application we type in the following command on the command line

docker run --rm -p 127.0.0.1:3000:3000 -d bkimminich/juice-shop

and open the started application in the browser:

open http://127.0.0.1:3000

The proxy for OWASP ZAP should be set in the browser (cf. chapter Configuring FoxyProxy).

Now we go to the login mask:

juicyshop1 juicyshop2

We’ll sign up with:

  • User: ' or 1=1;--
  • Password: anything

In the ‘History’ in OWASP ZAP we should see a POST:

juicyshop3

Username / Password Parameter

The OWASP-ZAP plugin in Jenkins needs for a successful authentication which parameters are used for the login:

juicyshop3b

In this case, you can determine them from the POST request:

  • {"email":"' or 1=1;--","password":"p"}

juicyshop3a

We can put that down in Jenkins:

juicyshop3c

We can also take the login credentials

  • Username: ' or 1=1;--
  • Password: beliebig

juicyshop3d

and the login URL from ZAP:

http://127.0.0.1:3000/rest/user/login

juicyshop3e

Whereby we still have to exchange the IP with the IP which is present in the Docker network, e.g.:

http://172.17.0.3:3000/rest/user/login

We’ll put that in Jenkins:

juicyshop3f

Logged-in String

To determine when the user is logged in, a logged-in string must be specified in Jenkins:

juicyshop4

If you click on the question mark on the right you will see how it is determined:

The Logged in indicator, when present in a response message (either the header or the body), signifies that the response message corresponds to an authenticated request.

juicyshop5

Therefore we look at the response in OWASP ZAP for a successful login:

juicyshop5b juicyshop6

And save this response under the name response.raw.out:

Right mouse button, ‘Save raw’, ‘Response’, ‘All’:

juicyshop6aa juicyshop6ab juiceshop8

Now we log out of Juicy Shop

juicyshop6ac

and try to log in with wrong credentials.

We use

  • User name: test
  • Password: test

juicyshop6ad

And search the POST request in ZAP:

juicyshop6a

Then we click on Response:

juicyshop6b

And save this one:

Right mouse button, ‘Save raw’:

juicyshop6d

‘Response’, ‘All’:

juicyshop6e

We’ll take another text file:

  • Save As: response2.out
  • File Format: Raw

juiceshop7 juiceshop8

If we now compare the two responses (e.g. using vim tools vimdiff response.out.raw response2.out.raw),

juicyshop9

we see that a login can be recognized by the occurrence of the string authentication in the response.

That means we set in Jenkins the following Reg-Ex for the Logged-In Indicator:

  • .*\Qauthentication\E.*

juicyshop10

Logged-out String

To determine when we will be logged out, we look at which string can only be found on login page. To do this, hover the mouse over ‘Login’ in the browser, press the right mouse button and examine the element:

juicyshoploggedout1

We use the string TITLE_LOGIN

juicyshoploggedout2

and put it in Jenkins:

  • .*\QTITLE_LOGIN\E.*

juicyshoploggedout3

Missing data

To complete our scan we enter the following parameters:

  • Session Properties
  • Include in Context: http://&#x3C;determined_IP&#x3E;:3000/.* e.g. http://172.17.0.3:3000/.*

juicyshopaddmissingdata1

Authentication should look like this:

juicyshopaddmissingdata2

For the ‘Attack Mode’ we set:

  • Starting Point: http://&#x3C;determined_IP&#x3E;:3000/, e.g. http://172.17.0.3:3000/. Attention: It is important to set the final slash, otherwise the spider and scanner may not work properly.
  • Spider Scan
  • Recurse: True
  • Subtree Only: True
  • Max Children To Crawl: 2
  • Active Scan
  • Policy: Default Policy, which scans with MEDIUM strength
  • Recurse: True

juicyshopaddmissingdata3

Finalize Run’ and the Post-Build actions we leave as usual:

juicyshopaddmissingdata4 juicyshopaddmissingdata5

Start Scan

Now we can start the build / scan. This will take some time.

buildnow scanowaspzapjuicyshop

OWASP Web Goat

Another application we will use to illustrate authentication is OWASP WebGoat.

We’ll launch it with you:

docker run -p 127.0.0.1:10394:8080 -it webgoat/webgoat-8.0 /home/webgoat/start.sh
open http://127.0.0.1:10394/WebGoat/

Creating a user

First we have to register a user:

webgoat_registernewuser1

We use

  • Username: webgoat
  • Password: webgoat

accept the terms of service and click on ‘Sign up’:

webgoat_registernewuser2

Logged-in / Logged-out Indicators

When calling the URL http://127.0.0.1:10394/WebGoat/ we are redirected to http://127.0.0.1:10394/WebGoat/login.

If we now log in with the credentials, have OWASP ZAP recorded, and take a closer look at requests and responses for a successful and unsuccessful login, we see:

  • Login-Parameter (POST): username=webgoat&password=webgoat
  • Username Parameter: username
  • Password Parameter: password
  • Login-URL: http://172.17.0.5:8080/WebGoat/login, die IP und Port sind diejenigen im Dockernetz. Beides kann über docker inspect <containerid_webgoat>|grep "IPA" (IP) bzw aus dem obigen docker-Aufruf ausgelesen werden (Port).
  • Username: webgoat
  • Password: webgoat

For a successful login:

webgoat_loginlogoutindicators1 webgoat_loginlogoutindicator1b

For a non-successful login:

webgoat_loginlogoutindicators2 webgoat_loginlogoutindicators2b

In the header the location is different:

webgoat_loginlogoutindicators3

This is what we use for the Logged-In Indicator:

.*\QLocation: http://172.17.0.5:8080/WebGoat/welcome.mvc\E.*

webgoat_loginlogoutindicators4

For the Logged-out-Indicator we take the URL of the login page:

.*\Qhttp://127.0.0.5:8080/WebGoat/login\E.*

If this is in one of the responses, we assume that the system has logged out the user.

Final setting WebGoat

The final setting for the scan then looks as follows:

owaspzapwebgoatfinal2 owaspzapwebgoatfinal1 owaspzapwebgoatfinal3 owaspzapwebgoatfinal4 owaspzapwebgoatfinal5

Best Practices

Due to the duration and traffic such a scan can take, it is recommended to run it only once overnight or once a week.

A daily scan can be set via ‘Build Triggers’:

owaspzapjenkinsbestpractices1

Summary

This article summarizes how to install and use OWASP ZAP in the Continuous Integration System Jenkins.