Day23，替你的Gitlab pipeline 添加点搞事

正文

在前面介绍gitlab-ci的pipeline中我仅仅只用到了build stage作为container image build的动作。

然而在实务上，我们可能还会有许多需要整合的stages单元测试、整合测试、各种布署细节、各种通报细节以及与gitlab issue、dashboard整合的动作。

这边我就准备来搞三个官方有在主导的安全项目，不过不是完全以官方的方式去进行，因为部分的功能在介面上需要enterpirse版本才能够支援，但不代表无法使用他，只是说我们无法在官方的介面上看到对应的功能画面。

三个动作则分别是 SAST、Container Scanning、DAST

SAST - Static Application Security Testing

静态程序码扫描，这个部分是完全开放的，也就是说无论是在CE/EE的gitlab版本中皆可以从介面上直接套用他，而他支援的语言扫描也非常多种，可以直接参考连结，而写法上大概就像这样，修改.gitlab-ci.yaml

stages:
- sast
sast:
  variables:
    SAST_EXCLUDED_ANALYZERS: bandit, brakeman, eslint, flawfinder, gosec, kubesec,
  nodejs-scan, phpcs-security-audit, pmd-apex, semgrep, sobelow, spotbugs
  stage: sast    
include:
- template: Security/SAST.gitlab-ci.yml

相关的variables都可以再去参考官方

输出的json则是像这样(特地找一个有vulnerability的)，最终都可以搭配jq做操作。

{
  "version": "14.0.0",
  "vulnerabilities": [
    {
      "id": "65f3cc30cdd0fea1d39c9f7b3300112aa84194829d19c8288038473877b1e549",
      "category": "sast",
      "name": "Weak random generator",
      "message": "Weak random generator",
      "cve": "sast-test/Controllers/WeatherForecastController.cs:33:SCS0005",
      "scanner": {
        "id": "security_code_scan",
        "name": "Security Code Scan"
      },
      "location": {
        "file": "sast-test/Controllers/WeatherForecastController.cs",
        "start_line": 33
      },
      "identifiers": [
        {
          "type": "security_code_scan_rule_id",
          "name": "SCS0005",
          "value": "SCS0005",
          "url": "https://security-code-scan.github.io/#SCS0005"
        }
      ]
    },
    {
      "id": "1a69e74a7f4a5242b1ff87276f31f151f8a1a7aacc27106a2991de3a0a7ca28e",
      "category": "sast",
      "name": "Weak random generator",
      "message": "Weak random generator",
      "cve": "sast-test/Controllers/WeatherForecastController.cs:34:SCS0005",
      "scanner": {
        "id": "security_code_scan",
        "name": "Security Code Scan"
      },
      "location": {
        "file": "sast-test/Controllers/WeatherForecastController.cs",
        "start_line": 34
      },
      "identifiers": [
        {
          "type": "security_code_scan_rule_id",
          "name": "SCS0005",
          "value": "SCS0005",
          "url": "https://security-code-scan.github.io/#SCS0005"
        }
      ]
    }
  ],
  "remediations": [],
  "scan": {
    "scanner": {
      "id": "security_code_scan",
      "name": "Security Code Scan",
      "url": "https://security-code-scan.github.io",
      "vendor": {
        "name": "GitLab"
      },
      "version": "3.5.3"
    },
    "type": "sast",
    "start_time": "2021-09-08T07:09:20",
    "end_time": "2021-09-08T07:09:38",
    "status": "success"
  }
}

Container Scanning

这边就是针对container的扫瞄，gitlab官方则是基於trivy、grype来使用，而因为我日常在使用的gitlab为CE版本这个部分就必须自己填上，会直接使用trivy的source image来做扫描，举例如下

stages:
- container_scan
image-scanner:
  image: 
    name: aquasec/trivy:latest
    entrypoint: [""]
  stage: container_scan
  before_script:
  - apk add --no-cache docker openrc 
  - rc-update add docker boot
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
  - |
    if [[ "$CI_COMMIT_BRANCH" == "master" ]]; then
      tag=":dev"
    else
      tag=":$CI_COMMIT_REF_NAME"
    fi
  - trivy image -f json -o report.json $CI_REGISTRY_IMAGE${tag}
  artifacts:
    paths: 
    - report.json    
  tags:
  - docker

trivy还有相关多的指令能使用，如也能够搭配 trivy image --exit-code 1 之类的来让你的pipeline fail，也能够做针对部分语言做package scanner的动作，有兴趣可以去阅读官方文件

一样会是json的输出，也可以搭配jq操作

[
  {
    "Target": "registry.gitlab.com/gurubear-ithome-13th/homelabapi:dev (debian 10.10)",
    "Class": "os-pkgs",
    "Type": "debian",
    "Vulnerabilities": [
      {
        "VulnerabilityID": "CVE-2011-3374",
        "PkgName": "apt",
        "InstalledVersion": "1.8.2.3",
        "Layer": {
          "Digest": "sha256:a330b6cecb98cd2425fd25fce36669073f593b3176b4ee14731e48c05d678cdd",
          "DiffID": "sha256:d000633a56813933cb0ac5ee3246cf7a4c0205db6290018a169d7cb096581046"
        },
        "SeveritySource": "nvd",
        "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2011-3374",
        "Description": "It was found that apt-key in apt, all versions, do not correctly validate gpg keys with the master keyring, leading to a potential man-in-the-middle attack.",
        "Severity": "LOW",
        "CweIDs": [
          "CWE-347"
        ],
        "CVSS": {
          "nvd": {
            "V2Vector": "AV:N/AC:M/Au:N/C:N/I:P/A:N",
            "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:L/A:N",
            "V2Score": 4.3,
            "V3Score": 3.7
          }
        },
        "References": [
          "https://access.redhat.com/security/cve/cve-2011-3374",
          "https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=642480",
          "https://people.canonical.com/~ubuntu-security/cve/2011/CVE-2011-3374.html",
          "https://seclists.org/fulldisclosure/2011/Sep/221",
          "https://security-tracker.debian.org/tracker/CVE-2011-3374",
          "https://snyk.io/vuln/SNYK-LINUX-APT-116518",
          "https://ubuntu.com/security/CVE-2011-3374"
        ],
        "PublishedDate": "2019-11-26T00:15:00Z",
        "LastModifiedDate": "2021-02-09T16:08:00Z"
      },
      .
      .
      . 大量略过
      .
      {
        "VulnerabilityID": "CVE-2021-37600",
        "PkgName": "util-linux",
        "InstalledVersion": "2.33.1-0.1",
        "Layer": {
          "Digest": "sha256:a330b6cecb98cd2425fd25fce36669073f593b3176b4ee14731e48c05d678cdd",
          "DiffID": "sha256:d000633a56813933cb0ac5ee3246cf7a4c0205db6290018a169d7cb096581046"
        },
        "SeveritySource": "nvd",
        "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2021-37600",
        "Title": "util-linux: integer overflow can lead to buffer overflow in get_sem_elements() in sys-utils/ipcutils.c",
        "Description": "An integer overflow in util-linux through 2.37.1 can potentially cause a buffer overflow if an attacker were able to use system resources in a way that leads to a large number in the /proc/sysvipc/sem file.",
        "Severity": "MEDIUM",
        "CweIDs": [
          "CWE-190"
        ],
        "CVSS": {
          "nvd": {
            "V2Vector": "AV:L/AC:H/Au:N/C:N/I:N/A:P",
            "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H",
            "V2Score": 1.2,
            "V3Score": 5.5
          },
          "redhat": {
            "V3Vector": "CVSS:3.1/AV:L/AC:H/PR:N/UI:R/S:U/C:N/I:N/A:H",
            "V3Score": 4.7
          }
        },
        "References": [
          "https://github.com/karelzak/util-linux/commit/1c9143d0c1f979c3daf10e1c37b5b1e916c22a1c",
          "https://github.com/karelzak/util-linux/issues/1395",
          "https://security.netapp.com/advisory/ntap-20210902-0002/"
        ],
        "PublishedDate": "2021-07-30T14:15:00Z",
        "LastModifiedDate": "2021-09-02T09:15:00Z"
      }
    ]
  }
]

实务上使用可能会要加上 --ignore-unfixed，不然可能就会像我这样多到炸裂~

DAST - Dynamic Application Security Testing

这边也是使用gitlab官方包好的image，不过非EE版本不支援从介面直接新增，所以我们需要去看他的open source并将加入到自己的pipeline当中，而这边的DAST则是基於ZAP这套开源扫描工具，我们撰写的方式也很简单

stages:
- dast
dast:
  stage: dast
  image:
    name: "$SECURE_ANALYZERS_PREFIX/dast:$DAST_VERSION"
  variables:
    GIT_STRATEGY: none
    DAST_VERSION: 1
    DAST_MARKDOWN_REPORT: report.md
    SECURE_ANALYZERS_PREFIX: "registry.gitlab.com/gitlab-org/security-products/analyzers"
  allow_failure: true
  script:
    - export DAST_WEBSITE="https://homelab.gurubear.info/"
    - /analyze
  artifacts:
    name: "dast-report"
    paths:
      - gl-dast-report.json
      - $DAST_MARKDOWN_REPORT

这边一样有大量变数可以参考官方设定，可以输出的有json、html、markdown

这边我就不输出json改输出markdown看看了~以下截图

本次使用的repo pipeline

以上就是不透过官方正规管道产出的3种扫描方式，如果有EE版本授权、Ultimate subscription的话，还是建议照着官方来，也能够配合搭配dashboard使用，相信能够更有效率。

闲聊

这一两年来资安的议题非常热络，市面也是充斥着各种原码扫描、端点扫描软件。自己是觉得在流程确立的情况下用什麽工具都好(具公信力的)，重点还是在有没有人能解决、有没有人要处理，不过现实总会有各种无奈~

<<: 【Day23】我变我变我变变变：如何改变HTML元素的样式

>>: android studio 30天学习笔记-day 8-基本介绍rxjava2

Day23，替你的Gitlab pipeline 添加点搞事

正文

闲聊

【D10】市场热络程度：每日市场成交资讯

Powershell 入门之可选参数和必选参数

第八天：用 Docker 运行 Gradle

【Day24】维持权限 — 隐藏後门(一)

Chrome 浏览器开发者工具

[面试][後端]请简述 Node.js 的 Event Loop

爬虫怎麽爬从零开始的爬虫自学 DAY13 python条件判断语法 if else elif

Day5 Game Frontend

Day 9 - 利用路由协议来组 SD-WAN 网路

第 16 集：Bootstrap 客制化 Sass 架构实战范例