Clever Geek Handbook

gitおよびansibleでのシークレットの検索の自動化

gitリポジトリに何が保存されているか知っていますか？何百ものコミットの中に、誤ってそこに到達した製品サーバーからのパスワードはありますか？





しかし、公開時にansibleスクリプトがクラッシュし、ログ内のパスワードが強調表示された場合はどうなりますか？





このようなチェックを自動化する方法とその結果について説明します。





こんにちは、Habr！

私の名前はオレグです。ロシア連邦のかなり大きな銀行で、IT部門のIT部門で働いています。





  (OPS)  (ansible)   . 





2 git ( , ), , , , OPS .





,   master  OPS. 





? git    ( ) OPS. Jenkins git - ( ) OPS.





git? ?





, . secret management,  HashiCorp Vault, CyberArk Conjur .





, .





, pull request .





, !





?

3 , :





 





---
    dev_ssh_username1: "admin"
    dev_ssh_password1: "admin123"

- .









shell command ansible





- name: Deploy
  hosts: all
  tasks:
    - name: Update
      shell: "update.sh --user={{ update_user }} --password={{ update_password }}"

, playbook -v .





$ ansible-playbook deploy.yml -i env/DEV/hosts -v
TASK [Update] ******************************************************************
changed: [192.168.1.2] => {"changed": true, "cmd": "/home/dev/update.sh --user=secret_user --password=secret_password", "delta": "0:00:05.056532", "end": "2020-11-06 09:53:09.397355", "rc": 0, "start": "2020-11-06 09:53:04.340823", "stderr": "", "stderr_lines": [], "stdout": "Update SUCCESS", "stdout_lines": ["Update SUCCESS"]}

, playbook ( -v)





$ ansible-playbook deploy.yml -i env/DEV/hosts
TASK [Update] ******************************************************************
fatal: [192.168.1.2]: FAILED! => {"changed": true, "cmd": "/home/dev/update.sh --user=secret_user --password=secret_password", "delta": "0:00:00.018710", "end": "2020-11-06 10:14:30.642419", "msg": "non-zero return code", "rc": 127, "start": "2020-11-06 10:14:30.623709", "stderr": "/bin/sh: /home/dev/update.sh:     ", "stderr_lines": ["/bin/sh: /home/dev/update.sh:     "], "stdout": "", "stdout_lines": []}

  environment, :





- name: Deploy
  hosts: all
  tasks:
    - name: Update
      shell: "/home/dev/update.sh $AUTH_DATA"
      environment:
        AUTH_DATA: "--user={{ update_user }} --password={{ update_password }}"

credentials . no_log.









withCredentials





Jenkins, credentials  withCredentials. pipeline credential, , . Jenkins .





, , :





node {
  stage('Jenkins Credentials | Decrypt Secret File') {
    withCredentials([file(credentialsId: 'credentials-id', variable: 'secretFile')]) {
      sh 'cat $secretFile'
    }
  }
}

, , . ,  secretFile  withCredentials.











, . ( ).





.  SonarQube   Checkmarx. .





- SonarQube Hard-coded credentials are security-sensitive





, . ,  .





C Checkmarx - . ( Application Security ). , :





import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
 
@RestController
public class DemoController {
    @RequestMapping("/show_me_creds")
    public @ResponseBody String show_me_creds() {
        String thisIsMyLittleSecret = "qwerty12345";
        return thisIsMyLittleSecret;
    }
}

opensource . google "find secrets in git" :





Gitleaks, :





  • regexp, ;





  • ;





  • json;





  • ;





  • .





- toml , : 





[[rules]]
  description = "generic secret regex"
  regex = '''secret(.{0,20})([0-9a-zA-Z-._{}$\/\+=]{20,120})'''
  tags = ["secret", "example"]

, : 





[[rules]]
  description = "entropy and regex example"
  regex = '''secret(.{0,20})([0-9a-zA-Z-._{}$\/\+=]{20,120})'''
  [[rules.Entropies]]
    Min = "4.5"
    Max = "4.7"

: " , ,    4,5 4,7, "





,  .





:





➜  ~  gitleaks --repo=gitleaks --repo=https://github.com/gitleakstest/gronit.git --verbose --pretty
INFO[2020-04-28T13:00:34-04:00] cloning... https://github.com/gitleakstest/gronit.git
Enumerating objects: 135, done.
Total 135 (delta 0), reused 0 (delta 0), pack-reused 135
{
        "line": "const AWS_KEY = \"AKIALALEMEL33243OLIAE\"",
        "offender": "AKIALALEMEL33243OLIA",
        "commit": "eaeffdc65b4c73ccb67e75d96bd8743be2c85973",
        "repo": "gronit.git",
        "rule": "AWS Manager ID",
        "commitMessage": "remove fake key",
        "author": "Zachary Rice",
        "email": "zricethezav@users.noreply.github.com",
        "file": "main.go",
        "date": "2018-02-04T19:43:28-06:00",
        "tags": "key, AWS"
}
...
...
WARN[2020-04-28T13:00:35-04:00] 6 leaks detected. 33 commits audited in 77 milliseconds 738 microseconds

, gitleaks - .





- whitelist , :





[[rules]]
  description = "entropy and regex example"
  regex = '''secret(.{0,20})['|"]([0-9a-zA-Z-._{}$\/\+=]{20,120})['|"]'''
  [[rules.Entropies]]
    Min = "4.5"
    Max = "4.7"
  [[rules.whitelist]]
    regex = '''secret.some_value_that_match_regexp_but_not_really_a_secret'''
    description = "ignore that string"

- pull request webhook Jenkins, , gitleaks - NEED WORK.





, , gitleaks ( ). 





ansible

, , playbook, Jenkins .





- name: Deploy
  hosts: all
  tasks:
    - name: Update
      shell: "update.sh --user={{ update_user }} --password={{ update_password }}"

ansible - , OPA (      ), python.





,  Ansible Lint.





Ansible, " " , .





2 :





  • ;





  • playbook. , . 





- python , id, short description, description tags ( Ansible Lint ) match matchtask, .   .





matchtask, task, . 





, ( ):





class CommandsInsteadOfModulesRule(AnsibleLintRule):
    id = '303'
    shortdesc = 'Using command rather than module'
    description = (
        'Executing a command when there is an Ansible module '
        'is generally a bad idea'
    )
    severity = 'HIGH'
    tags = ['command-shell', 'resources', 'ANSIBLE0006']
 
    _commands = ['command', 'shell']
    _modules = {
        'apt-get': 'apt-get',
# ,    
        'yum': 'yum',
    }
 
    def matchtask(self, file, task):
        if task['action']['__ansible_module__'] not in self._commands:
            return
 
        first_cmd_arg = get_first_cmd_arg(task)
        if not first_cmd_arg:
            return
 
        executable = os.path.basename(first_cmd_arg)
        if executable in self._modules and \
                boolean(task['action'].get('warn', True)):
            message = '{0} used in place of {1} module'
            return message.format(executable, self._modules[executable])

- task command shell  _modules - .





- (password/pass/login ) command shell . 





no_log, . no_log - .





, Gitleaks Ansible Lint, . , .. ( ).





withCredentials

, Jenkinsfile , ( ).





node {
  stage('Jenkins Credentials | Decrypt Secret File') {
    withCredentials([file(credentialsId: 'credentials-id', variable: 'secretFile')]) {
      sh 'cat $secretFile'
    }
  }
}

.





Jenkinsfile groovy,  Abstract Syntax Tree  .   AstBuilder  ,  withCredentials, withCredentials. , .





,   secretFile - stage,  secretFile . 





1000 , ,   .





, pipeline  Pipeline model definition plugin, pipeline json . - .





?

, 1000 pull request. 3% gitleaks, , ansible.





, OPS .





, :





  • https://nightfall.ai/resources/introducing-radar-api-detect-credentials-secrets-in-code-via-machine-learning/ - . - ?





  • https://www.shellcheck.net/ - shell .





  • https://github.com/PyCQA/bandit - security linter python.





  • https://twitter.com/leak_scavenger は、github、pastebin、ghostbinで秘密、クレジットカード、秘密鍵などをスキャンする興味深いTwitterボットです。もちろん、情報源は入手可能です。











More articles:


All Articles