Cloud Secret Managers
Compare and contrast AWS Secrets Manager and AWS Systems Manager Parameter Store for secrets management, and when would you use one over the other?
Explain how cloud secret managers such as AWS Secrets Manager, GCP Secret Manager, and Azure Key Vault provide centralized secret storage and management.
How are secrets stored in a cloud secrets manager?
How can you access secrets stored in a cloud secrets manager from your applications and services securely?
How do SaaS secret managers like Doppler or 1Password fit into a secrets management strategy, and what tradeoffs come with a third-party managed store?
What are the best practices for securing secrets in a cloud secret manager, and how can you ensure sensitive information remains protected?
What problems do dedicated secret management solutions like HashiCorp Vault or AWS Secrets Manager aim to solve, and what value do they add?
Secret Scanning & Leak Response
Describe the process of credential revocation and emergency rotation after a secret leak.
How do entropy-based and pattern-based detection strategies differ in secret scanners, and how do you deal with false positives?
What is push protection in secret scanning, and how does it prevent secrets from ever reaching a remote repository?
What is secret scanning, what tools or techniques are used for it, and when in the development lifecycle should scanning occur?
What steps should be taken immediately when a secret is accidentally leaked or committed to source control, and why is purging history alone insufficient?
Why is purging a secret from Git history alone insufficient, and how do secret scanners handle scanning full repository history versus just new commits?
Fundamentals & Classification
Describe the typical lifecycle of a secret, from generation to revocation.
Explain the concept of a secret in a secrets manager, including how secrets are stored and managed.
Explain the different types of secrets you need to govern and their varying risk profiles and recommended rotation frequencies.
How do you decide which pieces of configuration should be treated as secrets versus ordinary config, and what are the pitfalls of the 12-factor 'config in the environment' approach for secrets?
What are the fundamental security concerns that necessitate storing sensitive data like passwords or tokens as secrets?
What is a "secret" in the context of software systems, and how does it differ from ordinary configuration data?
What is secrets management, and why is it important for securing sensitive information in a DevOps environment?
What is the difference between static and dynamic secrets?
Why does secure secret generation matter, and what makes a strong, high-entropy generated secret compared to a human-chosen one?
Access Control Principles
Describe your approach to secrets management across development, staging, and production environments.
Discuss the concept of "separation of duties" in the context of secrets management.
How do you establish approval and policy workflows for accessing and managing secrets?
How does modern authentication shift focus from "What do you have?" to "Who are you?" in the context of machine identity for secrets access?
How does the principle of least privilege apply to secrets management in containerized environments, and why should secrets not be accessible to all containers?
What is 'need-to-know' access for secrets, and how does it differ from or complement least privilege?
What is the Principle of Least Privilege, and how does it apply to secrets management?
Why should secrets not be reused across development, staging, and production environments, and how does this limit blast radius?
Hashicorp Vault
Discuss the conceptual architecture of HashiCorp Vault, including secret engines, authentication methods, and policies.
Explain the purpose of different types of secret engines in HashiCorp Vault.
How does Vault store its data?
What are Namespaces in Vault?
What are policies in Vault?
What are the essential components or key features of a dedicated secrets management system or vault?
What are the limitations or operational challenges often associated with deploying and managing a self-hosted secrets management solution like HashiCorp Vault at scale?
What can typically be stored in a HashiCorp Vault, and what are its capabilities beyond simple key-value storage?
What is the concept of secret versions in Vault?
What is the difference between authentication and authorization in the context of Vault?
What is the significance of the Vault token?
Which tools enhance HashiCorp Vault for encryption, and what is the role of KMS and the Transit engine?
Key Management & Encryption
Explain encryption as a service, specifically how it relates to configuring a transit secret engine, encrypting/decrypting secrets, and rotating encryption keys.
Explain the concept of envelope encryption (DEK/KEK) and key hierarchies in the context of protecting secrets.
How can you manage and rotate encryption keys in Vault?
How do KMS key policies and IAM policies work together to control access to secrets and encryption keys?
How do you approach securing data in multi-cloud environments, specifically regarding key management?
How do you handle secrets management and encryption at rest to ensure secure storage of sensitive data like passwords and API keys?
How does a secrets vault typically secure sensitive data at rest, and what does encryption at rest for secrets mean?
How does key rotation affect previously encrypted data when using a KMS?
What are the differences between symmetric and asymmetric CMKs in AWS KMS, and when would you use each for secrets protection?
What are the key objectives of key management in a data protection plan?
What is the difference between a Customer Master Key (CMK) and a Data Encryption Key (DEK) in the context of secrets management and encryption?
What is the difference between AWS Key Management Service (KMS) and AWS Secrets Manager in terms of encryption and access control for stored secrets?
What is the role of a Hardware Security Module (HSM) in a secrets management solution?
What is the role of a Key Management Service (KMS) in a secure architecture, particularly concerning secrets management?
Dynamic & Short Lived Credentials
Explain how credential types are used to securely manage sensitive data like SSH keys, API tokens, and passwords.
How can dynamic SSH credentials or an SSH secrets engine be used to manage SSH access without distributing long-lived private keys?
How do you migrate an organization from long-lived static secrets to dynamic, short-lived credentials without disrupting running services?
How do you use API keys for system integrations, and what security considerations are important?
Secret Rotation
Explain strategies for performing secret rotation without downtime, such as versioning, dual/overlapping credentials, or graceful rollover.
How do "leases" and "TTLs" contribute to the security of secrets?
How does an automated rotation function work, and how can you customize it to meet specific secret rotation requirements?
What happens if an application is using an expired secret, and how should applications be designed to handle secret rotation gracefully without downtime?
What is secret rotation, why is it a crucial security practice, and how often should secrets be rotated?
Workload Authentication & Secret Zero
Explain the "Pull Mode" in authentication methods for secrets retrieval, such as when a SecretID is fetched from an AppRole.
Explain the "Push Mode" in authentication methods for secrets retrieval, such as when a custom SecretID is set against an AppRole by the client.
Explain the "secret zero" problem in secrets management, particularly in cloud-native applications, and why protecting secret zero is a primary concern.
How can OIDC-federated short-lived cloud credentials be used in CI/CD pipelines to enhance security compared to long-lived API keys?
How can the "secret zero" problem be addressed or mitigated?
How do Zero Trust architectures help mitigate the 'Secret Zero' problem?
What is workload identity (e.g., SPIFFE/SPIRE), and how does giving a workload a verifiable identity change how it authenticates to a secret store?
Which authentication methods would you use to authenticate a workload with a secret vault using a Kubernetes Service Account Token?
Which authentication methods would you use to authenticate a workload with a secret vault using OIDC or a JWT?
Least Privilege & Blast Radius
Explain the concept of "blast radius containment" in the event of a secret compromise.
How do dynamic secrets help mitigate the risk of secret leaks?
What are dynamic secrets and short-lived credentials, and why are they considered a best practice?
Why are short-lived secrets fundamentally safer than long-lived ones, and what operational challenges does making everything short-lived introduce?
Cicd & Gitops Secrets
Explain the concept of "masked" or "protected" variables in CI/CD systems.
How do Docker and Docker Compose secrets work, and how do BuildKit build secrets let you use a secret during an image build without baking it into a layer?
How do Sealed Secrets work cryptographically, and why does the asymmetric encryption model make them safe to commit to a Git repository?
How do you handle secrets in a CI/CD pipeline to prevent them from being exposed in logs, artifacts, or source code?
How do you manage secrets in a GitOps workflow where configurations are stored in Git, using tools like SOPS or git-crypt?
What is helm-secrets and how does it enable encrypting secret values within Helm charts for GitOps workflows?
Secret Delivery & Injection
Explain the difference between "pull" and "push" models for secret distribution to applications.
Explain various methods applications use to obtain secrets at runtime securely and discuss the trade-offs of each (environment variables, mounted files, sidecar agents, SDKs).
How does a sidecar or agent-based injection pattern (e.g., Vault Agent with auto-auth and templating) deliver secrets to an application, and what are its advantages over the app calling the secrets API directly?
What are the tradeoffs of caching secrets in application memory, and how do you balance availability against staleness and exposure risk?
What does it mean to template secrets into a configuration file at runtime, and what risks must you manage when rendering secrets into templates?
What is the difference between mounting secrets as files on tmpfs versus writing them to disk, and why does tmpfs matter for secret safety?
What role can init containers play in fetching and preparing secrets for an application, and how does that differ from a long-running sidecar?
Certificate & Pki Management
Explain the role of automated certificate issuance and renewal (e.g., ACME, cert-manager) in a secrets management strategy.
How are mTLS credentials (client certificates and keys) managed as secrets that must be issued, distributed, rotated, and revoked?
How do certificate revocation mechanisms like CRLs and OCSP fit into managing the lifecycle of TLS certificates as secrets?
How do you manage the lifecycle of TLS private keys and certificates, including issuance, storage, rotation, expiry monitoring, and revocation?
How does the Vault PKI secrets engine let you issue and manage short-lived certificates, and what advantage does that offer over long-lived TLS certs?
How should code-signing keys be managed and protected as secrets, given their high blast radius if compromised?
Kubernetes Secrets
Explain the security implications of Kubernetes Secret objects being base64-encoded by default rather than encrypted, and what measures can enhance their security at rest.
How can access to Kubernetes Secrets be restricted using RBAC?
How can external secret management systems like HashiCorp Vault or AWS Secrets Manager be integrated with Kubernetes for more robust secret handling?
How do external secret management systems like External Secrets Operator or Sealed Secrets address the limitations of native Kubernetes Secrets?
How does the External Secrets Operator synchronize secrets from an external store into Kubernetes, and what are the security implications of the synced copy living in etcd?
What are the common pitfalls or misconceptions when using Kubernetes Secrets?
What are the security concerns with storing sensitive information directly in Kubernetes Secret objects, and what are alternative, more secure methods for injecting secrets into pods?
What characterizes Kubernetes Secrets?
What is the CSI Secrets Store driver, and how does it allow secrets from external providers to be mounted into pods as volumes?
What is the difference between a Kubernetes ConfigMap and a Kubernetes Secret, and why are Kubernetes Secrets considered a better, though not perfect, solution for sensitive data?
Why should Kubernetes Secrets not be used directly in production?
Exposure Risks & Anti Patterns
How can secrets be exposed through /proc, child processes, or crash/core dumps, and what mitigations reduce that exposure?
How can secrets be kept out of command-line arguments and application logs?
How do you prevent secrets from being baked into Docker image layers?
What are the common failure patterns in secrets management within organizations?
What are the risks associated with injecting secrets via environment variables, and how can these risks be mitigated?
What are the risks of secrets leaking into Terraform state files, and how can this be prevented?
What is "secret sprawl," what are its implications for an organization's security posture, and how can it be prevented?
Why should secrets never be hardcoded in source code, committed to version control, or baked into container images, and what are the associated risks?
Governance Auditing & Resilience
How do you approach disaster recovery and backup for a secrets management system without creating new exposure of the secrets?
How do you audit and monitor secret activity in a secrets management system, and what services or strategies can be used for tracking changes and access?
How do you handle secrets in a DevSecOps/DevOps environment?
What challenges arise when replicating or synchronizing secrets across multiple regions or availability zones, and how do you keep them consistent and secure?
What is a secret inventory, and why is knowing where all your secrets live essential to managing them?
What is break-glass access to secrets, and how do you design an emergency access procedure that remains secure and auditable?
Vault Sealing & Key Quorum
How does HashiCorp Vault handle its own master key and the "seal/unseal" process?
What is auto-unseal, how does it differ from manual unsealing with a key quorum, and what are the security tradeoffs of delegating unseal to a cloud KMS?
What is Shamir's Secret Sharing, and how is it used to distribute Vault unseal keys among multiple key holders?