企业官网建设流程全解析

claude-skills SRE 事件响应与混沌工程实战指南

【免费下载链接】claude-skills67 Specialized Skills for Full-Stack Developers. Transform Claude Code into your expert pair programmer.项目地址: https://gitcode.com/GitHub_Trending/claud/claude-skills

本指南以 claude-skills 仓库中 sre-engineer 技能的 事件与混沌工程引用文档 为核心骨架，系统讲解事件分级与度量模型、事件响应 Runbook、无指责事后复盘（Blameless Postmortem）模板，以及从实验建模、故障注入到安全执行、Game Days 演练和成熟度评估的一整套混沌工程落地方法。读完本文，你将获得一套可以直接复制运行的 Python/YAML 代码资产，用于为自己的生产系统搭建事件管理闭环与韧性验证体系。

一、事件响应框架：用代码定义严重级与关键指标

事件管理的第一步是建立统一的分级标准和度量口径。原文档给出了一套可执行的事件模型：用Severity枚举定义严重级别，用Incident数据类跟踪事件生命周期，并自动计算三个 SRE 核心指标——检测时间、MTTR 和总时长。

from dataclasses import dataclass from datetime import datetime from enum import Enum from typing import List class Severity(Enum): """Incident severity levels.""" SEV1 = "critical" # Complete outage, major customer impact SEV2 = "high" # Partial outage, significant impact SEV3 = "medium" # Degraded performance, some users affected SEV4 = "low" # Minor issue, minimal impact @dataclass class Incident: """Incident tracking.""" id: str title: str severity: Severity started_at: datetime detected_at: datetime resolved_at: datetime | None = None root_cause: str | None = None impact: str | None = None @property def detection_time(self) -> float: """Time from start to detection in minutes.""" delta = self.detected_at - self.started_at return delta.total_seconds() / 60 @property def mttr(self) -> float | None: """Mean Time To Repair in minutes.""" if not self.resolved_at: return None delta = self.resolved_at - self.detected_at return delta.total_seconds() / 60 @property def total_duration(self) -> float | None: """Total incident duration in minutes.""" if not self.resolved_at: return None delta = self.resolved_at - self.started_at return delta.total_seconds() / 60 # Example incident incident = Incident( id="INC-2024-001", title="Database connection pool exhaustion", severity=Severity.SEV2, started_at=datetime(2024, 1, 15, 14, 30), detected_at=datetime(2024, 1, 15, 14, 35), resolved_at=datetime(2024, 1, 15, 15, 10), root_cause="Connection leak in payment service", impact="Payment processing delayed for 15% of users" ) print(f"Detection time: {incident.detection_time:.1f} minutes") print(f"MTTR: {incident.mttr:.1f} minutes") print(f"Total duration: {incident.total_duration:.1f} minutes")

这段代码的几个设计要点值得在实际落地时借鉴：

• 字段语义即指标口径：started_at（故障实际开始）、detected_at（监控告警或人工发现）、resolved_at（确认恢复）三个时间戳是全部指标计算的输入。detection_time衡量监控盲区，mttr衡量修复效率，total_duration衡量整体业务受损时长。
• SEV1–SEV4 分级：SEV1 为完全中断、重大客户影响；SEV2 为部分中断、显著影响；SEV3 为性能劣化、部分用户受影响；SEV4 为影响轻微的小问题。分级直接决定后续 Runbook 中走哪套响应流程、通知到哪个层级。
• 该模型与 sre-engineer 技能在 SKILL.md 中定义的职责一致：该技能定位为“定义 SLO、设计事件响应流程、产出监控配置与自动化脚本”，其核心工作流第 6 步要求“设计并执行混沌实验，并在标记实验完成前验证恢复满足 RTO/RPO 目标”。

二、事件响应 Runbook：检测—响应—解决的全流程剧本

仅有模型还不够，事件发生时团队需要一个“照着做就行”的剧本。原文档提供了完整的incident_response.yaml，将响应过程拆解为检测、按级别响应、角色分工、恢复收尾四个阶段：

# incident_response.yaml incident_response: detection: - "Acknowledge alert in PagerDuty" - "Join #incident-response Slack channel" - "Create incident doc from template" - "Assess severity (SEV1-4)" sev1_response: # Critical - all hands - "Page on-call lead + backup" - "Notify VP Engineering immediately" - "Start Zoom war room" - "Assign incident commander" - "Assign communication lead" - "Post status update every 15 minutes" sev2_response: # High - team response - "Page on-call engineer" - "Notify team lead" - "Create incident channel" - "Post status update every 30 minutes" roles: incident_commander: - "Coordinate response efforts" - "Make decisions quickly" - "Delegate tasks" - "Communicate with stakeholders" communication_lead: - "Post regular status updates" - "Notify affected customers" - "Update status page" - "Summarize timeline" on_call_engineer: - "Investigate root cause" - "Implement fixes" - "Verify resolution" - "Document actions taken" resolution: - "Verify metrics returned to normal" - "Monitor for 30 minutes" - "Post final status update" - "Schedule postmortem within 48 hours" - "Close incident"

该 Runbook 的编排原则与仓库其他 SRE 参考资料一脉相承：

• 告警必须可执行。监控与告警引用文档 强调“好的告警是可操作的，而非仅仅是信息性的”，并给出了page_worthy（即时用户影响、SLO 违规进行中、错误预算烧毁率超过 10x、安全事件、数据丢失风险）与not_page_worthy（无当前影响的预测性告警、信息类指标等）的区分标准，以及 critical/warning/info 三级告警路由策略。
• 沟通频次随严重级收紧：SEV1 每 15 分钟更新一次状态，SEV2 每 30 分钟一次，这一频次目标需要 communication lead 专职保证。
• 48 小时复盘时限：解决后 48 小时内必须安排事后复盘，防止关键上下文随记忆衰减丢失。

三、无指责复盘（Blameless Postmortem）：让每次事故都产生资产

复盘的目的不是追责，而是把事故变成可复用资产。原文档提供了一份可直接套用的复盘模板，涵盖摘要、影响、时间线、根因、解决措施、检测与行动项、经验教训七个部分：

# Postmortem: [Incident Title] **Date:** 2024-01-15 **Authors:** [Names] **Status:** Complete **Severity:** SEV2 ## Summary One-paragraph summary of what happened, impact, and resolution. ## Impact - **Duration:** 40 minutes (14:30 - 15:10 UTC) - **Users affected:** ~15% of payment transactions - **Revenue impact:** Estimated $X delayed - **SLO impact:** Consumed 2.3% of monthly error budget ## Timeline (all times UTC) | Time | Event | |-------|-------| | 14:30 | Deployment of payment-service v2.3.0 completed | | 14:32 | Error rate begins increasing | | 14:35 | Alert fires: HighErrorRate | | 14:36 | On-call engineer acknowledges | | 14:40 | Incident declared SEV2 | | 14:45 | Root cause identified: connection leak | | 14:50 | Rollback initiated | | 14:55 | Rollback completed | | 15:00 | Error rate returns to normal | | 15:10 | Incident resolved, monitoring continued | ## Root Cause The payment-service v2.3.0 deployment introduced a connection leak in the database connection pool. The new retry logic was not properly closing connections on timeout, causing the pool to exhaust after ~20 minutes. ## Resolution Rolled back to payment-service v2.2.1, which immediately resolved the issue. ## Detection **What went well:** - Alert fired within 5 minutes of issue start - Clear runbook helped quick diagnosis **What could be improved:** - Could have caught in staging with longer load test - Database connection pool metrics not monitored ## Action Items | Action | Owner | Priority | Due Date | |--------|-------|----------|----------| | Add connection pool monitoring | @alice | P0 | 2024-01-20 | | Extend staging load tests to 30min | @bob | P1 | 2024-01-25 | | Review all resource cleanup in retry logic | @charlie | P1 | 2024-01-30 | | Add integration test for connection leaks | @dave | P2 | 2024-02-05 | ## Lessons Learned **What went well:** - Quick detection and response - Effective team communication - Clear rollback procedure **What didn't go well:** - Issue not caught in pre-production testing - No monitoring for connection pool exhaustion **Where we got lucky:** - Issue occurred during low-traffic period - Only affected payment service, not critical systems

模板中的几个设计细节值得关注：

• “Where we got lucky”：明确要求写出这次运气成分，用于识别系统里的隐性脆弱点（例如“低流量时段才没酿成大祸”），是后续混沌实验选题的天然来源。
• 时间线精确到分钟：以 UTC 统一时区，逐条对齐部署、告警、确认、根因、回滚等动作，为 MTTR、检测时间等指标提供原始证据。
• SLO impact 量化：用“消耗了当月 2.3% 的错误预算”将单次事故与整体 SLO 管理体系挂钩，这与 错误预算策略引用文档 中基于剩余预算（healthy/warning/critical/exhausted）触发不同管控动作的机制是配套的。
• sre-engineer 技能在 SKILL.md 的 MUST DO 约束中明确要求“为所有事件编写无指责复盘”，MUST NOT 约束中明确“禁止跳过复盘或归咎于人”，这两条与模板中的“Lessons Learned”结构互为表里。

四、混沌工程实验建模：假设、爆炸半径与回滚计划

混沌工程的核心不是“乱搞”，而是用科学实验的方式主动验证系统韧性。原文档给出ChaosExperiment数据类，将每次实验定义为四个关键要素——假设（hypothesis）、爆炸半径（blast_radius）、回滚计划（rollback_plan）、成功标准（success_criteria），并用ExperimentStatus跟踪实验生命周期：

# chaos_experiment.py from dataclasses import dataclass from datetime import datetime from enum import Enum from typing import Callable class ExperimentStatus(Enum): """Chaos experiment lifecycle states.""" PLANNED = "planned" RUNNING = "running" SUCCESS = "success" FAILED = "failed" ABORTED = "aborted" @dataclass class ChaosExperiment: """Define a chaos engineering experiment.""" name: str hypothesis: str # What we expect to happen blast_radius: str # Scope of impact rollback_plan: str success_criteria: str status: ExperimentStatus = ExperimentStatus.PLANNED started_at: datetime | None = None completed_at: datetime | None = None observations: list[str] | None = None def should_abort(self, metrics: dict) -> bool: """Check if experiment should be aborted. Args: metrics: Current system metrics Returns: bool: True if experiment should abort """ # Abort if error rate exceeds 10% if metrics.get('error_rate', 0) > 0.10: return True # Abort if latency p99 exceeds 2 seconds if metrics.get('latency_p99', 0) > 2.0: return True return False # Example: Database failover experiment db_failover_experiment = ChaosExperiment( name="Database Primary Failover", hypothesis="System automatically fails over to replica within 30s with <1% error rate", blast_radius="Single database instance, 50% of production traffic", rollback_plan="Restore primary database immediately, redirect traffic", success_criteria="- Failover completes in <30s\n- Error rate <1%\n- No data loss", )

should_abort内置的两个中止阈值（错误率 >10% 或 p99 延迟 >2 秒）就是安全网的程序化表达。这一建模方式与仓库中 chaos-engineer 技能 的安全检查清单高度一致：

• 先验证稳态（Steady State）：注入故障前必须先定义并验证基线指标；
• 爆炸半径上限（Blast Radius Cap）：从最小影响范围开始，验证通过后再扩大；
• 自动化回滚 ≤30 秒：中止路径必须在实验开始前写好脚本并测试通过；
• 单变量原则：一次只改变一个故障条件；
• 生产环境必须有安全网：需要熔断器、功能开关或金丝雀隔离；
• 闭环（Close the Loop）：每个实验都必须产出书面学习总结和至少一条可跟踪的改进项。

混沌实验设计引用文档 进一步给出了更精细的假设公式化写法——“Given [正常状态], when [故障发生], then [预期行为], measured by [指标]”，并通过BlastRadiusConfig.validate()强制执行安全规则：生产环境超过 10% 流量必须同时具备功能开关与自动回滚，实验最长时长超 10 分钟需审批。这些都是对ChaosExperiment建模的有效补充。

五、故障注入模式：延迟、杀 Pod 与网络分区

原文档提供了一套基于Protocol的注入器接口，所有注入器都遵循inject()+rollback()对称契约，保证“注入什么就能撤回什么”：

# chaos_patterns.py - Common chaos engineering patterns import time import random from typing import Protocol class ChaosInjector(Protocol): """Interface for chaos injection.""" def inject(self) -> None: """Inject chaos into the system.""" ... def rollback(self) -> None: """Remove chaos and restore normal operation.""" ... class LatencyInjector: """Inject artificial latency into requests.""" def __init__(self, target_service: str, latency_ms: int): self.target_service = target_service self.latency_ms = latency_ms def inject(self) -> None: """Add latency using iptables or proxy.""" # Example using tc (traffic control) on Linux import subprocess subprocess.run([ "tc", "qdisc", "add", "dev", "eth0", "root", "netem", "delay", f"{self.latency_ms}ms" ]) def rollback(self) -> None: """Remove latency.""" import subprocess subprocess.run(["tc", "qdisc", "del", "dev", "eth0", "root"]) class PodKiller: """Kill pods to test resilience.""" def __init__(self, namespace: str, label_selector: str, kill_percentage: float = 0.5): self.namespace = namespace self.label_selector = label_selector self.kill_percentage = kill_percentage self.killed_pods = [] def inject(self) -> None: """Randomly kill pods matching selector.""" import subprocess # Get pods result = subprocess.run( ["kubectl", "get", "pods", "-n", self.namespace, "-l", self.label_selector, "-o", "name"], capture_output=True, text=True ) pods = result.stdout.strip().split('\n') num_to_kill = int(len(pods) * self.kill_percentage) pods_to_kill = random.sample(pods, num_to_kill) # Kill selected pods for pod in pods_to_kill: subprocess.run(["kubectl", "delete", pod, "-n", self.namespace]) self.killed_pods.append(pod) def rollback(self) -> None: """Pods will be recreated by deployment controller.""" # Wait for pods to be recreated time.sleep(30) class NetworkPartition: """Simulate network partition between services.""" def __init__(self, source_pod: str, target_service: str): self.source_pod = source_pod self.target_service = target_service def inject(self) -> None: """Block network traffic using iptables.""" import subprocess subprocess.run([ "kubectl", "exec", self.source_pod, "--", "iptables", "-A", "OUTPUT", "-d", self.target_service, "-j", "DROP" ]) def rollback(self) -> None: """Restore network traffic.""" import subprocess subprocess.run([ "kubectl", "exec", self.source_pod, "--", "iptables", "-D", "OUTPUT", "-d", self.target_service, "-j", "DROP" ])

三个注入器的技术要点：

• LatencyInjector：通过 Linuxtc netem在网卡层注入固定延迟，rollback()删除 qdisc 规则即可瞬时恢复。注意在容器/多网卡环境中应把eth0替换为实际业务网卡。
• PodKiller：按 label selector 列出 Pod 后随机删除指定百分比，rollback()依赖 Deployment 控制器的自愈能力等待 Pod 重建——这正是 Kubernetes 天然支持的高可用机制，也是混沌实验“撤回即恢复”的典型代表。
• NetworkPartition：在 Pod 内用iptables OUTPUT链 DROP 目标服务的流量，模拟服务间网络分区，rollback()用-D删除规则恢复。
• 三种模式与 混沌工具引用文档 中列出的生态工具（Chaos Monkey 随机终止实例、Gremlin 的 CPU/网络攻击、Litmus、Chaos Mesh、Toxiproxy 网络代理故障、Pumba 容器故障）可互为替换：自研注入器适合轻量定制场景，而 Litmus 等平台适合在 Kubernetes 上做声明式、可审计的故障注入。

六、安全执行器：带约束的 ChaosRunner

有了实验定义和注入器，还需要一个“带安全带”的执行器。原文档给出了ChaosRunner——它负责前置检查、基线采集、周期监控、约束中止和强制回滚，是整条混沌工程链路中最关键的工程化组件：

# chaos_runner.py - Safe chaos experiment execution from dataclasses import dataclass from datetime import datetime, timedelta import time @dataclass class SafetyConstraints: """Safety constraints for chaos experiments.""" max_error_rate: float = 0.10 # 10% max_latency_p99: float = 2.0 # 2 seconds max_duration_minutes: int = 15 business_hours_only: bool = True class ChaosRunner: """Safely execute chaos experiments with monitoring.""" def __init__(self, safety: SafetyConstraints): self.safety = safety def run_experiment( self, experiment: ChaosExperiment, injector: ChaosInjector, get_metrics: Callable[[], dict], ) -> ChaosExperiment: """Execute chaos experiment safely. Args: experiment: Experiment definition injector: Chaos injector implementation get_metrics: Function to fetch current metrics Returns: Updated experiment with results """ # Pre-flight checks if self.safety.business_hours_only: current_hour = datetime.now().hour if 9 <= current_hour <= 17: # Business hours experiment.status = ExperimentStatus.ABORTED experiment.observations = ["Aborted: Business hours constraint"] return experiment # Baseline metrics baseline_metrics = get_metrics() print(f"Baseline metrics: {baseline_metrics}") # Start experiment experiment.status = ExperimentStatus.RUNNING experiment.started_at = datetime.now() experiment.observations = [] try: # Inject chaos print(f"Injecting chaos: {experiment.name}") injector.inject() # Monitor for max duration start_time = datetime.now() max_duration = timedelta(minutes=self.safety.max_duration_minutes) while datetime.now() - start_time < max_duration: time.sleep(10) # Check every 10 seconds current_metrics = get_metrics() experiment.observations.append( f"{datetime.now().isoformat()}: {current_metrics}" ) # Check safety constraints if experiment.should_abort(current_metrics): print("ABORTING: Safety constraint violated") experiment.status = ExperimentStatus.ABORTED break else: # Completed successfully experiment.status = ExperimentStatus.SUCCESS except Exception as e: print(f"ERROR: {e}") experiment.status = ExperimentStatus.FAILED experiment.observations.append(f"Exception: {str(e)}") finally: # Always rollback print("Rolling back chaos injection") injector.rollback() experiment.completed_at = datetime.now() return experiment # Example usage def get_current_metrics() -> dict: """Fetch metrics from Prometheus.""" # In real implementation, query Prometheus return { 'error_rate': 0.02, # 2% 'latency_p99': 0.45, # 450ms } safety = SafetyConstraints(business_hours_only=False) runner = ChaosRunner(safety) experiment = ChaosExperiment( name="Kill 50% of API pods", hypothesis="API remains available with 50% pod loss", blast_radius="50% of API pods", rollback_plan="Pods auto-restart via deployment", success_criteria="Error rate <5%, latency p99 <1s", ) injector = PodKiller( namespace="production", label_selector="app=api", kill_percentage=0.5, ) result = runner.run_experiment(experiment, injector, get_current_metrics) print(f"Experiment status: {result.status.value}")

ChaosRunner的执行时序可以总结为五个阶段，也是任何混沌实验都应遵循的黄金流程：

1. 前置检查（Pre-flight）：默认约束business_hours_only=True，在 9:00–17:00 营业时间内直接拒绝执行，避免实验叠加在业务高峰上。生产环境建议保持开启。
2. 基线采集：注入前先取一次基线指标，供实验后对比“是否回到了稳态”。
3. 注入与监测循环：每 10 秒采样一次指标并追加到observations，形成实验期间的完整观测记录。
4. 约束中止：should_abort在错误率 >10% 或 p99 延迟 >2 秒时立即中止，将实验标记为ABORTED。
5. 强制回滚：无论成功、中止还是异常，finally块中的injector.rollback()保证注入一定会被撤回，completed_at记录结束时间。

关于“稳态验证”，chaos-engineer 的 SKILL.md 通过一个完整的 Litmus ChaosEngine 示例展示了生产级的同款流程：先用kubectl get deploy/kubectl top pods验证基线（p99 <200ms、错误率 <0.1%），再通过PODS_AFFECTED_PERC: 33限制爆炸半径，实验过程中用kubectl logs观察日志，一旦稳态被破坏立即kubectl patch chaosengine ... engineState:stop中止并确认rollout status。这印证了ChaosRunner的设计并非纸上谈兵，而是与主流混沌平台的执行范式完全一致。

七、Game Days：定期的“受控灾难”演练

混沌实验解决“单点韧性”，Game Days 则解决“团队能力与流程验证”。原文档给出了完整的gameday_plan.yaml，把演练设计为日期、时长、参与方、目标、场景、成功标准与安全措施的结构化计划：

# gameday_plan.yaml gameday: date: "2024-02-15" duration: "2 hours" participants: - SRE team - Backend engineers - On-call rotation objectives: - Test incident response procedures - Validate monitoring and alerting - Practice communication protocols - Identify gaps in runbooks scenarios: - scenario: "Database Primary Failure" inject: "Terminate primary database pod" expected: "Automatic failover to replica in <30s" - scenario: "API Service Overload" inject: "Generate 10x normal traffic" expected: "Rate limiting activates, no errors" - scenario: "Network Partition" inject: "Block traffic between API and database" expected: "Circuit breaker opens, graceful degradation" success_criteria: - "All scenarios handled without escalation" - "MTTR <30 minutes for all scenarios" - "Documentation updated with learnings" - "Action items created for gaps" safety: - "Run in staging environment first" - "VP Engineering notified beforehand" - "Abort plan ready for each scenario" - "Customer support team on standby"

每个场景都遵守同一结构：inject（注入什么故障）+expected（期望的系统行为）。这与前文ChaosExperiment的 hypothesis + success_criteria 一脉相承，只是从“单次实验”放大到“多场景演练”。

仓库中的 Game Days 规划引用文档 提供了更完整的配套材料，可作为这套计划的延伸：

• Surprise Scenarios（惊喜场景库）：刻意保留未提前公布的多故障组合（如数据库连接池泄漏 + 缓存同时失效），验证团队真实的应急响应能力；
• 观测指标：GameDayMetrics类量化了检测时间、响应时间、恢复时间、峰值错误率、告警漏报数等，success_rate()按 RTO/RPO/零客户影响/零漏报四项计算演练通过率；
• 行动项闭环：演练结束后 30 分钟内做即时复盘，1 周内完成详细事后分析并生成带 Owner 和截止日期的行动项清单。

八、混沌工程成熟度评估：从零散试验到文化

混沌工程的最终目标是成为一种团队文化而非偶发活动。原文档用ChaosMaturity枚举定义了五个成熟度等级，并给出了assess_maturity判定函数：

from enum import IntEnum class ChaosMaturity(IntEnum): """Chaos engineering maturity levels.""" NONE = 0 # No chaos testing AD_HOC = 1 # Occasional manual tests SCHEDULED = 2 # Regular game days CONTINUOUS = 3 # Automated in CI/CD CULTURE = 4 # Embedded in development def assess_maturity(practices: dict[str, bool]) -> ChaosMaturity: """Assess chaos engineering maturity level.""" if not practices.get('any_chaos_testing'): return ChaosMaturity.NONE if not practices.get('regular_game_days'): return ChaosMaturity.AD_HOC if not practices.get('automated_chaos'): return ChaosMaturity.SCHEDULED if not practices.get('chaos_in_cicd'): return ChaosMaturity.CONTINUOUS return ChaosMaturity.CULTURE # Example assessment current_practices = { 'any_chaos_testing': True, 'regular_game_days': True, 'automated_chaos': True, 'chaos_in_cicd': False, } maturity = assess_maturity(current_practices) print(f"Chaos maturity: {maturity.name}") # Target: CONTINUOUS or CULTURE

五个等级的含义与进阶路径：

对照仓库中的 混沌工具引用文档，从 SCHEDULED 迈向 CONTINUOUS 的典型落地方式是：在 GitHub Actions 中用cron: '0 10 * * 1-5'定时触发混沌测试 Job（安装 Litmus → 应用pod-delete实验 → 等待 ChaosEngine 完成 → 验证错误率 <1% → 失败时通知 Slack），或在 Jenkins 中构建带 ENVIRONMENT/CHAOS_TYPE/DURATION 参数化选择的流水线，并在 Pre-flight 阶段校验错误率，这正是assess_maturity中“自动化 + 进 CI/CD”两个条件的可执行注解。

九、与整个 SRE 技能体系的衔接

从仓库源码结构看，incident-chaos.md并非孤立的单点文档，而是 sre-engineer 技能五大参考文档之一（SKILL.md 中的 Reference Guide 表列出了slo-sli-management.md、error-budget-policy.md、monitoring-alerting.md、automation-toil.md、incident-chaos.md五份主题引用），它们共同构成事件与混沌工程实践的完整闭环：

• 事件检测依赖监控：监控与告警引用文档 提供四金信号（延迟、流量、错误、饱和度）的 Prometheus recording rules 与 SLO 烧毁率告警，以及每类告警必须链接 Runbook 的规范——正是事件 Runbook 中detection阶段的输入；
• 事故影响对标错误预算：错误预算策略引用文档 定义了剩余预算从 healthy（>75%）到 exhausted（0%）的四个状态及对应管控动作（如 25% 以下冻结非关键功能开发），复盘模板中的“SLO impact”字段即由此计算；
• 修复手段可以自动化：自动化与 toil 削减引用文档 展示了SelfHealer健康检查自动修复与AutomatedRunbook自动化 Runbook 的执行模式，事件解决阶段的常见操作（如数据库 failover 的四个步骤）可以直接脚本化；
• 复用方：仓库中独立的 chaos-engineer 技能 在related-skills中明确声明与sre-engineer互为关联技能，前者的实验设计、工具生态和 Game Days 模板正是后者混沌工程模块的深化实现。

从应用场景看，这套体系最典型的落地路径是：先用监控四金信号 + 烧毁率告警建立事件检测能力 → 用Incident模型 +incident_response.yaml建立事件响应流程 → 用无指责复盘模板沉淀每次事故 → 再用ChaosExperiment+ChaosRunner主动验证和补齐系统韧性短板 → 通过 Game Days 锻炼团队 → 最终用assess_maturity持续跟踪并推动混沌工程从零散试验走向 CI/CD 常态化，形成“监控发现问题—事件快速响应—复盘沉淀经验—混沌实验预防”的可靠性增强闭环。

十、如何获取与使用这些资产

以上所有代码与模板均可在当前仓库中直接查看与复用：

• 事件响应框架、Runbook、复盘模板、混沌实验建模、故障注入模式、安全执行器、Game Days 计划与成熟度评估：skills/sre-engineer/references/incident-chaos.md（本文核心出处）；
• sre-engineer 技能整体约束（MUST DO / MUST NOT DO）、核心工作流与输出模板：skills/sre-engineer/SKILL.md；
• 配套的五份参考文档：slo-sli-management.md、error-budget-policy.md、monitoring-alerting.md、automation-toil.md；
• 混沌工程的深化资料：chaos-engineer/SKILL.md、experiment-design.md、chaos-tools.md、game-days.md、kubernetes-chaos.md、infrastructure-chaos.md。

这些 Python 数据类与 YAML 模板可以直接作为你自己团队事件管理与混沌工程体系的原型：复制incident_response.yaml作为团队 Runbook 骨架，用ChaosRunner的SafetyConstraints约束你的每一次故障注入实验，再把gameday_plan.yaml落地为下个月的定期演练计划。唯一需要提醒的是，混沌实验默认面向 staging 环境，若确需在生产执行，务必像BlastRadiusConfig.validate()与business_hours_only约束要求的那样，先配好功能开关、自动回滚与审批流程再动手。

【免费下载链接】claude-skills67 Specialized Skills for Full-Stack Developers. Transform Claude Code into your expert pair programmer.项目地址: https://gitcode.com/GitHub_Trending/claud/claude-skills