HMT Solutions 360 Monitoring — 50 technical service runbooks with escalation owner, SLA and SLO
Simulation environment. Runbook steps are drafted operational guidance for NOC training — confirm exact commands with each component owner before production use.
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Supplier Governance Engine verification queue depth · Governance review monitor | RB-SUPPLIERGOV | Marketplace Ops | 99.5% monthly availability | 99.9% of requests under warning threshold |
| B2B Marketplace Engine marketplace RPC failure rate · Marketplace RPC monitor | RB-B2BENGINE | Marketplace On-Call | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Marketplace Commission Engine commission calc failure rate · Commission calc monitor | RB-COMMISSIONENGINE | Marketplace Engineering | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Platform Finance Engine reconciliation mismatch rate · Reconciliation monitor | RB-PLATFORMFINANCE | Platform Finance | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| ZATCA Fatoora API Client ZATCA API failure rate · Clearance probe | RB-ZATCACLIENT | Compliance On-Call | 99% clearance within 5 minutes | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| CRM Engine campaign processing error rate · Campaign processor monitor | RB-CRMENGINE | CRM Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Loyalty Engine loyalty posting failure rate · Points ledger monitor | RB-LOYALTYENGINE | CRM Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Customer Account Engine p95 customer query latency · Customer RPC monitor | RB-CUSTOMERENGINE | CRM Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Credit & Aging Engine aging recalculation error rate · Aging recalculation monitor | RB-CREDITENGINE | Finance Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Collections Engine follow-up queue depth · Follow-up queue monitor | RB-COLLECTIONSENGINE | Collections Ops | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| PWA Offline / Sync Service offline sync queue depth · Sync queue monitor | RB-PWASYNC | Frontend Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| E-Commerce Adapter Layer stock sync lag · Sync worker monitor | RB-ECOMADAPTER | Integrations On-Call | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Public API / API Key Service API 5xx rate · Edge API monitor | RB-PUBLICAPI | Integrations Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Webhook Dispatcher webhook delivery failure rate · Delivery queue monitor | RB-WEBHOOKDISPATCH | Integrations Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Error & Health Monitoring reported errors (24h) · Error intake monitor | RB-ERRORMONITOR | SRE On-Call | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Accounting Posting Guard posting errors · Posting error monitor | RB-ACCOUNTINGGUARD | Finance Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Product Catalog Engine p95 catalog query latency · Catalog RPC monitor | RB-CATALOGENGINE | Inventory Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Inventory Ledger Engine ledger write failure rate · Ledger write monitor | RB-LEDGERENGINE | Inventory On-Call | 99.5% monthly availability | 99.99% successful ledger writes |
| Barcode Service scan decode failure rate · Scan pipeline monitor | RB-BARCODESVC | Inventory Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Email Service email failure rate · Delivery monitor | RB-EMAILSVC | Notification On-Call | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Push Notification Service push failure rate · Push fan-out monitor | RB-PUSHSVC | Notification Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Performance / RPC Monitoring p95 RPC latency · RPC metrics collector | RB-PERFMONITOR | SRE | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Search Service p95 search latency · Search probe | RB-SEARCHSVC | Data Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Platform Governance / Audit platform audit write failure rate · Platform audit monitor | RB-PLATFORMGOV | Compliance Engineering | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Data Import Service import batch failure rate · Import batch monitor | RB-IMPORTSVC | Data Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Store Alert Service open store alerts · Alert intake monitor | RB-ALERTSVC | Store Operations | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Web / PWA Presentation p95 page render latency · Synthetic browser check | RB-WEBPWA | Frontend On-Call | 99.9% monthly availability | p95 render < 1500ms for 99% of sessions |
| Authentication Client login failure rate · Auth flow probe | RB-AUTHCLIENT | Identity On-Call | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Tenant / Active Store Context context resolution error rate · Context resolution probe | RB-TENANTCTX | Platform Core | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Store / Branch Service p95 RPC latency · RPC monitor | RB-STOREBRANCH | Platform Core | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Supabase PostgreSQL Data Layer p95 query latency · Database health monitor | RB-DBLAYER | Data Platform On-Call | 99.95% monthly availability | p95 query < 350ms |
| Authorization / Permission Guards policy denial anomaly rate · Policy evaluation monitor | RB-AUTHZ | Security Engineering | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Audit Service audit write failure rate · Audit write monitor | RB-AUDITSVC | Compliance Engineering | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Warehouse / Location Engine p95 transfer RPC latency · Transfer RPC monitor | RB-WAREHOUSEENGINE | Inventory Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Reporting & Analytics Service p95 report build time · Report job monitor | RB-REPORTINGSVC | Data Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Platform Administration Service admin action failure rate · Admin action monitor | RB-PLATFORMADMIN | Platform Governance | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Supplier Master Service p95 supplier query latency · Supplier RPC monitor | RB-SUPPLIERMASTER | Procurement Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Procurement Engine PO transition failure rate · PO lifecycle monitor | RB-PROCUREMENTENGINE | Procurement Team | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Supplier Account / Payment Engine supplier posting failure rate · Supplier ledger monitor | RB-SUPPLIERACCOUNT | Finance Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Rental Engine rental cycle error rate · Rental cycle monitor | RB-RENTALENGINE | Sales Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Rental Contract Signature Service signature OTP failure rate · Signature OTP monitor | RB-RENTALSIGNATURE | Sales Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Subscription Engine renewal failure rate · Renewal job monitor | RB-SUBSCRIPTIONENGINE | Platform Finance | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Moyasar Payment Gateway Client payment gateway failure rate · Charge probe | RB-MOYASARCLIENT | Platform Finance On-Call | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Sales / POS Engine checkout failure rate · Checkout transaction monitor | RB-POSENGINE | Sales On-Call | 99.9% monthly availability | 99.9% of requests under warning threshold |
| Payment Recording / Allocation allocation error rate · Allocation job monitor | RB-PAYMENTALLOC | Finance Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Promotions Engine p95 rule evaluation latency · Rule evaluation monitor | RB-PROMOENGINE | Sales Platform | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Accounting / GL Engine journal posting error rate · Journal posting monitor | RB-GLENGINE | Finance On-Call | 99.5% monthly availability | 99.99% balanced journal postings |
| VAT / ZATCA Document Engine document generation error rate · UBL generation monitor | RB-ZATCADOC | Compliance Engineering | 99.5% monthly availability | 99.9% of requests under warning threshold |
| Technical service | Runbook | Escalation owner | SLA | SLO |
|---|---|---|---|---|
| Support / SLA Service SLA breach rate · SLA breach monitor | RB-SUPPORTSVC | Support Operations | 99.5% monthly availability | 99.9% of requests under warning threshold |
| SMS Service SMS outbox backlog · SMS outbox monitor | RB-SMSSVC | Notification Team | 99.5% monthly availability | 99.9% of requests under warning threshold |