/19-ab-test-setup

name: 19-ab-test-setup
description: "Khi nguoi dung can thiet lap A/B test cho ads, landing page, email, hoac product. Cung dung khi nguoi dung nhac 'A/B test', 'split test', 'test creative', 'test copy', 'test landing', 'test gia', 'kiem dinh thong ke', 'sample size', 'testing framework', 'tim winning ad', 'test hypothesis ads', 'test log'. Skill nay giup chon bien can test, tinh sample size, setup tracking, va phan tich ket qua thong ke (significance) — phu hop cho team khong phai data scientist."
metadata:
  version: 1.1.0
  category: performance
license: MIT
related:
  - product-marketing-context
  - 03-danh-gia-hieu-suat
  - 05-copy-quang-cao
  - 12-brief-landing-page
  - 13-phan-tich-du-lieu
  - 21-audit-ads-performance
  - 52-account-structure
  - 55-scaling-ads

A/B Test Setup

Thu thap thong tin

Buoc 0: Kiem tra context

Doc `.agents/product-marketing-context.md` neu co.

Buoc 1: Hoi user

1. **Test cai gi?** (Headline ads / Landing page / Email subject / Gia / CTA button / Creative video) 2. **Metric chinh?** (CTR / Conversion rate / CPMess / Revenue / Open rate) 3. **Traffic hang ngay cua trang/ads?** (De tinh sample size can thiet) 4. **Muc tieu test?** (Tang X% metric chinh / Tim ra version tot nhat)

---

7 Nguyen tac A/B test

1. Test 1 bien duy nhat

**Quy tac cot loi.** Neu test 2 bien cung luc → khong biet bien nao tac dong.

• XAU: Doi headline + doi CTA + doi anh → khong ket luan duoc
• TOT: Chi doi headline (A vs B), moi thu khac giu nguyen

2. Gia thuyet co the kiem dinh

**Format:** "Neu lam [X], metric [Y] se tang [Z%]"

• TOT: "Neu doi CTA tu 'Dang ky' sang 'Nhan tu van mien phi', conv rate tang 15%"
• XAU: "Copy moi tot hon" (khong cu the, khong do duoc)

3. Sample size du

**Khong duoc dung test som.**

• Toi thieu: **100 conversion moi variant** (khong phai 100 visitor)
• Tot nhat: Tinh truoc (xem cong thuc ben duoi)

4. Thoi gian du

**Test tung tuan tron** (khong 3 ngay, khong 10 ngay) — khach mua hanh vi khac nhau cac ngay.

• Toi thieu: 7 ngay
• Nen co: 14 ngay

5. Khong peek som

**Khong check ket qua moi gio.** Dung ket qua cho den khi het thoi han + du sample.

• Peek som + stop som = false positive.

6. Significance > 95%

**Chi tin khi p-value < 0.05.** Neu khong: ngau nhien, khong phai do bien cua ban.

7. Documented

**Ghi lai:**

• Gia thuyet
• Ngay bat dau / ket thuc
• Sample size
• Ket qua
• Ket luan

→ De sau nay ban khong test lai cai da test.

---

Cong thuc tinh Sample Size

Cong thuc don gian

Sample size/variant =
  16 × p × (1-p) / MDE²

Trong do:
  p = Baseline conversion rate hien tai (vi du 0.02 = 2%)
  MDE = Minimum Detectable Effect (muc do thay doi toi thieu ban muon phat hien, vi du 0.2 = 20% tang)

Vi du

**Landing page hien co conv rate 3%, muon phat hien thay doi 20%+:**

• p = 0.03
• MDE = 0.2 × 0.03 = 0.006 (tuc tang tu 3% len 3.6%)
• Sample size = 16 × 0.03 × 0.97 / 0.006² = 12,933 visit/variant

=> Can 25,866 visit total, neu 500 visit/ngay → 52 ngay.

**Landing page hien conv rate 10%, muon phat hien 20%+:**

• p = 0.10
• MDE = 0.02 (tu 10% len 12%)
• Sample size = 16 × 0.10 × 0.90 / 0.02² = 3,600/variant

=> 7,200 visit total, 500/ngay → 14 ngay (kha thi).

Quy tac nhanh

| Traffic/ngay | Conv rate | Thoi gian can | Co nen test? | |-------------|-----------|--------------|-------------| | < 100 visit | Bat ky | 2+ thang | Khong — traffic thap | | 100-500 | 2-5% | 3-6 tuan | Co, nhung kien nhan | | 500-2K | 2-5% | 2-3 tuan | Co — ly tuong | | 2K+ | Bat ky | 1-2 tuan | Co — nhanh |

**Neu traffic thap** (< 100/ngay) → khong test, thay vao do tap trung tang traffic truoc.

---

8 Yeu to nen test (uu tien)

1. Headline

**Tac dong cao nhat.** 80% nguoi doc headline, 20% doc body → headline thay doi toi uu = bien nho nhat.

Idea:

• Question vs Statement
• Number specific vs generic
• Outcome-focused vs Feature-focused
• Short (5-7 tu) vs Long (12-15 tu)

2. CTA Button

**Do thay doi duoc de test.** Thuong tang 5-25%.

Test:

• Text: "Dang ky" vs "Nhan tu van mien phi" vs "Bat dau ngay"
• Color: Xanh primary vs Do vs Cam
• Size: Standard vs Lon
• Position: Tren cung vs Cuoi trang

3. Hero Image / Video

Test:

• Anh san pham vs Anh nguoi dung
• Video vs Anh tinh
• Testimonial video vs Demo video

4. Price Display

Test:

• 299K vs 299,000 vs 299.000đ
• Co discount strikeout vs khong
• Monthly vs Yearly highlight

5. Social Proof

Test:

• So con so vs Review chi tiet
• Logo bao chi vs So khach
• Testimonial video vs text

6. Form Fields

Test:

• 3 fields vs 5 fields vs 7 fields
• Label tren vs ben trai
• Placeholder example vs khong co

7. Email Subject Line

Test:

• Question vs Benefit
• Personalization ([Ten]) vs Khong
• Emoji vs Khong (VN: khong emoji tot hon)

8. Ad Creative

Test:

• Hook 3s dau khac nhau
• UGC vs Brand
• Format: Single image vs Carousel vs Video

---

Ma tran uu tien bien test

Khong test theo hung. Chon bien theo **tac dong** truoc, **do de** sau.

| Bien | Tac dong len ket qua | Do kho thuc hien | Uu tien | Metric chinh | |------|---------------------|-----------------|---------|-------------| | Creative — hook 0-3s | Rat cao (quyet dinh CTR va CPM) | De — chi doi 3 giay dau | **1** | Hook rate, CTR | | Audience / tep | Cao (ai nhin thay ads) | De — doi setting | **2** | CPM, CPL | | Offer / CTA | Rat cao (ho nhan duoc gi) | Trung binh — can quyet dinh kinh doanh | **3** | CVR, CPL | | Landing page | Cao (trai nghiem sau click) | Kho — can dev/design | **4** | CVR, bounce | | Ad copy (than bai) | Trung binh | De | **5** | CTR, CVR | | Mau sac / font / chi tiet nho | Thap | De | **6** | — |

**Quy tac:** het bien uu tien 1 roi moi xuong 2. Dung nhay thang vao test mau nut khi hook chua bao gio duoc test.

---

Quy trinh test ads tren nen tang

Thu tu test creative (chay lan luot, khong chay song song)

| Vong | Test gi | Cach lam | Metric doc | |------|--------|---------|-----------| | 1 | **Hook 0-3s** | 3-5 hook