Data Retention Policy

This document gives example configuration centered on the Act on Prevention of Transfer of Criminal Proceeds (犯罪収益移転防止法, commonly abbreviated "犯収法" and referred to below as "the Act"), and describes the data lifecycle design implemented in Merlon. For other jurisdictions (including GDPR), the deploying organization must evaluate and configure its own retention policy according to the laws that apply to it and its own retention obligations. Retention periods per data category are also managed in the application layer's retention_policies table (migrations/017_retention.sql) and must be kept in sync with the values in this document.

Retention requirements under the Act

Record type	Retention period	Basis
Transaction records	7 years	The Act (from the end of the transaction)
Verification records (identity verification)	7 years	The Act (from the end of the transaction)

Seven years is represented as 2555 days. This is the seeded value for the transaction_data, customer_data, alert_case_data, and cdd_score_history rows in the retention_policies table (see Configuration Reference for how retention periods are administered through that table and its API, rather than through config.yaml).

Default retention periods (RET-001, RET-002)

The values below are initial defaults, not fixed values. Deploying organizations must set a positive retention period for every data category in accordance with applicable law, contractual obligations, audit requirements, and their own deletion policy. Configuration changes are recorded in the audit log, and data is physically deleted 30 days after its retention deadline is reached.

Data category	Default retention period	Reference point	Configurable
Transaction data	7 years (2555 days)	Date the transaction occurred (transactions.executed_at)	Yes, to any positive number of days
Customer data	7 years (2555 days)	Date of the last transaction	Yes, to any positive number of days
Alerts / cases	7 years (2555 days)	resolved_at / closed_at (unresolved/open records are excluded)	Yes, to any positive number of days
Audit logs	10 years (3650 days)	Date the log was recorded (created_at)	Yes, to any positive number of days
CDD score history	7 years (2555 days)	scored_at	Yes, to any positive number of days

Data lifecycle design in Merlon

Under the Auditability First principle, reproducibility of decision rationale takes top priority.

Data category	Lifecycle
Audit logs	Append-only for the duration of the retention period. Logically deleted once the deadline is reached, then physically deleted 30 days later
Customer data	Retained for 7 years after the transaction relationship ends. Once the retention period has elapsed, direct PII within attributes can be anonymized in response to an Act on the Protection of Personal Information (個人情報保護法, "APPI") deletion request (RET-004, api/internal/retention/anonymize.go)
Rule definitions	All versions are retained permanently (never deleted, so that past decisions remain reproducible)
Score history	Tracks the evolution of risk ratings for the duration of the configured retention period. Logically deleted once the deadline is reached, then physically deleted 30 days later
Transaction data	Logically deleted after the configured retention period, then physically deleted 30 days later

PurgeJob runs daily in PostgreSQL-backed deployments. At each configured retention cutoff it records purge_marked_at, and normal API reads exclude marked data. After a 30-day grace period it physically deletes marked transaction data, alert/case data, CDD score history, customer data, dependent screening results, and audit logs. A customer remains until independently retained child records complete their own lifecycle.

Why rule definitions are retained permanently

For the duration of the retention period, the rule definitions and score history that were in effect at the time make it possible to reproduce the rationale behind a decision. Overwriting or deleting a rule definition would destroy the ability to reproduce past decisions, so all versions are kept. Score history, by contrast, is deleted after the user-configured retention period plus the 30-day grace period elapses, so reproducibility is not guaranteed beyond that window.

Archive strategy (partitioning template for new deployments)

PostgreSQL declarative partitioning (PARTITION BY RANGE) is effective for deployments handling large data volumes, but this project does not retrofit partitioning onto tables that are already live in production (ADR-0010). The following is an example DDL template for monthly RANGE partitions that a newly-provisioned deployment can adopt from scratch.

-- audit_logs: example monthly partitioning keyed on created_at
CREATE TABLE audit_logs (
    id              BIGSERIAL,
    user_id         VARCHAR(255),
    action          VARCHAR(100) NOT NULL,
    resource_type   VARCHAR(100) NOT NULL,
    resource_id     VARCHAR(255),
    details         JSONB DEFAULT '{}',
    ip_address      INET,
    user_agent      TEXT,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (id, created_at)
) PARTITION BY RANGE (created_at);

CREATE TABLE audit_logs_2026_07 PARTITION OF audit_logs
    FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');
-- Add a CREATE TABLE ... PARTITION OF ... statement each month going forward
-- (consider an extension such as pg_partman if you want to automate this)

-- transactions: example monthly partitioning keyed on executed_at
CREATE TABLE transactions (
    id                    UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    customer_id           UUID NOT NULL,
    external_id           VARCHAR(255) NOT NULL,
    amount                NUMERIC(20,2) NOT NULL,
    currency              VARCHAR(3) NOT NULL,
    direction             VARCHAR(10) NOT NULL,
    executed_at           TIMESTAMPTZ NOT NULL,
    created_at            TIMESTAMPTZ NOT NULL DEFAULT now()
    -- (see migrations/002_transactions_alerts.sql for the actual column list)
) PARTITION BY RANGE (executed_at);

CREATE TABLE transactions_2026_07 PARTITION OF transactions
    FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');

Operational guidance:

• For old partitions past the retention period (10 years for audit logs, 7 years for transactions), move them to cold storage or detach them (DETACH PARTITION). Before physically deleting anything, the deploying organization should implement and verify deletion procedures that meet its own legal and audit requirements.
• Operating on individual partitions keeps the cost of applying retention policy (range scans, purges) constant even for very large tables.
• A deployment already running non-partitioned tables that wants to migrate to partitioning generally needs to follow the sequence "create new table → write to both in parallel → migrate data → cut over reads." This migration can be disruptive and may involve downtime, so this repository's automated migrations do not provide it. Plan any such migration separately.
• Partitioning strategy, capacity planning, and migration guidance for the broader set of ever-growing tables, including customer_score_history and alerts, are covered in a separate ADR.

This project does not retrofit partitioning onto tables that are already live in production (ADR-0010, ADR-0011). The following is an example DDL template for monthly RANGE partitions that a newly-provisioned deployment can adopt from scratch, covering customer_score_history and alerts (see ADR-0010 for audit_logs/transactions).

-- customer_score_history: example monthly partitioning keyed on scored_at
CREATE TABLE customer_score_history (
    id           UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    customer_id  UUID NOT NULL,
    risk_score   NUMERIC NOT NULL,
    risk_tier    TEXT NOT NULL,
    factors      JSONB,
    rule_set_id  TEXT,
    rule_set_version INTEGER,
    scored_at    TIMESTAMPTZ NOT NULL DEFAULT now()
) PARTITION BY RANGE (scored_at);

CREATE TABLE customer_score_history_2026_07 PARTITION OF customer_score_history
    FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');
-- Add a CREATE TABLE ... PARTITION OF ... statement each month going forward

-- alerts: example monthly partitioning keyed on detected_at
CREATE TABLE alerts (
    id          TEXT PRIMARY KEY,
    customer_id TEXT NOT NULL,
    scenario_id TEXT NOT NULL,
    severity    TEXT NOT NULL,
    status      TEXT NOT NULL,
    score       NUMERIC,
    description TEXT,
    detected_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at  TIMESTAMPTZ NOT NULL DEFAULT now()
) PARTITION BY RANGE (detected_at);

CREATE TABLE alerts_2026_07 PARTITION OF alerts
    FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');
-- Add a CREATE TABLE ... PARTITION OF ... statement each month going forward

• Once customer_score_history rows pass the user-configured retention period, they are physically deleted 30 days after purge_marked_at is set. When partitioning, confirm the retention policy and grace period before detaching an old partition slated for deletion.
• alerts underpin case management and audit evidence, so keep detached partitions as an archive rather than dropping them.
• For the general procedure a deployment already running non-partitioned tables would follow to migrate to partitioning, GIN index operator-class selection guidance, capacity planning, read-replica routing policy, and recommended autovacuum tuning, see docs/operations/partitioning-guide.md.

Limits of audit-log tamper resistance

The self-hosted edition provides after-the-fact verification via merlon-audit verify, but database-enforced immutability, WORM storage, and hash chaining are not included in the current release. The deploying organization is responsible for configuring database roles, backups, access control, and monitoring. See Regulatory Scope for details.