Markdown and Mermaid Writing

Here's the section, ready to paste into your README. Both diagrams use standard Mermaid, which GitHub renders natively in light and dark mode.

## How the nightly pipeline works

Every night a scheduler triggers an ingest job. The job pulls CSV files from an SFTP server and checks each file's schema. Files that fail go to a quarantine folder, and files that pass are loaded into a staging table. A transform job then builds the reporting tables from staging, and a Slack message is sent at the end of the run.

### Pipeline flow

```mermaid
flowchart LR
    accTitle: Nightly Pipeline Flow
    accDescr: A scheduler triggers ingest from SFTP. Each CSV is validated. Bad files go to quarantine, good files load into staging, then the transform builds reporting tables and Slack is notified.

    schedule([⏰ Nightly trigger]) --> ingest[📥 Pull CSVs from SFTP]
    ingest --> validate{🔍 Schema valid?}
    validate -->|No| quarantine[🗂️ Move to quarantine]
    validate -->|Yes| staging[💾 Load into staging table]
    staging --> transform[⚙️ Build reporting tables]
    transform --> notify([📤 Send Slack message])
    quarantine --> notify

    classDef good fill:#dcfce7,stroke:#16a34a,stroke-width:2px,color:#14532d
    classDef bad fill:#fee2e2,stroke:#dc2626,stroke-width:2px,color:#7f1d1d

    class staging,transform good
    class quarantine bad
```

### Interaction between components

```mermaid
sequenceDiagram
    accTitle: Nightly Pipeline Sequence
    accDescr: The scheduler starts the ingest job, which downloads files from SFTP, validates each one, quarantines bad files or loads good ones into the database, runs the transform, and posts a summary to Slack.

    participant sched as ⏰ Scheduler
    participant ingest as 📥 Ingest job
    participant sftp as 🖥️ SFTP server
    participant db as 💾 Database
    participant slack as 📤 Slack

    sched->>ingest: Trigger nightly run
    ingest->>sftp: List and download CSV files
    sftp-->>ingest: CSV files

    loop 🔄 Each file
        ingest->>ingest: 🔍 Validate schema
        alt ✅ Schema valid
            ingest->>db: Load into staging table
            db-->>ingest: Rows loaded
        else ❌ Schema invalid
            ingest->>ingest: 🗂️ Move to quarantine folder
        end
    end

    ingest->>db: Run transform (build reporting tables)
    db-->>ingest: Reporting tables updated

    ingest->>slack: 📤 Post run summary
    slack-->>ingest: Message accepted
```

A few details are my assumptions, so please check them against your setup:

• Who runs what: You didn't say which component runs validation and the transform. I showed the ingest job doing validation and loading, then triggering the transform in the database. If the transform is a separate job, add it as a sixth participant between the database and Slack.
• Failures: The Slack message is sent after both good and bad files, as in the flow diagram. If it should only fire on success, or should list the quarantined files, say so and I'll adjust both diagrams.
• Quarantine location: I didn't say whether the folder is on the SFTP server or local storage. Name it in the intro if readers will need to find it.

I couldn't render the diagrams here, but they follow standard Mermaid syntax. You can paste them into mermaid.live, or preview the README on GitHub, to confirm they display.