Arquitectura del ecosistema de ventas

Diagrama de contexto

graph TB
    subgraph "Orígenes de venta"
        POS["🏪 PoS Tiendas"]
        ECOM["🛒 eCommerce"]
        SUBS["📋 Suscripciones"]
        GAS["⛽ Gasolinera (fuera de alcance)"]
    end

    subgraph "Middleware — Yoda"
        YODA["Yoda"]
        PUBSUB["Pub/Sub"]
    end

    subgraph "GCP — Procesamiento"
        direction TB
        DF["Dataflow"]
        BIN2RAW["bin2raw.jar (RF05)"]
        BRONZE["Capa Bronze (GBQ)"]
        DBT["dbt (Silver)"]
    end

    subgraph "Sistemas destino"
        RDO["RDO Corporate"]
        SMART["Smart v2"]
    end

    subgraph "Resiliencia (este proyecto)"
        MON["🔍 Monitorización (RF06)"]
        REC["🔄 Recuperación (RF01/RF03)"]
        BACKUP["📦 Backups Cajas (RF02)"]
        ECOM_MON["🛒 Control eCom (RF07/RF08)"]
    end

    POS --> YODA
    ECOM --> YODA
    YODA --> PUBSUB
    PUBSUB --> DF
    SUBS -.->|RF04| PUBSUB
    DF --> BIN2RAW
    BIN2RAW --> BRONZE
    BRONZE --> DBT
    DF --> RDO

    MON -->|vigila| DF
    MON -->|vigila| PUBSUB
    REC -->|reprocesa| PUBSUB
    BACKUP -->|sube a GCS| DF
    ECOM_MON -->|vigila| YODA

    style GAS stroke-dasharray: 5 5,stroke:#f66
    style SMART stroke-dasharray: 5 5,stroke:#f66

Diagrama de componentes detallado

El siguiente diagrama muestra todos los componentes del ecosistema de ventas, sus interacciones, los flujos de datos y los puntos de monitorización y reconciliación que introduce este proyecto.

graph TB
    %% DIAGRAM CONVENTIONS & STANDARDS REFERENCE
    %% 1. [ ] SYSTEM: Standard rectangle for main units (Apps, Servers).
    %% 2. ("<i>«process»</i>") SERVICE: Standard technical process (Blue).
    %% 3. ("<i>«karaf»</i>") ESB: Karaf Integration Process (Alcampo Red).
    %% 4. ("<i>«pos»</i>") POS: PoS Transaction Sync (Alcampo Red).
    %% 5. ("<i>«yoda»</i>") CORE: Yoda OMS Process (Yoda Green).
    %% 5. ([<i>«gcf»</i>]) SERVERLESS: Google Cloud Function (Google Blue).
    %% 6. [( )] STORAGE/DB: Vertical cylinder for static persistence (Databases & Buckets).
    %%    - Wood: Staging (4th Place/Madera).
    %%    - Bronze: Bronze Layer (3rd Place).
    %%    - Silver: Silver Layer (2nd Place).
    %%    - Black: Ocado (External Source/Charcoal).
    %% 7. {{ }} QUEUE/INTERFACE: Hexagon for data in transit/APIs.
    %%    - Green: Gravitee API Gateway.
    %%    - Blue: Google Cloud Pub/Sub.
    %%    - Thick Blue: Core GCF Monitoring & Reconciliation.
    %% 8. [<i>«elt»</i>] / [<i>«dbt»</i>]: Data Movement & Transformation.
    %%    - Dark Blue: Fivetran (ELT).
    %%    - Orange: dbt (Transformation).
    %% 9. [<i>«jar»</i>] LIBRARY: Rectangular artifact with jar stereotype.
    %% 10. <i>«stereotype»</i>: Professional UML notation (always lowercase: «gbq», «yoda», «gcs»).
    %% 11. Alerts (🚩): 
    %%    - Detects: GCF Monitors that identify the issue.
    %%    - Raises: Source components that provoke the issue.

    %% LINE / ARROW CONVENTIONS
    %% 1. --> (Solid Line): DATA FLOW / PUSH (Source -> Destination)
    %% 2. <--> (Bidirectional): READ/WRITE CYCLE (Process <-> State/DB)
    %% 3. -.-> (Dotted + Arrow): READ / QUERY (Process pulls from Storage)
    %% 4. -.- (Dotted, no arrow): LIBRARY ASSOCIATION (embedded component)
    %% 5. Messages (arrows) are lowercase unless terms require capitalization (e.g. JSON, API).

    %% STYLE DEFINITIONS BY ROLE
    classDef middleware fill:#e1f5fe,stroke:#01579b,stroke-width:2px,color:#000000;
    classDef system fill:#f5f5f5,stroke:#616161,stroke-width:4px,color:#000000;
    classDef library fill:#fff9c4,stroke:#fbc02d,stroke-width:2px,color:#000000;
    classDef karaf fill:#ffcdd2,stroke:#e30613,stroke-width:2px,color:#000000;
    classDef pos fill:#ffcdd2,stroke:#e30613,stroke-width:2px,color:#000000;
    classDef yoda fill:#dcedc8,stroke:#558b2f,stroke-width:2px,color:#000000;
    classDef gravitee fill:#dcedc8,stroke:#558b2f,stroke-width:2px,color:#000000;
    classDef gcf fill:#e3f2fd,stroke:#4285f4,stroke-width:2px,color:#000000;
    classDef gcf_monitor fill:#e3f2fd,stroke:#1565c0,stroke-width:6px,color:#000000;
    classDef gcs fill:#e3f2fd,stroke:#4285f4,stroke-width:2px,color:#000000;
    classDef elt fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000000;
    classDef dbt fill:#ffccbc,stroke:#ff694b,stroke-width:2px,color:#000000;

    classDef bronze fill:#ffe0b2,stroke:#bf360c,stroke-width:2px,color:#000000;
    classDef silver fill:#eeeeee,stroke:#616161,stroke-width:2px,color:#000000;
    classDef staging fill:#d7ccc8,stroke:#5d4037,stroke-width:2px,color:#000000;
    classDef ocado_db fill:#37474f,stroke:#263238,stroke-width:2px,color:#ffffff;

    %% ARROW STYLE LEGEND (Arrow color = Source Node stroke)
    %% ******************************************************
    %% RULE: ARROW COLOR MATCHES SOURCE NODE STROKE COLOR.
    %% ******************************************************
    %% - Grey (#616161): System / Silver.
    %% - Red (#e30613): Karaf.
    %% - Google Blue (#4285f4): GCS / Pub/Sub.
    %% - Thick Blue (#1565c0): GCF Monitor / Fivetran.
    %% - Green (#558b2f): Gravitee / Yoda.
    %% - Middleware Blue (#01579b): Dataflow.
    %% - Bronze (#bf360c): BQ Bronze.
    %% - Wood (#5d4037): BQ Staging.
    %% - Charcoal (#263238): Ocado DB.
    %% - Orange (#ff694b): dbt.
    %% - Yellow (#fbc02d): Library (dotted).

    %% NODE DEFINITIONS

    %% --- SOURCES & EXTERNAL ---
    subgraph POS_Stores ["PoS Servers"]
        direction TB
        POS_001["PoS Server - Store 001"]:::system
        POS_Sync_001("<i>«pos»</i><br/>PoS Transaction Sync"):::pos

        POS_002["PoS Server - Store XXX"]:::system
        POS_Sync_002("<i>«pos»</i><br/>PoS Transaction Sync"):::pos
    end

    Gasolinera["Gas Station DB Server"]:::system

    Yoda_PoS("<i>«yoda»</i><br/>PoS Transaction Sync<br/> <code>Data v2 + RDO</code><hr/><div style='text-align:left'>Raises:<br/>🚩 <font color='#b71c1c'>RDO_PENDING</font></div>"):::yoda
    Yoda_eCom("<i>«yoda»</i><br/>Ocado Orders Sync<hr/><div style='text-align:left'>Raises:<br/>🚩 <font color='#b71c1c'>PIPELINE_STALLED</font><br/>🚩 <font color='#b71c1c'>GAP_ANALYSIS</font></div>"):::yoda
    Yoda_Datafeeds("<i>«yoda»</i><br/>Ocado Datafeeds Sync<hr/><div style='text-align:left'>Raises:<br/>🚩 <font color='#b71c1c'>DATAFEED_MISSING</font></div>"):::yoda

    %% --- MIDDLEWARE & APIs ---
    API_POS{{"<i>«gravitee»</i><br/>PoS Transaction Sync API <br/> <code>Data v2 + RDO</code>"}}:::gravitee
    API_Smart{{"<i>«gravitee»</i><br/>Transaction Operational API <br/> <code>Smart v2</code>"}}:::gravitee
    Smart_App["Smart V2"]:::system

    ESB_Backup("<i>«karaf»</i><br/>PoS Transaction Backup<hr/><div style='text-align:left'>Raises:<br/>🚩 <font color='#b71c1c'>BACKUP_MISSING</font><br/>🚩 <font color='#b71c1c'>PHYSICAL_MISSING</font></div>"):::karaf

    SFTP_Backup["sFTP PoS"]:::system
    GCS_Backup_Bucket[("<i>«gcs»</i><br/>PoS Transaction Backup Bucket")]:::gcs
    GCS_Datafeeds_Bucket[("<i>«gcs»</i><br/>Ocado Datafeeds Bucket")]:::gcs

    %% --- RECONCILIATION & AUDIT ---
    transaction_pos_backup_reconciler("<i>«gcf»</i><br/><b>transaction_pos_backup_reconciler</b><hr/><div style='text-align:left'>Detects:<br/>🚩 <font color='#b71c1c'>BACKUP_MISSING</font><br/>🚩 <font color='#b71c1c'>PHYSICAL_MISSING</font></div>"):::gcf_monitor
    transaction_integrity_controller("<i>«gcf»</i><br/><b>transaction_integrity_controller</b><hr/><div style='text-align:left'>Detects:<br/>🚩 <font color='#b71c1c'>NO_ITEMS / CORRUPT</font><br/>🚩 <font color='#b71c1c'>RDO_PENDING</font><br/>🚩 <font color='#b71c1c'>PIPELINE_STALLED</font><br/>🚩 <font color='#b71c1c'>GAP_ANALYSIS</font><br/>🚩 <font color='#b71c1c'>DATAFEED_MISSING</font></div>"):::gcf_monitor
    Google_Chat{{"Google Chat"}}:::middleware

    %% --- OPERATIONS & PIPELINE ---
    %% ASYNC MESSAGING & INTEGRATION
    PubSub_PoS{{"<i>«pub/sub»</i><br/>Pending PoS Transactions" <br/> <code>Data v2</code>}}:::gcf
    PubSub_eCom{{"<i>«pub/sub»</i><br/>Pending eCom Orders"}}:::gcf
    Dataflow("<i>«process»</i><br/>Dataflow<hr/><div style='text-align:left'>Raises:<br/>🚩 <font color='#b71c1c'>NO_ITEMS / CORRUPT</font></div>"):::middleware
    API_RDO{{"<i>«gravitee»</i><br/>Digital Ticket API <br/> <code>RDO</code>"}}:::gravitee
    RDO["RDO"]:::system
    Ocado["Ocado"]:::system

    %% BigQuery Processing Project
    BQ_Staging[("<i>«gbq»</i><br/>Staging")]:::staging
    BQ_Bronze[("<i>«gbq»</i><br/>Bronze")]:::bronze
    BQ_Silver[("<i>«gbq»</i><br/>Silver")]:::silver
    BQ_Ocado[("<i>«gbq»</i><br/>Ocado Datafeeds")]:::ocado_db
    dbt_retail("<i>«dbt»</i><br/>dbt retail"):::dbt

    %% FLOWS & RELATIONS (Data Flow: Source -> Destination)

    %% Core Sales Ingestion
    POS_001 -->|NRT transaction sync| POS_Sync_001
    POS_Sync_001 --> API_POS
    POS_002 -->|NRT transaction sync| POS_Sync_002
    POS_Sync_002 --> API_POS
    POS_Stores -->|batch transaction backup| SFTP_Backup

    %% Backup Pipeline
    SFTP_Backup -->|reads ZIPs| ESB_Backup
    ESB_Backup -->|stores ZIPs| GCS_Backup_Bucket

    %% GCF_1 Recovery Flow
    GCS_Backup_Bucket -.->|expands ZIPs| transaction_pos_backup_reconciler
    transaction_pos_backup_reconciler -->|"reconciles PoS missing transaction"| API_POS
    transaction_pos_backup_reconciler <-->|read / write<br/>transaction status| BQ_Staging

    %% GCF_2 Audit Objectives (Consolidated Diagnostic Inputs)
    BQ_Ocado -.->|"eCom audit<br/>(returns & subscriptions)"| transaction_integrity_controller
    BQ_Silver -.->|"master filtering<br/>(active store context)"| transaction_integrity_controller
    BQ_Bronze -.->|"data integrity<br/>(header vs lines join)"| transaction_integrity_controller
    transaction_integrity_controller <-->|manages status <br/>+<br/> audit| BQ_Staging

    %% GCF_2 Recovery Actions
    transaction_integrity_controller -->|notifies critical errors| Google_Chat
    transaction_integrity_controller -->|"reprocesses eCom<br/>(publishes eCom JSON)"| PubSub_eCom
    transaction_integrity_controller -->|"reprocesses PoS<br/>(failed transactions)"| API_POS

    %% API Gravitee Outputs
    API_POS -->|proxies| Yoda_PoS
    Yoda_PoS -->|creates or updates| API_RDO
    Yoda_PoS -->|publishes raw JSON| PubSub_PoS
    Yoda_PoS <-->|read / write<br/>transaction status| BQ_Staging

    %% External & Analytical
    Ocado -->|GET /orders| Yoda_eCom
    Yoda_eCom -->|publishes eCom JSON| PubSub_eCom
    Yoda_eCom <-->|read / write<br/>transaction status| BQ_Staging
    Ocado -->|reads datafeeds| Yoda_Datafeeds
    Yoda_Datafeeds -->|stores datafeeds| GCS_Datafeeds_Bucket
    Yoda_Datafeeds -->|writes datafeeds| BQ_Ocado
    Gasolinera -->|extracts raw transactions| Fivetran("<i>«elt»</i><br/>Fivetran"):::elt
    Fivetran -->|loads raw transactions| BQ_Bronze

    %% Smart App Interaction
    Yoda_PoS -->|provides details| API_Smart
    API_Smart -->|creates or updates| Smart_App

    %% Data Pipeline
    PubSub_PoS -->|subscribes to| Dataflow
    PubSub_eCom -->|subscribes to| Dataflow
    Dataflow -->|writes models| BQ_Bronze
    Dataflow -->|saves transaction info| BQ_Staging
    BQ_Ocado -->|reads eCom returns & subscriptions| dbt_retail
    BQ_Bronze -->|reads PoS transactions + eCom orders| dbt_retail
    dbt_retail -->|transforms to silver| BQ_Silver

    %% Embedded Components (No flow)
    Jar_B2R["<i>«jar»</i><br/>bin2raw.jar"]:::library
    Jar_R2R["<i>«jar»</i><br/>raw2rich.jar"]:::library


    Jar_R2R -.-|embeds| Dataflow
    Jar_R2R -.-|embeds| Smart_App

    API_RDO -->|creates or updates| RDO

    Yoda_PoS -.-|embeds| Jar_B2R

    %% Invisible layout link to push Staging deeper in TB
    Google_Chat ~~~ BQ_Staging

    %% LINK STYLING (Arrow color = Source Node stroke)
    %% Edge index reference (43 edges, 0-42):
    %% 0:  POS_001->Sync      1:  Sync->API_POS      2:  POS_002->Sync
    %% 3:  Sync->API_POS      4:  POS->SFTP (Batch)  5:  SFTP->ESB
    %% 6:  ESB->GCS_Backup    7:  GCS_Backup-.->rec  8:  rec->API_POS
    %% 9:  rec<->Staging      10: Ocado-.->integ     11: Silver-.->integ
    %% 12: Bronze-.->integ    13: integ<->Staging    14: integ->Chat
    %% 15: integ->PubSub_eCom 16: integ->API_POS     17: API_POS->Yoda
    %% 18: Yoda->API_RDO      19: Yoda->PubSub_PoS   20: Yoda<->Staging
    %% 21: Ocado->Yoda_eCom   22: Yoda_eCom->PubSub  23: Yoda_eCom<->Staging
    %% 24: Ocado->Yoda_DF     25: Yoda_DF->GCS_DF    26: Yoda_DF->BQ_Ocado
    %% 27: Gas->Fivetran      28: Fivetran->Bronze   29: Yoda_PoS->API_Smart
    %% 30: API_Smart->Smart   31: PubSub_PoS->DF     32: PubSub_eCom->DF
    %% 33: DF->Bronze         34: DF->Staging        35: BQ_Ocado->dbt
    %% 36: Bronze->dbt        37: dbt->Silver        38: Yoda-.-Jar_B2R
    %% 39: Jar_R2R-.-DF       40: Jar_R2R-.-Smart    41: API_RDO->RDO
    %% 42: (invisible) Chat~~~Staging

    %% System Grey (#616161) - idx: 0,2,4,5,21,24,27
    linkStyle 0,2,4,5,21,24,27 stroke:#616161,stroke-width:2px;

    %% Karaf/POS Red (#e30613) - idx: 1,3,6
    linkStyle 1,3,6 stroke:#e30613,stroke-width:2px;

    %% GCS Google Blue Dotted Read (#4285f4) - idx: 7
    linkStyle 7 stroke:#4285f4,stroke-width:2px,stroke-dasharray: 2 3;

    %% GCF Monitor Thick Blue (#1565c0) - idx: 8,9,13,14,15,16
    linkStyle 8,9,13,14,15,16 stroke:#1565c0,stroke-width:2px;

    %% Ocado Charcoal Dotted Read (#263238) - idx: 10
    linkStyle 10 stroke:#263238,stroke-width:2px,stroke-dasharray: 2 3;

    %% Silver Dotted Read (#616161) - idx: 11
    linkStyle 11 stroke:#616161,stroke-width:2px,stroke-dasharray: 2 3;

    %% Bronze Dotted Read (#bf360c) - idx: 12
    linkStyle 12 stroke:#bf360c,stroke-width:2px,stroke-dasharray: 2 3;

    %% Ocado Charcoal Solid (#263238) - idx: 35
    linkStyle 35 stroke:#263238,stroke-width:2px;

    %% Bronze Solid (#bf360c) - idx: 36
    linkStyle 36 stroke:#bf360c,stroke-width:2px;

    %% Gravitee / Yoda Green (#558b2f) - idx: 17,18,19,20,22,23,25,26,29,30,41
    linkStyle 17,18,19,20,22,23,25,26,29,30,40 stroke:#558b2f,stroke-width:2px;

    %% Fivetran Dark Blue (#1565c0) - idx: 28
    linkStyle 28 stroke:#1565c0,stroke-width:2px;

    %% PubSub Google Blue (#4285f4) - idx: 31,32
    linkStyle 31,32 stroke:#4285f4,stroke-width:2px;

    %% Middleware Blue (#01579b) - idx: 33,34
    linkStyle 33,34 stroke:#01579b,stroke-width:2px;

    %% dbt Orange (#ff694b) - idx: 37
    linkStyle 37 stroke:#ff694b,stroke-width:2px;

    %% Library Yellow Dotted Association (#fbc02d) - idx: 38,39,41
    linkStyle 38,39,41 stroke:#fbc02d,stroke-width:1px,stroke-dasharray: 5 5;

    %% Invisible layout link - idx: 42

Capas de datos

Decisión clave: la librería bin2raw