eCommunications Industry | Day 2 - Topic 5

Data Export Options

Export Formats

Excel (.xlsx)

Best For: Data analysis, pivot tables, further manipulation

  • Features: Multiple sheets, formulas, formatting preserved
  • Use Case: Sales manager exports pipeline → Creates pivot by stage/rep → Shares with team
  • Limitations: Max 1M rows per sheet (Excel limit)
  • Example: Sheet 1: Opportunities (500 rows), Sheet 2: Products (1,200 rows), Sheet 3: Summary (pivot table)

CSV (.csv)

Best For: Data import, integration, bulk operations

  • Features: Plain text, comma-separated, universal compatibility
  • Use Case: Export contacts → Import into email marketing tool (Mailchimp, HubSpot)
  • Limitations: No formatting, single sheet, no formulas
  • Example: 10,000 customer emails exported for newsletter campaign

PDF

Best For: Presentations, archiving, read-only sharing

  • Features: Pixel-perfect, charts/images, print-ready
  • Use Case: Executive summary for board meeting (10-page report with charts)
  • Limitations: Not editable, can't manipulate data
  • Example: Quarterly revenue report with company logo, charts, narrative

HTML

Best For: Email body, web publishing, interactive viewing

  • Features: Clickable links, responsive, works in browsers/email
  • Use Case: Daily report auto-emails as HTML (no attachment, view inline)
  • Limitations: Layout may break in different email clients
  • Example: Morning sales summary with clickable opportunity links

XML

Best For: System integration, data exchange, programmatic processing

  • Features: Structured, machine-readable, supports nested data
  • Use Case: Export opportunities → Import into data warehouse (ETL process)
  • Limitations: Not human-readable, requires parsing
  • Example: Nightly export of 50,000 transactions to accounting system

Export Workflows

Manual Export (User-Initiated)

  1. User runs report "Opportunity Pipeline"
  2. Report displays in browser (147 opportunities)
  3. Click Export button
  4. Select format: Excel
  5. Choose options:
    • Include: All columns (or select specific columns)
    • Format: Original (with colors/formatting) OR Plain (raw data)
    • File name: "Pipeline_2025-11-05.xlsx"
  6. Click Download → File saved to Downloads folder
  7. Time: 5 seconds (for 147 rows)

Scheduled Export (Automated)

  1. Admin configures: "Daily Contact Export"
  2. Schedule: Every day at 2 AM
  3. Format: CSV
  4. Destination: FTP server (ftp://partner.com/uploads/)
  5. Process: Siebel → Exports 50,000 contacts → Uploads to FTP → Partner imports into their system
  6. Notification: Email to admin if export fails

API Export (Programmatic)

  • Scenario: External reporting tool (Tableau) needs live data
  • Integration: Tableau calls Siebel REST API → Returns JSON with opportunity data
  • Frequency: On-demand (when user refreshes Tableau dashboard)
  • Authentication: OAuth token (secure)

Bulk Export Strategies

Handling Large Datasets

Problem: Export 1 million opportunities → Excel crashes, browser times out

Solution 1: Pagination

  • Export in batches: 10,000 rows per file
  • Result: 100 files (Opportunities_Part1.csv, Opportunities_Part2.csv, ... Part100.csv)
  • User combines files or processes separately

Solution 2: Compression

  • Export to ZIP file (contains CSV inside)
  • 1M rows × 50 columns = 500 MB uncompressed → 50 MB compressed (10:1 ratio)
  • Faster download, less storage

Solution 3: Database Direct Access

  • Provide read-only database credentials (SELECT privilege only)
  • User connects via SQL client (SQL Developer, Toad) → Runs custom queries → Exports directly
  • No Siebel UI involved (bypasses application layer for speed)

Solution 4: Data Warehouse

  • Siebel → Nightly ETL → Data Warehouse (Snowflake, Redshift)
  • Users query data warehouse (not production Siebel) → No impact on CRM performance
  • Data Warehouse optimized for analytics (columnar storage, fast queries)

Export to External Systems

Use Case 1: Marketing Automation

Scenario: Send newsletter to 25,000 contacts
  • Export: Contacts BC → Filter: Email Opt-In = 'Yes' → CSV with columns: Email, First Name, Last Name, Company
  • Upload: Import CSV into Mailchimp
  • Campaign: Mailchimp sends personalized emails ("Hi {First Name}, ...")
  • Sync Back: Mailchimp exports opens/clicks → Import back into Siebel (track engagement)

Use Case 2: Accounting System

Scenario: Invoice generation (Siebel CRM → QuickBooks)
  • Export: Closed opportunities → CSV with: Customer ID, Amount, Invoice Date, Line Items
  • QuickBooks: Import CSV → Creates invoices automatically (no manual entry)
  • Frequency: Daily at 5 PM (after sales team updates deals)
  • Validation: QuickBooks flags errors (invalid customer ID) → Admin resolves → Re-import

Use Case 3: Business Intelligence

Scenario: Executive dashboard in Tableau
  • Connection: Tableau → Siebel database (ODBC connection, read-only)
  • Data: Live queries (Tableau queries S_OPTY table in real-time)
  • Refresh: Dashboard updates every 15 minutes (or on-demand)
  • Visualizations: Advanced charts not available in Siebel (Sankey diagrams, treemaps, etc.)

Export Security Considerations

Data Loss Prevention

  • Limit Export Size: Max 10,000 rows per export (prevent bulk data theft)
  • Approval Required: Exports > 5,000 rows require manager approval
  • Watermarking: Add "Exported by John Smith on Nov 5, 2025" to each row (track leaks)
  • Audit Logging: Log all exports (user, timestamp, # rows, destination)

Encryption

  • File Encryption: Auto-encrypt Excel files with password (prevent unauthorized access if file shared)
  • Transport Encryption: HTTPS/SFTP (not FTP) → Data encrypted in transit
  • At-Rest Encryption: Files stored on server encrypted (compliance requirement)

Access Control

  • Export Permissions: Not all users can export (separate permission from "view")
  • Example: Sales Rep can view pipeline report (in Siebel UI) but cannot export to Excel (prevent data leaks)

Performance Optimization

Slow Export Issues

  • Symptom: Export 10,000 rows takes 5 minutes (should be < 30 seconds)
  • Causes:
    • Complex calculated fields (recalculated for each row)
    • Multiple joins (query hits 10 tables)
    • No indexes (database does full table scan)

Optimizations

  • Pre-Aggregation: Store calculated values in database (update nightly) → Export reads pre-calculated values (fast)
  • Indexed Columns: Ensure filters use indexed columns (WHERE Close_Date → index on Close_Date)
  • Limit Columns: Export only needed columns (not all 150 columns)
  • Off-Peak: Schedule large exports at night (avoid peak hours)
  • Streaming: Stream data to file (don't load all 1M rows into memory first)

Best Practices

  • File Naming: Include date/time (Pipeline_2025-11-05_0915.xlsx) → Avoid overwriting previous exports
  • Include Metadata: First row: "Exported on Nov 5, 2025 at 9:15 AM by John Smith from Siebel CRM"
  • Test Imports: Before mass export, test with 10 rows → Verify import works → Then export all
  • Version Control: If export format changes (new column added) → Notify recipients, update documentation
  • Cleanup: Delete old exports after 90 days (storage management, security)
  • Error Handling: If export fails (timeout), retry automatically (up to 3 times)
  • User Training: Educate users on export limits, security, proper file handling

Example: Partner Data Feed

Requirement: Daily export of customer data to partner for co-marketing
  • Data: Accounts that opted-in to partner sharing (5,000 accounts)
  • Columns: Company Name, Industry, Employee Count, Revenue Range, Contact Name, Contact Email
  • Format: CSV (pipe-delimited, not comma → Company names may have commas)
  • Schedule: Daily at 3 AM (after nightly data updates)
  • Delivery: SFTP to partner server (sftp://partner.com/inbox/)
  • Filename: ACME_Customers_YYYYMMDD.csv (e.g., ACME_Customers_20251105.csv)
  • Encryption: PGP-encrypted (partner decrypts with private key)
  • Validation: File must have header row + 5,000 data rows (±50 tolerance)
  • Alert: If file not uploaded by 4 AM → Email to integration team
  • Audit: Log each export (timestamp, # rows, file size, MD5 checksum)