# Goals

The primary objective of this WorkDash analysis workflow is to enable continuous, high-fidelity analysis of communication and workflow data within a home lab environment while enforcing strict privacy boundaries. The system must satisfy four core goals:

1. **Local-First Privacy Architecture**: All raw email and Microsoft Teams data must be ingested, processed, classified, and stored exclusively on-premises. No raw content, metadata, or intermediate embeddings may traverse the local network boundary unless explicitly redacted or synthesized.
2. **Fine-Grained Local Intelligence**: A local large language model (LLM) must perform detailed classification tasks including PII detection, sentiment analysis, topic clustering, urgency scoring, action item extraction, and thread context mapping. Classifications must be deterministic, auditable, and configurable via confidence thresholds.
3. **Benchmark-Ready Output Generation**: Public-facing reports must be generated using either fully synthetic data or rigorously redacted aggregate summaries. The synthetic corpus must preserve statistical distributions, temporal patterns, and topic diversity without containing any real-world private information.
4. **Actionable Owner Reporting**: The reporting plane must remain highly useful to the home lab owner. Dashboards, trend analyses, anomaly flags, and productivity metrics must be structured for immediate operational decision-making, not merely academic benchmarking.

The workflow is designed as a modular pipeline: Ingestion → Local Classification → Redaction/Synthesis → Aggregation → Reporting. Each stage enforces data sovereignty, model transparency, and human oversight where necessary.

# Data That Must Stay Local

All raw and intermediate data must remain within the home lab's physical and logical perimeter. The following data categories are strictly local:

- **Raw Communication Content**: Full email bodies, attachments, Teams chat messages, channel posts, meeting transcripts, and reply chains.
- **Metadata**: Sender/recipient addresses, timestamps, thread IDs, read receipts, delivery status, and API pagination tokens.
- **Intermediate Artifacts**: Model embeddings, classification logits, confidence scores, redaction masks, synthetic generation seeds, and pipeline logs.
- **Storage Architecture**: Data is stored on encrypted local volumes (e.g., ZFS with AES-256 or LUKS2). Access is restricted via POSIX permissions and local service accounts. No cloud sync, backup to external providers, or telemetry endpoints are permitted.
- **Network Isolation**: The ingestion and processing nodes operate on a dedicated VLAN with outbound traffic blocked except for verified local model updates (SHA-256 hashed) and internal DNS resolution. Inbound access is restricted to localhost and authorized local management interfaces.
- **Data Lifecycle**: 
  1. Ingest via local IMAP/Graph API proxy.
  2. Chunk and embed locally.
  3. Classify using local model.
  4. Redact or synthesize.
  5. Aggregate into metrics.
  6. Delete raw content after configurable retention window (default: 30 days) or archive encrypted locally.
- **Access Controls**: Role-based local access (owner, analyst, service account). All access is logged to an immutable local audit store. No remote authentication or OAuth tokens are persisted beyond session scope.

# Local Classification

Classification is performed entirely on-premises using a quantized LLM (e.g., Llama-3-8B-Instruct, Mistral-7B, or Phi-3-medium) running via Ollama or vLLM on a local GPU. The pipeline is designed for fine-grained, structured output with explicit confidence scoring.

**Classification Taxonomy**:
- **PII Detection**: Names, phone numbers, addresses, IP addresses, account numbers, license plates, internal IDs.
- **Sentiment & Tone**: Positive, neutral, negative, urgent, frustrated, collaborative, directive.
- **Topic Clustering**: Project delivery, infrastructure, vendor management, compliance, personal, administrative, security incident.
- **Urgency & SLA**: Low, medium, high, critical, time-sensitive.
- **Action Extraction**: Tasks, deadlines, owners, dependencies, meeting outcomes.
- **Thread Context**: Reply chain depth, original vs. forwarded, cross-platform references (email ↔ Teams).

**Pipeline Mechanics**:
1. **Chunking**: Messages are split into semantic units (≤512 tokens) with overlap for context preservation.
2. **Embedding & Routing**: Lightweight embedding model routes chunks to specialized classification prompts based on content type.
3. **Structured Output**: LLM returns JSON with fields: `classification`, `confidence`, `pii_flags`, `action_items`, `topic`, `sentiment`, `urgency`.
4. **Threshold Routing**:
   - `confidence ≥ 0.85`: Auto-process, apply redaction/synthesis.
   - `0.60 ≤ confidence < 0.85`: Flag for human review queue.
   - `confidence < 0.60`: Drop or escalate to full manual review.
5. **Fine-Grained Detail Extraction**: The model is prompted to extract nested details such as vendor contract clauses, infrastructure change requests, compliance keywords (GDPR, HIPAA, internal policy refs), and cross-references between email and Teams threads.

**Model Governance**:
- Prompts are version-controlled and stored locally.
- Few-shot examples are curated from redacted historical data.
- Model updates require hash verification and rollback capability.
- Inference is batched to optimize GPU utilization and reduce latency.

# Redaction Strategy

Redaction is a multi-pass, context-aware process designed to eliminate private content while preserving analytical utility. The strategy operates in three layers:

**Layer 1: Deterministic Masking**
- Regex-based extraction of known PII patterns (emails, phone numbers, IPs, credit cards, SSNs).
- Replacement with structured tokens: `[REDACTED:EMAIL]`, `[REDACTED:PHONE]`, etc.
- Applied before LLM processing to reduce hallucination risk.

**Layer 2: Context-Aware LLM Redaction**
- The local model evaluates surrounding context to identify implicit PII (e.g., "the new server at 192.168.1.45", "John from finance").
- Uses entity resolution to maintain consistency across threads.
- Outputs redacted text with semantic placeholders: `[REDACTED:PERSON]`, `[REDACTED:PROJECT]`, `[REDACTED:INTERNAL_SYSTEM]`.

**Layer 3: Post-Processing Verification**
- Entropy analysis checks for residual patterns that could reconstruct private data.
- Cross-validation against a local PII dictionary and known internal naming conventions.
- Fallback: If verification fails, the chunk is dropped or escalated to human review.

**Attachment Handling**:
- PDFs, images, and Office documents are processed via local OCR and text extraction.
- Extracted text undergoes the same redaction pipeline.
- Binary attachments are either discarded after text extraction or replaced with synthetic metadata placeholders.

**Redaction Output**:
- Redacted text is used for metric aggregation and owner reporting.
- Raw content is never stored alongside redacted versions in the same accessible directory.
- Redaction masks are logged for auditability but do not contain reversible data.

# Synthetic Test Generation

Synthetic data generation enables benchmarking, model stress testing, and public reporting without privacy exposure. The process ensures statistical fidelity while guaranteeing zero data leakage.

**Generation Methodology**:
1. **Distribution Matching**: Real metadata (timestamps, senders, topics, thread lengths) is sampled to establish baseline distributions.
2. **Content Generation**: The local LLM generates messages using constrained prompts that preserve tone, structure, and topic diversity but replace all entities with fictional equivalents.
3. **Template Augmentation**: High-frequency communication patterns (e.g., status updates, incident reports, meeting summaries) are generated via parameterized templates seeded with synthetic variables.
4. **Cross-Platform Consistency**: Synthetic Teams messages and emails are generated in parallel to maintain realistic reply chains, forwarding patterns, and cross-references.

**Privacy Guarantees**:
- No real names, addresses, IPs, or internal project codes are used.
- Synthetic corpus is cryptographically hashed and version-controlled.
- Generation scripts are deterministic given a fixed seed, enabling reproducibility.
- KL divergence checks ensure synthetic distributions match real metadata within acceptable tolerances (e.g., ±5% for topic frequency, ±10% for response time distributions).

**Usage in Benchmarking**:
- Synthetic data is used for public reports, model evaluation, pipeline regression testing, and stress validation.
- Real redacted data is used for owner dashboards and internal trend analysis.
- Both datasets are processed through the same pipeline to ensure metric comparability.

# Metrics To Keep

The reporting plane focuses on actionable, owner-centric metrics that reflect communication health, productivity, and system performance. All metrics are aggregated locally and exported as structured JSON/CSV.

**Communication Volume & Patterns**:
- Messages/emails per day/week/month
- Peak activity hours and day-of-week distribution
- Thread depth and reply chain length
- Cross-platform migration rate (email ↔ Teams)

**Response & Resolution Latency**:
- Time to first response (median, p95, p99)
- Average resolution time per topic
- SLA compliance rate (if defined)
- After-hours communication ratio

**Topic & Urgency Distribution**:
- Topic breakdown (project, admin, vendor, security, personal)
- Urgency distribution over time
- Sentiment trend analysis (rolling 7-day windows)
- Action item extraction rate and completion tracking

**Anomaly & Compliance Flags**:
- Unusual sender frequency or new external contacts
- Sudden spikes in after-hours activity
- High sentiment variance or frustration indicators
- Compliance keyword triggers (internal policy, data handling, security incidents)

**System & Pipeline Health**:
- Model inference latency (ms/token)
- Pipeline throughput (messages/hour)
- Error rates and drop reasons
- Storage utilization and retention compliance
- Redaction accuracy rate (human-verified)

Metrics are stored in a local time-series database (e.g., InfluxDB or SQLite with time-indexed tables) and visualized via a local dashboard (Grafana, Metabase, or custom Streamlit app). All exports are encrypted and require local authentication.

# Human Review

Human oversight is integrated at critical decision points to ensure accuracy, compliance, and model alignment. The review workflow is designed to be lightweight but rigorous.

**Review Queues**:
- Medium-confidence classifications (`0.60–0.85`)
- Redaction verification failures
- Synthetic data approval requests
- Anomaly flag validation
- Model drift indicators (confidence distribution shifts)

**Review Interface**:
- Local web UI with side-by-side raw/redacted/synthetic views
- Annotation tools for correcting classifications, adding missing PII flags, or adjusting thresholds
- Feedback submission that updates local prompt templates and few-shot examples
- Immutable audit logs of all decisions, timestamps, and reviewer IDs

**Operational Cadence**:
- Daily: Spot-check 5% of flagged items, review anomaly queue
- Weekly: Full audit of redaction accuracy, synthetic data validation, metric reconciliation
- Monthly: Model evaluation, threshold tuning, pipeline performance review, retention policy compliance check

**Owner Authority**:
- Final decision on redaction thresholds and drop policies
- Approval of synthetic generation parameters
- Definition of topic/urgency taxonomies
- Access to raw data (if needed for legal or operational reasons)

**Continuous Improvement**:
- Review feedback is batched and used to fine-tune local prompts or retrain lightweight classifiers.
- Model versioning ensures reproducibility and rollback capability.
- Human-in-the-loop metrics are tracked to measure review load and adjust automation thresholds.

# Failure Modes

The workflow anticipates and mitigates common failure modes through architectural safeguards, monitoring, and fallback procedures.

**PII Leakage**:
- *Cause*: Incomplete redaction, model hallucination, regex bypass.
- *Mitigation*: Multi-layer redaction, entropy verification, fallback to drop, strict access controls, regular PII dictionary updates.

**Model Drift**:
- *Cause*: Prompt degradation, distribution shift, outdated few-shot examples.
- *Mitigation*: Confidence distribution monitoring, periodic re-evaluation against redacted benchmarks, synthetic stress tests, human review feedback loop.

**Pipeline Bottlenecks**:
- *Cause*: GPU saturation, large attachments, high message volume.
- *Mitigation*: Async processing, batch inference, resource monitoring, graceful degradation (drop low-priority chunks), queue prioritization.

**Synthetic Data Drift**:
- *Cause*: Over-fitting to real patterns, loss of diversity, distribution mismatch.
- *Mitigation*: KL divergence checks, version-controlled generation, human validation, fallback to real redacted data for critical metrics.

**False Positives/Negatives**:
- *Cause*: Ambiguous context, domain-specific terminology, threshold misalignment.
- *Mitigation*: Confidence routing, human review queue, threshold tuning, domain-specific prompt fine-tuning.

**Compliance & Retention Risks**:
- *Cause*: Accidental cloud sync, unencrypted storage, extended retention.
- *Mitigation*: Network isolation, encryption at rest, automated retention policies, audit logs, legal review of data handling procedures.

**System Availability**:
- *Cause*: Hardware failure, power loss, software crashes.
- *Mitigation*: Local backups (encrypted), UPS integration, service restart scripts, stateless pipeline design where possible.

# Example Private Summary

**WorkDash Private Analysis Report – Week 42**
*Generated: 2024-10-18 | Owner: Home Lab Admin | Data Retention: 30 days*

**Executive Overview**
Communication volume increased by 18% compared to the previous week, driven by infrastructure migration discussions and vendor onboarding. Response latency improved by 12%, with 85% of urgent items addressed within 4 hours. Sentiment remains neutral-to-positive, with minor frustration spikes around network configuration changes.

**Key Metrics**
- Total Messages: 1,247 (Email: 412, Teams: 835)
- Avg Response Time: 2.4 hours (p95: 6.1 hours)
- Topic Distribution: Infrastructure (34%), Vendor (22%), Admin (18%), Project Delivery (15%), Personal (11%)
- Action Items Extracted: 47 | Completed: 31 | Pending: 16
- Anomalies Detected: 3 (new external vendor contact, after-hours spike on 10/14, high sentiment variance in thread #8842)

**Redaction & Classification Summary**
- PII Flags: 89 (emails, phone numbers, internal IP ranges)
- Redaction Accuracy: 98.2% (human-verified)
- Drop Rate: 1.4% (low-confidence chunks)
- Synthetic Data Used: 0% (private report uses redacted real data)

**Actionable Insights**
1. Infrastructure migration thread requires cross-team alignment. Schedule sync for 10/22.
2. Vendor onboarding delayed due to missing compliance documentation. Escalate to legal.
3. After-hours communication correlates with deployment windows. Consider shift-based on-call rotation.
4. Sentiment dip in network team suggests tooling friction. Review monitoring dashboard usability.

**System Health**
- Model Inference Latency: 142 ms/token
- Pipeline Throughput: 310 messages/hour
- Storage Utilization: 68% (encrypted volume)
- Retention Compliance: 100%

**Next Steps**
- Review pending action items by 10/20
- Update vendor compliance checklist
- Adjust urgency thresholds for deployment-related messages
- Schedule monthly model evaluation

# Example Publishable Summary

**WorkDash Benchmark Report – Q3 2024**
*Generated: 2024-10-18 | Audience: Public/Research | Data Source: Synthetic + Redacted Aggregates*

**Executive Overview**
This report presents aggregated communication analytics derived from a privacy-preserving analysis pipeline. All raw content has been redacted or replaced with statistically matched synthetic data. Metrics reflect realistic home lab workloads while ensuring zero exposure of private information.

**Key Metrics (Aggregate)**
- Total Messages Analyzed: 4,812 (Synthetic: 60%, Redacted Real: 40%)
- Avg Response Time: 2.6 hours (p95: 6.4 hours)
- Topic Distribution: Infrastructure (32%), Vendor (24%), Admin (19%), Project Delivery (14%), Personal (11%)
- Action Item Extraction Rate: 3.8% of messages
- Anomaly Detection Precision: 91.5% | Recall: 87.2%

**Redaction & Synthesis Validation**
- PII Removal Rate: 100% (verified via entropy analysis and human spot-checks)
- Synthetic Distribution Match: KL divergence < 0.08 across all metadata dimensions
- Pipeline Error Rate: 1.1% (auto-dropped or escalated)
- Benchmark Reproducibility: Seed-controlled generation, version-locked prompts

**Synthetic Example (Redacted Context)**
> *Thread: Network Migration Planning*
> `[REDACTED:PERSON]` initiated discussion regarding infrastructure upgrade. Timeline: 3 weeks. Dependencies: `[REDACTED:VENDOR]` hardware delivery, `[REDACTED:INTERNAL_SYSTEM]` configuration. Urgency: High. Sentiment: Neutral. Action: Schedule alignment meeting.
> *Note: All entities replaced with semantic placeholders. Structure and tone preserved for benchmark validation.*

**Benchmark Utility**
- Demonstrates viability of local-first communication analysis
- Validates redaction pipeline accuracy under real-world distributions
- Provides reproducible synthetic corpus for model evaluation
- Shows actionable metric aggregation without privacy compromise

**Limitations & Scope**
- Synthetic data approximates but does not replicate exact real-world edge cases
- Metrics reflect home lab scale; enterprise scaling may require distributed processing
- Public report excludes compliance-specific flags and internal policy references

**Reproducibility**
- Generation scripts: `synth_gen_v2.1`
- Model: `llama-3-8b-instruct-q4_K_M`
- Redaction pipeline: `redact_multi_pass_v3`
- All artifacts version-controlled and hash-verified

# Field Classification Table

| Field Name              | Source          | Keep/Drop/Redact | Rationale                                                                 | Handling Method                                  |
|-------------------------|-----------------|------------------|---------------------------------------------------------------------------|--------------------------------------------------|
| Sender Address          | Email/Teams     | Redact           | Contains PII and internal routing info                                    | `[REDACTED:SENDER]` or synthetic alias           |
| Recipient Address       | Email/Teams     | Redact           | Contains PII and internal routing info                                    | `[REDACTED:RECIPIENT]` or synthetic alias        |
| Subject Line            | Email/Teams     | Redact           | May contain project names, client info, or sensitive topics               | Semantic masking or synthetic replacement        |
| Message Body            | Email/Teams     | Redact           | Raw content contains PII, context, and private details                    | Multi-pass LLM + regex redaction                 |
| Timestamps              | Email/Teams     | Keep             | Essential for latency, volume, and trend analysis                         | Retain as-is, aggregate locally                  |
| Thread ID               | Email/Teams     | Keep             | Enables reply chain and context mapping                                   | Retain as-is, hash if needed for privacy         |
| Attachment Metadata     | Email/Teams     | Redact           | Filenames may contain project codes or client names                       | Replace with `[REDACTED:ATTACHMENT_TYPE]`        |
| Attachment Content      | Email/Teams     | Drop/Redact      | Binary/text may contain PII or sensitive data                             | Extract text → redact → discard binary           |
| Sentiment Score         | Local Model     | Keep             | Analytical metric, no private content                                     | Store as numeric value                           |
| Topic Label             | Local Model     | Keep             | Aggregatable classification, no PII                                       | Store as categorical value                       |
| Urgency Score           | Local Model     | Keep             | Actionable metric, no private content                                     | Store as numeric/categorical value               |
| Action Items            | Local Model     | Redact           | May contain names, deadlines, or project references                       | Extract → redact entities → store as structured  |
| PII Flags               | Local Model     | Keep             | Metadata about classification, not raw PII                                | Store as boolean/enum list                       |
| Confidence Score        | Local Model     | Keep             | Pipeline quality metric                                                   | Store as float                                   |
| Pipeline Version        | System          | Keep             | Auditability and reproducibility                                          | Store as string                                  |
| Raw Email Headers       | Email           | Drop             | Contains routing info, IPs, and potential PII                             | Discard after ingestion                          |
| Teams Meeting Transcript| Teams           | Redact           | Contains names, decisions, and sensitive discussions                      | LLM redaction + semantic masking                 |
| Read Receipts           | Email/Teams     | Drop             | Low analytical value, high privacy risk                                   | Discard after ingestion                          |
| API Pagination Tokens   | Teams/Email     | Drop             | Ephemeral, no analytical value, potential security risk                   | Discard after use                                |
| Synthetic Seed          | Generator       | Keep             | Reproducibility for benchmarking                                          | Store as hash/integer                            |
| Redaction Mask          | Pipeline        | Keep             | Audit trail for verification                                              | Store as structured JSON                         |
| Owner Dashboard Export  | Aggregator      | Keep             | Actionable metrics for home lab owner                                     | Export as encrypted CSV/JSON                     |
| Public Benchmark Export | Aggregator      | Keep             | Safe for sharing, uses synthetic/redacted data                            | Export as sanitized JSON                         |

This table ensures every data field is explicitly categorized, with clear handling methods that align with privacy, analytical utility, and benchmark requirements. The pipeline enforces these decisions at ingestion, classification, and export stages, guaranteeing that raw private content never leaves the local environment while maintaining a highly useful reporting plane for the home lab owner.