Data Engineering Copilot listing changes

How this plugin's listing changed from one daily snapshot to the next, with its rank by placement that day and a day, a week and four weeks later. Rank movement after a change shows what happened next, not what caused it; the directory-wide page compares it with every other plugin.

6 changes since November 2, 2026, newest first. A dash means no rank was recorded that day; “due” means the day hasn't come yet.

Field	Change	Age	Rank by placement	Then	Category rank, 7d
Nov 2 → Nov 3 · 6 changes
Capabilities	Added: Architect batch, CDC, streaming, lakehouse and warehouse data systems Debug duplicates, data loss, latency, failures and stale pipelines from evidence Plan safe backfills, replay, checkpoint recovery and rollback strategies Review CDC merge ordering, delete propagation and idempotent writes Diagnose Spark and Databricks query plans, shuffle, skew, spill and file layout Design Microsoft Fabric Lakehouse, OneLake, shortcuts and deployment flows Show 6 moreShow fewer Design dbt/SQL transformations, model grain, incremental updates and contracts Review Airflow orchestration, retries, scheduling and backfill safety Build data quality, freshness, reconciliation and observability checks Assess access control, governance, PII handling and data lineage Coordinate supported specialist agents and evidence-based verification Prepare version-aware implementation plans and release/incident evidence Removed: Design reliable batch, CDC, streaming, and lakehouse architectures Debug failed, slow, duplicated, stale, or backlogged production pipelines Plan safe replay, backfill, recovery, and checkpoint strategies Review Delta MERGE logic for keys, ordering, duplicates, updates, and deletes Diagnose Spark and Databricks performance using plans, shuffle, spill, skew, and file evidence Design Structured Streaming state, watermark, checkpoint, and sink behavior Show 4 moreShow fewer Create production-oriented SQL, PySpark, Databricks, and Airflow patterns Build data-quality gates, reconciliation, freshness checks, and operational runbooks Evaluate retry safety, idempotency, schema evolution, and downstream contracts Use current official docs for Spark, Databricks, Delta Lake, Airflow, and runtime-specific behavior	30–90 d	#5475	1d#5514 39places down 7ddue Nov 10 28ddue Dec 1	#272 due Nov 10
Description	617 chars 718 chars rewritten, 27% kept · added: build, optimize, dependable, platforms, work, apache +4 more · removed: engineering, copilot, helps, improve, airflow, observability +4 more
Skills	Added: build-data-pipeline debug-data-pipeline design-cdc-pipeline design-data-quality design-fabric-lakehouse design-orchestration Show 11 moreShow fewer design-streaming design-warehouse-dbt modernize-data-platform observe-data-platform orchestrate-data-agents plan-backfill-replay plan-data-platform respond-data-incident review-data-model secure-data-platform tune-spark-databricks
Keywords	Added: fabric onelake dbt
Version	0.1.0 0.2.0
One-liner	Debug production data systems Build reliable data pipelines