Naveed Mohiuddin

Benda Infotech — Remote, US

• Build cloud data pipelines across AWS, Azure, and Databricks covering batch ingestion, transformation, validation, and reporting use cases — working across both stacks depending on where the source systems and consumers live.
• On AWS, develop Lambda and EventBridge-based ingestion into S3 with clearly separated raw and curated zones, designed so any load can be traced back to its source file and reprocessed in isolation without touching downstream data.
• On Azure, build ADF and Databricks pipelines using Delta Lake medallion layers (Bronze/Silver/Gold) for incremental ingestion, transformation, and curated reporting tables that analysts query directly.
• Write PySpark and Spark SQL logic for data cleaning, validation, joins, reconciliations, and curated table creation — standardizing types and applying business rules before data reaches reporting layers.
• Use Delta MERGE INTO alongside deterministic S3 object naming so that reruns are safe by design, which cut down the duplicate-load issues that previously required manual cleanup.
• Tune Athena and Delta workloads using columnar Parquet, partitioning, partition pruning, OPTIMIZE, and Z-ORDER to reduce both query latency and bytes scanned per query.
• Add record-count, load-completion, and quarantine checks at layer boundaries so failed or incomplete loads are caught in the pipeline rather than discovered by an analyst looking at a broken dashboard.
• Manage orchestration through Airflow and MWAA alongside ADF and Databricks Workflows, configuring retries, alerting, task dependencies, and parameterized backfill for reprocessing historical windows.
• Build Athena, Databricks SQL, QuickSight, and Power BI-ready datasets for operational and business reporting, and partner with analysts to translate reporting requirements into source-to-target mappings, validation rules, and reusable data models.

Applied Information Sciences — Hyderabad, India

• Built and supported Azure-based data pipelines for policy, claims, and operational data spanning ADF, Databricks, Snowflake, and SQL-based source systems.
• Developed ADF ingestion pipelines pulling from SQL Server, Oracle, REST APIs, and flat files into ADLS Gen2 and Snowflake, handling the schema and connectivity differences each source type brought with it.
• Used control-table driven pipeline patterns so onboarding a new source became largely a configuration change rather than net-new ADF development, which cut repeated build work across the team.
• Implemented full-load and incremental extraction using watermark columns, runtime parameters, and validation checkpoints so pipelines could run reliably on a schedule without manual date handling.
• Wrote PySpark, Spark SQL, and SQL transformations to standardize schemas, apply validation rules, and prepare curated datasets for the Snowflake warehouse.
• Built reconciliation queries comparing source and target counts, checking load timestamps, and surfacing duplicate or missing records ahead of reporting cycles — catching issues before business users did.
• Investigated production failures across ADF, Databricks, Snowflake, and Splunk, tracing root cause through logs and dashboards, then executing controlled reruns and verifying downstream tables were correct.
• Maintained runbooks covering recurring failure patterns, validation steps, rerun procedures, and escalation paths so on-call handoffs didn't depend on tribal knowledge.
• Worked with business analysts, QA, and support teams to confirm reported data issues, validate fixes, and keep reporting cycles moving during month-end crunch periods.