Responsibilities
  • Desi...">
    Back to Jobs

    Senior Data Engineer

    Remote, USA Full-time Posted 2025-07-27











    Responsibilities


    • Design, build, and optimize ETL pipelines using AWS Glue 3.0+ and PySpark.

    • Implement scalable and secure data lakes using Amazon S3, following bronze/silver/gold zoning.

    • Write performant SQL using AWS Athena (Presto) with CTEs, window functions, and aggregations.

    • Take full ownership from ingestion → transformation → validation → metadata → documentation → dashboard-ready output.

    • Build pipelines that are not just performant, but audit-ready and metadata-rich from the first version.

    • Integrate classification tags and ownership metadata into all columns using AWS Glue Catalog tagging conventions.

    • Ensure no pipeline moves to QA or BI team without validation logs and field-level metadata completed.

    • Develop job orchestration workflows using AWS Step Functions integrated with EventBridge or CloudWatch.

    • Manage schemas and metadata using AWS Glue Data Catalog.

    • Take full ownership from ingestion → transformation → validation → metadata → documentation → dashboard-ready output.

    • Ensure no pipeline moves to QA or BI team without validation logs and field-level metadata completed.

    • Enforce data quality using Great Expectations, with checks for null %, ranges, and referential rules.

    • Ensure data lineage with OpenMetadata or Amundsen and add metadata classifications (e.g., PII, KPIs).

    • Collaborate with data scientists on ML pipelines, handling JSON/Parquet I/O and feature engineering.

    • Must understand how to prepare flattened, filterable datasets for BI tools like Sigma, Power BI, or Tableau.

    • Interpret business metrics such as forecasted revenue, margin trends, occupancy/utilization, and volatility.

    • Work with consultants, QA, and business teams to finalize KPIs and logic.

    • Build pipelines that are not just performant, but audit-ready and metadata-rich from the first version.

    • Integrate classification tags and ownership metadata into all columns using AWS Glue Catalog tagging conventions.

    • This is not just a coding role. We expect the candidate to think like a data architect within their module – designing pipelines that scale, handle exceptions, and align to evolving KPIs.

























    Essential Skills
    Job


    • Strong hands-on experience with AWS: Glue, S3, Athena, Step Functions, EventBridge, CloudWatch, Glue Data Catalog.

    • Programming skills in Python 3.x, PySpark, and SQL (Athena/Presto).

    • Proficient with Pandas and NumPy for data wrangling, feature extraction, and time series slicing.

    • Strong command over data governance tools like Great Expectations, OpenMetadata / Amundsen.

    • Familiarity with tagging sensitive metadata (PII, KPIs, model inputs).

    • Capable of creating audit logs for QA and rejected data.

    • Experience in feature engineering – rolling averages, deltas, and time-window tagging.

    • BI-readiness with Sigma, with exposure to Power BI / Tableau (nice to have).

    Personal


    • Excellent communication and collaboration skills with data scientists, QA, and business users.

    • Self-starter with strong problem-solving and critical thinking abilities.

    • Ability to translate business KPIs and domain requirements into technical implementations.

    • Detail-oriented with a high standard of data quality and compliance.

    • Demonstrated accountability, confidentiality, and ethical standards in handling data.























    Preferred Skills
    Job


    • Experience in domains such as logistics, supply chain, enterprise finance, or B2B analytics.

    • Knowledge of ML pipelines and I/O formats like Parquet/JSON.

    • Familiarity with data modeling and KPI interpretation.

    • Working knowledge of Agile methodology.

    Personal


    • Ability to multitask and handle fast-paced delivery environments

    • Proactive mindset with a collaborative and mentoring approach

    • Demonstrates ownership, initiative, and a commitment to high performance














    Other Relevant Information


    • Bachelor's or master’s degree in computer science, Data Engineering, or a related field

    • Minimum 6–9 years of hands-on data engineering with at least 3 years in AWS-native data pipelines and governance


    • Success in This Role Means:

    • You ship production-grade pipelines with built-in data quality and lineage.

    • You reduce QA overhead by catching edge cases early through code.

    • You create pipelines that are explainable to BI users, architects, and QA without extra walkthroughs.

    • You’re the go-to person in your module for business logic clarity and data accuracy.

    • This role offers the flexibility of working remotely in India.




    LeewayHertz is an equal opportunity employer and does not discriminate based on race, color, religion, sex, age, disability, national origin, sexual orientation, gender identity, or any other protected status. We encourage a diverse range of applicants.

    Apply To This Job

    Apply Now

    Similar Jobs

    Sr Staff Machine Learning Engineer, GenAI

    Remote, USA Full-time

    Staff Product Manager, Observability

    Remote, USA Full-time

    Business Development Representative

    Remote, USA Full-time

    Copywriter

    Remote, USA Full-time

    Collections Account Manager

    Remote, USA Full-time

    Virtual Assistant - Appointment Setter

    Remote, USA Full-time

    Human Resource Coordinator

    Remote, USA Full-time

    US Real Estate Financial Admin

    Remote, USA Full-time

    Director, Strategic Marketing

    Remote, USA Full-time

    Customer Support Representatives

    Remote, USA Full-time

    Blue Cross blue shield Careers Remote Jobs

    Remote, USA Full-time

    LPN(Hire on Bonus!) Paonia Colorado Location

    Remote, USA Full-time

    Data Analyst (Entry Level)

    Remote, USA Full-time

    Pre-Sales Solutions Engineer

    Remote, USA Full-time

    [Remote-Position] Marketing Jobs Southwest Airlines

    Remote, USA Full-time

    Business Client Relationship Manager

    Remote, USA Full-time

    Manager, Brand Management, Gold Peak

    Remote, USA Full-time

    Urgently Require Instructional Assistant Professor - Public Health in Mississippi

    Remote, USA Full-time

    Senior Web Developer

    Remote, USA Full-time

    Part-Time Delta Airlines Data Entry Remote Jobs: Earn $/Hour - WFH

    Remote, USA Full-time

    Find the best remote jobs in USA - employelevatenetwfh