Big Data Architect Contractor
Job Title - Big Data Architect
Location – Dallas, TX (100% remote)
Duration – 4 months with possible extension
- Job Description -As a Big Data Architect Contractor, you will support the project team by designing and implementing large-scale data solutions to meet business needs. Design and develop scalable big data architectures to handle large volumes of data. Collaborate with stakeholders to understand data requirements and translate them into technical solutions. Implement data integration, data processing, and data storage solutions using big data technologies. Ensure data security, data quality, and data governance standards are met. Optimize data architectures for performance, scalability, and cost-efficiency.
- Platform Management: Install, configure, and upgrade HVR (now Fivetran HVR) software, including the Hub, agents (HVAs), and associated components, across source and target systems.
- Connection & Replication Setup: Establish and manage data source connections for diverse databases (e.g., Oracle, SQL Server, SAP HANA, PostgreSQL, MySQL) and cloud platforms (e.g., Snowflake, Databricks, BigQuery, Redshift). Configure robust replication channels, defining source/target locations, selecting tables, and setting up data transformations.
- Performance Optimization: Fine-tune HVR configurations to maximize data replication performance, including adjusting batch sizes, parallelization, and network settings. Proactively identify and address system capacity bottlenecks (e.g., storage, CPU, memory, network bandwidth) by recommending improvements.
- High-Volume CDC Implementation: Develop and execute strategies for high-volume data replication and efficient log-based Change Data Capture (CDC).
- Operational Health: Ensure HVR agents are running optimally and maintaining reliable connectivity to all source and target endpoints. Key Skills & Qualifications - Candidates must have:
- HVR/Fivetran Expertise: Minimum of 5 years of hands-on experience with HVR (or Fivetran HVR) administration, configuration, and advanced troubleshooting.
- Database Proficiency: Strong command of various database technologies (e.g., Oracle, SQL Server, SAP HANA, PostgreSQL, MySQL) and their respective Change Data Capture (CDC) mechanisms.
- Cloud Data Integration: Proven experience with major cloud platforms (AWS, Azure, GCP) and cloud data warehouses (Snowflake, BigQuery, Redshift).
- Data Concepts: Solid understanding of data warehousing concepts, ETL/ELT processes, and common data integration patterns.
- Problem-Solving: Excellent analytical and problem-solving abilities to diagnose and resolve complex data flow issues.
- Communication: Strong communication and collaboration skills to work effectively with diverse technical and business teams.
- Automation: Scripting proficiency (e.g., Shell scripting, Python) for automating tasks and enhancing operational efficiency.
- Networking & Security: Fundamental knowledge of network configurations and security principles as they apply to data replication.
Apply Job!
Apply to this Job