Job Overview:
We are searching for a seasoned Data Engineer with significant expertise in designing data pipelines and warehouses in cloud environments, particularly on Google Cloud Platform. The ideal candidate should be well-versed in handling large-scale datasets, optimizing data flows, and implementing cloud-based data solutions.
Primary Responsibilities:
Architect and develop data integration workflows to handle diverse, large data sets using cloud-based solutions.
Build and maintain efficient data models and optimize data processing pipelines for performance and scalability.
Collaborate with teams located both onsite and offshore to ensure smooth delivery of data projects.
Utilize Google Cloud Platform services such as BigQuery, Dataflow, and Pub/Sub to create secure and scalable data infrastructures.
Embrace test-driven development practices to ensure code quality and reliability.
Engage in CI/CD processes with tools like GitHub and Jenkins to automate testing and deployment.
Develop custom scripts and applications using Java or Python to extend the capabilities of Google Cloud Platform, leveraging its SDKs and APIs.
Write complex SQL queries to transform and analyze data, ensuring efficiency across various use cases.
Required Skills and Experience:
A minimum of 8 years working with data engineering, including ETL/ELT frameworks and large-scale data warehouse architectures.
At least 5 years focused on data modeling and building robust pipelines for processing massive data volumes.
Proficient in Google Cloud products, particularly BigQuery and Dataflow, with hands-on experience in implementation.
Strong ability to write optimized SQL and work with either Python or Java for automating tasks.
Solid understanding of Agile methodologies and DevOps practices.
Bonus Qualifications:
Google Cloud Professional Data Engineer certification is highly desirable.
Knowledge of NoSQL databases and their integration within cloud environments.
Experience in the retail sector would be a significant asset.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job...ShalePro Energy Services, LLC is a natural gas service company. ShalePro Energy Services offers a full suite of best-in-class oil and natural gas field services spanning the Appalachian Basin - Marcellus/Utica shale play, as well as basins across the US. We have more than...
...training, and assist the company's cross-departmental collaboration4. Regularly feedback students' situation with teaching and research tutors and local supervisors; Cooperate with the teaching and research team to compile various types of course outline, assist the...
...May make field changes that do notalter the scope and cost of construction. May schedule and observe contractors materials, structure... ...required. We are an E-Verify employer. We will provide the Social Security Administration (SSA) and, if necessary, the Department of...
...Inc., is an established growing company looking for a motivated Truck Drivers. Pay is dependent on experience. Paid vacation, insurance... ...Machinery has grown from the ground up into a full-service heavy/highway and civil site construction contractor comprised of over...
...Job purpose: The Office Assistant will assist with daily office operations, including call center, accounting, and audit functions. They will assist with a smooth, productive working environment that enables us to service our clients with the highest level of quality...