Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

ZWlTZm5ua05aTHd6UHgyRWtIT3lqZmFhV2c9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

Somewhere

Graphic Designer/ Illustrator Job at Somewhere

 ...Graphic Designer & Illustrator Location : 100% Remote - Open for LATAM Region Hours : Full Time preferred (Part Time Option Possible) Salary Range : $1,800 - $2,500 Based on Experience Type of Contrac t: Independent Contractor Agreeement About The Role... 

Trilogy

Event Producer, IgniteTech (Remote) - $60,000/year USD Job at Trilogy

Step into the future of event production where AI meets precision and creativity. IgniteTech is redefining how events and communities are run...  ...tools. About IgniteTech If you want to work hard at a company where you can grow and be a part of a dynamic team, join... 

Compass Group

Senior Marketing Manager - University of Idaho Job at Compass Group

 ...Location: University of Idaho Salary: $80,000 - $85,000 At over 300 college & universities around the country, Chartwells Higher...  ...from you. Job Summary Bring Campus to Life Through Creativity Become Our Next Marketing Manager at the University of Idaho... 

MobileHRnow LLC

SHIP REPAIR SUPERINTENDENT Job at MobileHRnow LLC

 ...various stakeholders like crew, ship owners, and regulatory bodies. Responsibilities Manage a workforce on-board customer ships, barges, and tugboats for maintenance and repairs. Communicate with Project Managers to develop plans and time frames for efficient... 

Wyandotte Municipal Services

Data Entry Clerk & Office Assistant - US Remote Job at Wyandotte Municipal Services

 ...Position Overview: We are seeking motivated individuals to join our team remotely as Data Entry Clerks & Office Assistants. This flexible role is ideal for candidates at all experience levels who are looking to work from home. Whether you're starting your career or bringing...