Sr. Databrick Developer

Job Description: Azure Databricks Developer

Position Overview:
We are seeking a skilled Azure Databricks Developer to join our team. As an Azure Databricks Developer, you will be responsible for designing, developing, and maintaining data processing workflows and analytics solutions using Azure Databricks. Your primary focus will be on leveraging Databricks features and capabilities to build scalable and efficient data pipelines and perform data transformations. You will collaborate with cross-functional teams to deliver high-quality data solutions that meet business requirements.

Responsibilities:
- Design, develop, and deploy data processing workflows and data analytics solutions using Azure Databricks.
- Build and maintain scalable and efficient data pipelines, data ingestion, and data transformation processes.
- Collaborate with data engineers, data scientists, and other stakeholders to understand business requirements and translate them into technical specifications.
- Implement and optimize ETL/ELT processes for data extraction, cleansing, and loading from various sources into Databricks.
- Develop and maintain data models and perform data validation and testing to ensure data quality and accuracy.
- Conduct performance tuning and optimization of Databricks workloads to improve overall system efficiency.
- Monitor and troubleshoot data processing workflows, identify issues, and propose solutions to ensure the reliability and integrity of data.
- Stay up to date with the latest advancements in Azure Databricks and related technologies, and recommend innovative solutions to enhance data processing capabilities.

Qualifications:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Solid experience working with Azure Databricks, including cluster management, notebook development, and job scheduling.
- Proficiency in programming languages such as Python, SQL, and Scala, with the ability to write efficient and optimized code.
- Strong understanding of distributed computing principles and experience with big data processing frameworks like Spark.
- Hands-on experience with data integration, ETL/ELT processes, and data warehousing concepts.
- Familiarity with Azure services such as Azure Data Lake Storage, Azure SQL Database, Azure Event Hubs, and Azure Cosmos DB.
- Knowledge of data modeling and data visualization techniques.
- Experience with version control systems, CI/CD pipelines, and agile development methodologies.
- Strong problem-solving and analytical skills with the ability to troubleshoot and resolve complex technical issues.
- Excellent communication and collaboration skills to work effectively in cross-functional teams.

Preferred Qualifications:
- Relevant certifications in Azure or Databricks.
- Experience with machine learning frameworks and data science workflows.
- Familiarity with streaming data processing using technologies like Apache Kafka or Azure Event Hubs.
- Knowledge of data governance, security, and compliance practices in Azure.

As required by the equal pay and transparency acts, the expected base salary for this position is $120k-$150k + Bonus+ Benefits

The expected pay is determined based on a variety of factors including, but not limited to, depth of experience in the practice area. Employees are eligible to participate in Hitachi Vantara's bonus/variable/commission pay programs, where applicable, and are subject to the program's conditions and restrictions.

Our Company

Hitachi Vantara is part of the Global Hitachi family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society.

Our people are our biggest asset, they drive our innovation advantage and we strive to offer a flexible and collaborative workplace where they can thrive. Diversity of thought is welcomed and our employee base is represented by several active Employee Resource Group communities. We offer industry leading benefits packages (flexible working, generous pension and private healthcare) and promote a creative and inclusive culture. If driving real change gives you a sense of pride and you are passionate about powering social good, we'd love to hear from you.

Our Values

We strive to create an inclusive environment for all and are open to considering home working, compressed/flexible hours and flexible arrangements. Get in touch with us to explore how we might be able to accommodate your specific needs.

We are proud to say we are an equal opportunity employer and welcome all applicants for employment without attention to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status. With Japanese roots going back over 100 years, our culture is founded on the values of our parent company expressed as the Hitachi Spirit:

Wa - Harmony, Trust, Respect

Makoto - Sincerity, Fairness, Honesty, Integrity

Kaitakusha-Seishin - Pioneering Spirit, Challenge