Senior Data Engineer
About Hitachi Consulting
Hitachi Consulting is the digital solutions and professional
services business of Hitachi Ltd., a global technology leader. Hitachi
Consulting is a catalyst for positive business change, propelling organizations
to drive innovation and to accelerate digital transformation. We have particular
strengths in the domains of ‘Digital’, ‘Internet of Things’, ‘Analytics’ and
ERP and provide a range of supporting services (industry expertise, project
management, process design, change management) required to achieve success.
Hitachi Consulting helps organizations leverage data as a
strategic asset to develop new revenue streams, innovate faster and respond to
global dynamics with insight and agility. We collaborate with clients to create
solutions that help maximize operational efficiency and deliver measurable,
sustainable business results.
As a Senior Data Engineer you will be responsible for designing, developing, and optimizing data warehouses, data pipelines, and infrastructure to drive intelligent applications and products as part of Hitachi Consulting and for our clients. The Senior Data Engineer will work closely with Hitachi and client personnel including product owners, data scientists, other architects and developers, project/program managers, and executive leadership. This is a highly technical position, but also requires strong communication and team leadership skills.
The Senior Data Engineer will be responsible for:
- Designing optimal Big Data/ Data Warehouse data pipeline/orchestration architectures - on-premise or within cloud environments
- Constructing new data warehouse or Big Data platforms, primarily in cloud infrastructure, and developing pipelines, ETL and analytical processes to support products and solutions.
- Production of advanced analytical solutions, including models, pipelines, and infrastructure to deliver real-time insights, predictions or prescriptions within our customers’ operations.
- Developing strategies, policies and best practices regarding Big Data and Cloud data acquisition, integration and analytics. Preparing technical design and implementation documents where needed.
- Work with product manager/product owner and development teams to ensure all project requirements are met.
- Provides project team leadership and mentoring.
- Hands-on experience in designing and implementing scalable, performant data platforms, and ability to deliver proof of concepts using a variety of tools/platforms including the Hadoop ecosystem (Hadoop, Hbase, Hive, Nifi, Spark, Livy).
- Experience working with Cloud environments and databases (GCP, Azure, AWS and associated Cloud data platforms/services such as Azure Data Lake, AWS Aurora, AWS RDS, Snowflake, Cosmos, Redbricks, etc.).
- Thorough understanding of Big Data, and execution experience with Enterprise Data Warehouses (EDWs) and knowledge of BI/DW solutions.
- Experience developing and tuning ETL processes and utilizing industry-standard ETL tools.
- Experience constructing and using more modern Data Pipelining and Data Orchestration tools or services such as (Azure Data Factory, Azure Databricks, Apache Airflow/Nifi, Pentaho, etc.).
- Advanced working SQL knowledge and experience working with relational databases, as well as a variety of databases including graph databases such as Neo4j, Cosmos.
- Familiarity with data-oriented scripting languages: shell, Python/R/Java/Scala/Perl.
- Working knowledge of message queuing and stream processing
- Familiarity working with big data formats (Avro, Parquet, Orc, etc.).
- Experience productionizing data models and pipelines.
- Behavioral competencies:
- Positive, winning attitude.
- Passion for continual learning.
- Excellent track record of customer responsiveness and customer interaction skills.
- Communication Skills: strong presentation, and oral and written
- Operates with a sense of urgency.
- Upholds honesty and integrity.
- Ability to work in a collaborative, agile environment:
- Data Engineers will be members of and work on a Balanced Team or Scrum Team (including Product Managers, Designers, Software Engineers, and Data Scientists).
- Products/ Solutions are generally developed in an Agile Scrum environment.
- Familiarity with data modeling approaches such as ERD and concepts behind relational, normalized and star schema database architectures.
- Structured design, coding and testing experience
- Experience in DevOps (Team City, Azure DevOps)
- Experience working in Google Cloud Platform
- Experience with Message Brokers (RabbitMQ, Kafka, Pub/Sub)
- Experience working on, optimizing, and tuning performance of common databases: Oracle, MySQL, Postgres, Any NoSQL.
- Experience working in Professional Services / Consulting on high impact, data-oriented projects with smaller, independent project teams.
- Experience with BI Reporting and Analytical Tools such as Tableau, Business Objects, SAS, Cognos, etc.
- Experience working with multiple operating systems, Linux/Unix in particular.
- VMware or Virtualization technology experience
- Periodic travel is required to meet client needs and timetables
- Ability to be stationed at and work from an out-of-town client site for short periods of time
Inclusion and Diversity
Hitachi Consulting is an inclusive Equal Employment Opportunity employer that considers applicants without regard to gender, gender identity, sexual orientation, race, religion, disability, nationality, age, ethnicity, disabled or veteran status, or any other characteristic protected by the law.
Hitachi Consulting is committed to building an inclusive work environment that leverages the diverse talent and perspectives of our global workforce. We value the unique background, experience and capability of each employee and embrace differences. As a global company, we understand, appreciate and respect the diversity of our people and foster a collaborative work environment with equal opportunity for all employees. Inclusion and diversity are a fundamental strength of Hitachi Consulting that adds value for our company, our clients, our suppliers and the communities in which we live and work.
If you need a reasonable accommodation to apply for a job at Hitachi, please send the nature of request and contact information firstname.lastname@example.org.
Hitachi Consulting offers excellent benefits and career growth to all employees!
Hitachi Consulting is an Equal Opportunity Employer M/F/D/V
If you need a reasonable accommodation to apply for a job at Hitachi, please send the nature of request and contact information email@example.com. Queries other than accommodation requests will not be responded to.