Join our Talent Network

Skip to main content

Corporate Information  |  Careers

Careers Home > Job Search Results > Big-data Engineer and Administrator

Big-data Engineer and Administrator

This job posting is no longer active.

Location: Lisbon Portugal
Job ID: 1018875HV
Date Posted: Oct 26, 2022
Segment: IT
Business Unit: Hitachi Vantara
Company Name: Hitachi Vantara Corporation
Profession (Job Category): General Management

Share: mail
Save Job Saved

Meet our Team

We represent Hitachi Vantara to enterprise clients across industries, establishing business relationships to understand customer challenges so that we can deliver profitable business for Hitachi products, services and solutions. We collaborate as a team and cross-functionally to ensure the success of our customers; success that is celebrated and shared. Our solutions bring value to every line of business and we need people like you to build those deep relationships and to passionately articulate our value proposition.

Job Responsibilities:
  • Participate in developing high load MapReduce platform hosted on-premise and AWS.
  • Participate in architectural decision making on on-premise migration to AWS EMR
  • Be ready to assist in integration of new data streaming sources (like REST API, SFTP flat files)
  • Direct communication with the customer

What you bring to the team
  • 3+ years in a Big-data Engineer role;
  • Sound knowledge of Hadoop Architecture and distributed data environments;
  • Excellent functional knowledge and experience with big data components such as Kafka, Spark,Hadoop, Hbase, Hive;
  • Experience in optimizing hadoop/hive settings for maximum throughput (includes compaction; hive metadata; TeZ Containerization;);
  • Experience in shell scripting, have knowledge of scripting languages such as Python and be exposed to programming languages such as Java;
  • Knowledge of Table definitions, file formats, UDF, Data Layout (Partitions and Buckets);
  • Monitor Cluster performance, resource usage, backup and mirroring, provisioning and automation (Administration);
  • ProdOps experience on Storm/Kafka.
  • Experience in analyzing MapReduce jobs logs and error handling;
  • AWS/Cloud: Sound knowledge of Cloud basics: S3, EMR. Need experience in identifying opportunities to improve processes, operations and automation using IaC;
  • Experience ingesting data from multiple data sources such as REST API, SFTP flat files, Streaming data using Apache Storm, Kafka Streams or others etc.

  • Ability to work under US East Coast schedule

Will be a plus:
  • Degree in Computer Science, IT, or similar field; a Master's is a plus;
  • Data engineering certification (e.g AWS Certified Big Data - Specialty) is a plus;
  • Experience with Agile methodology

Our Company

Hitachi Vantara is part of the Global Hitachi family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society.

Our people are our biggest asset, they drive our innovation advantage and we strive to offer a flexible and collaborative workplace where they can thrive. Diversity of thought is welcomed and our employee base is represented by several active Employee Resource Group communities. We offer industry leading benefits packages (flexible working, generous pension and private healthcare) and promote a creative and inclusive culture. If driving real change gives you a sense of pride and you are passionate about powering social good, we'd love to hear from you.

Our Values

We strive to create an inclusive environment for all and are open to considering home working, compressed/flexible hours and flexible arrangements. Get in touch with us to explore how we might be able to accommodate your specific needs.

We are proud to say we are an equal opportunity employer and welcome all applicants for employment without attention to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status. With Japanese roots going back over 100 years, our culture is founded on the values of our parent company expressed as the Hitachi Spirit:

Wa - Harmony, Trust, Respect

Makoto - Sincerity, Fairness, Honesty, Integrity

Kaitakusha-Seishin - Pioneering Spirit, Challenge
Share: mail

Similar Jobs

Tech BI Business Analyst

Hitachi Vantara

Senior BI Developer

Hitachi Vantara

Data Architect

Hitachi Vantara

SQL Developer

Hitachi Vantara

Devops Engineer

Hitachi Vantara

Senior BI Developer

Hitachi Vantara

Product Manager

Hitachi Vantara

Fullstack/ FE Developer

Hitachi Vantara

Azure Cloud Engineer

Hitachi Vantara

Senior .NET Developer

Hitachi Vantara