Java + Big Data Tech Lead

The Company

Hitachi Vantara, a wholly-owned subsidiary of Hitachi, Ltd., guides our customers from what's now to what's next by solving their digital challenges. Working alongside each customer, we apply our unmatched industrial and digital capabilities to their data and applications to benefit both business and society. More than 80% of the Fortune 100 trust Hitachi Vantara to help them develop new revenue streams, unlock competitive advantages, lower costs, enhance customer experiences, and deliver social and environmental value.

The Role

Total Experience - 12-14 Years

Must Have:

1. Basics of Distributed computing
   1. MapReduce
   2. Distributed computing vs RDBMS/ scale up vs scale out
2. Hands on experience in any one of the programming languages (Java, Python, Scala)
3. Understanding of Linux and Bash scripting
4. Knowledge of SQL
5. Basics of Hadoop framework, problem patterns that can be solved like filtering, aggregation, joins etc
6. Understanding of Spark concepts like RDD, Dataframes, Clousures etc., has implemented at least one project using Spark and Scala
7. Should have worked on at least 1-2 bigdata projects (Could be ingestion, ETL processing) on the Cloudera Platform
8. Understanding of Hive/Pig, concepts like partitioning, bucketing, metastore, schema on read vs schema on write, SerDe
9. Solid programming fundaments /design concepts.
10. In depth understanding of different batch and stream processing technologies and NoSQL storage
11. Demonstrated work experience as an Sr.Developer/ Jr. Architect role in Bigdata/Cloud and opensource technology stack.
12. Should be able to articulate, suggest right use of technology stack for different use cases with reasoning.
13. Understanding of Lambda, Kappa architecture
14. Should have participated or able to suggest right hardware choices, platform components, distributions etc.

Good to Have:

1. Programming concepts
   1. Object oriented vs Functional programming concepts
   2. Design patterns (Singleton, Immutable, Factory)
2. MapReduce Programming like Combiner, Partitioiner, InputFormat/OutputFormat, Serialization
3. Distributed Computing
   1. Scale up vs Scale out
4. Scala hands on, SparkSQL, dataframes etc.
5. Understanding of different storage formats Avro, RCFile, ORC, Parquet
6. Has worked/working on any one of the cloud platform AWS, Azure, GCP
7. Has worked/working on any one of the bigdata platforms like Hortonworks, Cloudera, Datastacks, Databricks
8. Aware of latest technology trends in streaming, real-time, batch processing frameworks (Storm, Apache Beam, Flink, Spark, Kafka Connect etc)
9. Certified in any of the bigdata distribution (Hortonworks/Cloduera/Databricks/Datastacks)

We are an equal opportunity employer. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.