Big Data Engineer

Website European Recruitment

Responsibilities:
▪ Ingest the data captured from various sensors as well as from external world into the big data ecosystem
▪ Perform the data quality checks on the input data and publish the results
▪ Transform the data into various ecosystem formats as required by various AD feature teams
▪ Transform the data and suitably encrypt it to make it available to external world
▪ Perform extensive data search on various aspects of data quality, data transformation, data storage, data security, data access and so on
▪ Perform the data backup and archival activities
▪ Monitor and tune the performance of the individual module and overall system
▪ Work on end-to-end automation of various work flows
▪ Adhere to various best practice and standards

Required Skills:
▪ Experience with Hadoop and its ecosystem in design, development and implementation.
▪ Strong experience in working on Apache spark using Scala.
▪ Insights into data search (Solr/ Elasticsearch) and data processing in Hadoop/ Big data environment.
▪ Writing high-performance, reliable and maintainable code in languages like Scala and Python.
▪ Track record of working on perform analysis of vast data stores and uncover insights while maintaining the security and data privacy.
▪ Exposure to automotive domain will be an added advantage.

Desirable skills:
▪ A computer science or equivalent degree from a reputed institute
▪ At least 4-5 years of experience in the big data domain
▪ Ability to mentor and lead a small team
▪ Proven ability to learn and adopt new technology
▪ Proficient understanding of code versioning tools like Git, build tool like Maven, CI/CD tool like Zuul or Jenkins and Ansible
▪ Knowledge of Agile methodology for delivering software solutions

 

Keywords: Bog Data / Hadoop / Embedded / Apache spark / Scala / Python / Git / Maven / CI/CD / Zuul / Jenkins / Ansible / Agile / Automotive 

 

By applying to this role you understand that we may collect your personal data and store and process it on our systems. For more information please see our Privacy Notice https://eu-recruit.com/about-us/privacy-notice/

Categories: Automotive

  Salary: Negotiable

  Location: Munich, Germany

  Job type: Permanent, Full Time