We are looking for a Big Data Engineer to join a huge travel company based in central London. This contract will run until the end of the year, where you will be part of the client's HCOM Analytics platform team and will be responsible for:
- Setting up and maintaining Big data analytics applications in cloud such as Qubole, R server and more
- Developing tools to automate usage monitoring, system health, executing upgrades and building DR for these applications
- Help users with training and best practices to write efficient queries and processes to analyse the data in the most efficient and performant manner
- Evaluating new tools
· Application support experience
· Linux/Unix Operating Systems
· Experience working with AWS platform (Understanding of VPC, Security Groups, Networking)
· Experience with deploying applications & tools on AWS & administration of tools (Managing Users, groups and Permissions)
· Experience with building, testing and deploying tools in Development & Production
· Experience with building monitoring around tools availability, usage and performance
· Experience with executing platform upgrades, patches and back-up
· Java, SQL, & Scripting (Python or Shell) knowledge
· Terraform experience would be a plus
· Knowledge of multiple cloud platforms (AWS / Google) would be a plus
· Excellent communication
Big Data specific:
· Experience with configuring, optimising and supporting Hadoop platform
· Experience with writing and optimising Hive & Spark queries and jobs
· Knowledge of configuring R server would be a plus
· Design, develop, debug, and modifies components of software applications and tools. Drive technical direction at the application/product level for small to mid-sized projects.
· Design, spec, schedule and provide quality implementation of a component or feature (typically 1-3 months in duration).
· Provide alternative solutions to a given problem.
· Resolve problems and roadblocks, with management assistance if required.
· Follow through on details and drive issues to closure.
· Complete documentation and procedures for installation and maintenance.
· Actively participate in group technology reviews to critique work of self and others.
· Work collaboratively with all members of technical staff.
· Collaborate with program management and testing peers in the development of assigned components.
· Participates in and provides input to requirements definition.
· Usually provides solid judgement calls and trade- off analysis.