Senior Big Data System Engineer - 165728
Our team maintain and develop a big data cyber monitoring platform which ensures the information security for a UK critical national infrastructure customer. Comprised of Elasticsearch and HDFS data lakes and built on a Linux estate, the platform supports a suite of cutting-edge cyber security and data science tools to ensure protection from cyber threats. The team uses DevOps methodology to deliver continuous improvements that ensure the platform meets rapidly changing security requirements.
In this role you will be a key driver in improving our customer’s cyber capability. The customer will look to you as a technical expert for a range of activities and you will have the opportunity to develop your skills in many areas. You must be a self-motivated individual, with the ability to make changes that improve our services and will be involved in short, medium and long term strategy for this customer and wider BT security policy.
You'll have the following responsibilities
To aide in the continuous development of the platform, actively looking to enhance functionality and usability
To work closely with a team of security analysts to develop and improve security use cases
To develop a big data streaming pipeline in Apache Nifi or Logstash
To work on continuous integration and development toolsets (Jenkins) to deploy new features
Maintain an up to date knowledge of the cyber security field and apply this to ensure our customer’s infrastructure remains protected
You'll have the following skills & experience
Able to creatively apply a wide range of technical and/management principles
Developing data science solutions to big data and cyber security problems
Linux operating systems skills (patching, support, development)
Knowledge of Elasticsearch (ELK) and Hadoop big data systems. Experience of Cyber Monitoring tools (SIEMs); Code automation tools (Puppet, Ansible) and Scripting skills (Bash, Python, Java)
Knowledge or experience with CI/CD tools (Jenkins), large scale ETL using Apache Nifi or Pentaho
Ability to build relationships with senior customers and stakeholders
Qualified to a degree level or with significant technical experience
This role is a unique opportunity to work in a challenging, fast paced environment where you can bring your own experience and expertise to make a difference to our customers. In return you’ll learn from a highly skilled team and develop your skills in big data applications, data science and cyber security toolsets.
Working Monday to Friday within our core operating hours of 08.00-17.00, there is local flexibility on start/finish times, with a 9 day fortnight being based on our customer site. (And one day a fortnight based either at home or other office location)
You will be expected to obtain and maintain a MOD DV clearance, so if you don’t currently have one, we will need to undertake pre security checks before starting the role with us.