7 months ago
Unfortunately this job has now expired. However you can view all of our Live jobs here.
A financial institution is looking to recruit a Senior Data Engineer to be responsible for the development and optismisation of the collection and data flow of the organisation.
You will be expected to build relationships and collaborate with stakeholders at all levels across business, operations and technology teams.
Duties and responsibilities
Building of data sets and driving business decisions through the use of analytical and statistical approach
Designing, developing, testing and supporting data collection and integration with batched ETL and Real-time data processing
Writing scalable code for managing Batch and Real-time data processing, using onsite/cloud tools and storage
Improving existing batch based ETLs
Helping in maintaining the existing infrastructure and its migration
Maintaining records and documentation
Building analytical tools that utilise the data pipeline
Have a high interest in keeping up to date with latest best practices in the industry
Skills and Experience
Experience in designing and implementing data streaming solutions involving technologies such as Kafka, NiFi and Spark Streaming
Strong analytical and troubleshooting skills working in a Linux environment
Experience with Linux based scripting tools such as shell and bash
Experience with programming languages like Java and Python
Good knowledge of cloud technologies and their tools in relation to data transformation
Good knowledge of PostgreSQL streaming replication and RHEL based Linux
Familiarity with the use of Talend and Nifi
Familiarity with data structures such as JSON, XML and CSV
Familiarity with data architecture and SQL querying on Oracle, MS SQL, PostgreSQL and HIVE
Excellent verbal and written communication skills in English