Data Engineer
A Data Engineer at Confidential develops high-scale data platforms, managing and processing tens of TB of data daily using Spark, Airflow, and Kubernetes.
About the role
Join an Elite Data Engineering Unit! A leading technology company is looking for a brilliant Data Engineer to develop high-scale platforms and technologies. This is a unique opportunity to join a high-performance team dealing with massive data scales, distributed systems, and complex architectural challenges.
What you’ll do:
In this role, you will be involved in developing platforms that process data at a massive scale.
- Scale & Processing: Design, build, and deploy solutions to manage and process tens of TB a day (both stream and batch processing).
- Infrastructure: Build multi-region, distributed, fully automated, and monitored environments. Manage big data cluster plans for products.
- Optimization: Identify, design, and implement process improvements—automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
- Data Modeling & Quality: Design, build, and launch new data models and ETL (extraction, transformation, loading) processes in production. You will own data quality for your allocated areas.
- Ownership: Lead the design for new products and initiatives while integrating with currently implemented best practices.
Requirements:
- Experience: 4+ years of experience working as a Data Engineer.
- Big Data Stack: Experience working on big data with distributed computing (Spark, Airflow, Kubernetes) - Must.
- Cloud: Strong experience orchestrating cloud services (Amazon EC2).
- Architecture: Knowledge of distributed in-memory architecture using cache (Memcached, Redis, etc.).
- Skills: Passionate about data with strong analytical and technical skills.
Advantages:
- Experience with Python or Scala.
- Background in building CI/CD pipelines for data.
Perks:
- Work with an "Elite unit" on cutting-edge data challenges.
- Opportunity to handle massive scale (Petabytes of data).
- Dynamic and technical work environment.
If you are a data enthusiast who loves building robust systems at scale, this is your next challenge.
Apply today and power the next generation of data infrastructure!
Frequently Asked Questions
What will I do as a Data Engineer at Confidential?
As a Data Engineer, you will develop high-scale data platforms, design data models, build ETL processes, and manage big data clusters, processing tens of TB of data daily.
What skills and experience are required for this Data Engineer role?
You need 4+ years of data engineering experience, expertise in big data with distributed computing (Spark, Airflow, Kubernetes), and strong experience with cloud services like Amazon EC2.
What are the perks of working as a Data Engineer at Confidential?
You'll work in an "Elite unit" on cutting-edge data challenges, handle massive data scales (Petabytes), and enjoy a dynamic technical environment. Experience with Python/Scala is a plus.
Ready to apply?
Attach your CV and we’ll review your profile. If there’s a strong match, we’ll reach out.