1 to 25 of 77 Remote Apache Spark Jobs

Active IRS - MBI is Required (badging could take 1-6 months without) 5+ years of Data Engineer/Software Development experience 5+ years Python and Java 5+ years with Apache Spark (Spark SQL, Spark Streaming, data transformations, performance optimization, and tuning) 5+ years experience with cloud-based data platforms (Databricks or AWS EMR) Proficiency with Git More ❯

What you'll do Lead the design of scalable, secure data architectures on AWS. Build and optimise ETL/ELT pipelines for batch and streaming data. Deploy and manage Apache Spark jobs on Databricks and Delta Lake. Write production-grade Python and SQL for large-scale data transformations. Drive data quality, governance, and automation through CI/CD … scientists, analysts, and business stakeholders. Mentor and guide data engineering teams. What we're looking for Proven experience in senior/principal data engineering roles. Expertise in AWS, Databricks, Apache Spark, Python, and SQL . Strong background in cloud-native data platforms, real-time processing, and data lakes. Hands-on experience with tools such as Airflow, Kafka, Docker More ❯

Salary: 50.000 - 60.000 € per year Requirements: • 3+ years of hands-on experience as a Data Engineer working with Databricks and Apache Spark • Strong programming skills in Python, with experience in data manipulation libraries (e.g., PySpark, Spark SQL) • Experience with core components of the Databricks ecosystem: Databricks Workflows, Unity Catalog, and Delta Live Tables • Solid understanding of data … our growing Data & AI team and work hands-on with the Databricks platform, supporting clients in solving complex data challenges. • Designing, developing, and maintaining robust data pipelines using Databricks, Spark, and Python • Building efficient and scalable ETL processes to ingest, transform, and load data from various sources (databases, APIs, streaming platforms) into cloud-based data lakes and warehouses • Leveraging … and cloud technologies to continuously improve tools and approaches Technologies: AI AWS Azure CI/CD Cloud Databricks DevOps ETL GCP Support Machine Learning Power BI Python PySpark SQL Spark Terraform Unity GameDev Looker SAP More: NETCONOMY has grown over the past 20 years from a startup to a 500-people team working across 10 European locations, with a More ❯

a Senior Data Engineer, Tech Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in Python, Pyspark, Scala or Java. Experience operating More ❯

/medical devices preferred but not required) Strong Python programming and data engineering skills (Pandas, PySpark, Dask) Proficiency with databases (SQL/NoSQL), ETL processes, and modern data frameworks (Apache Spark, Airflow, Kafka) Solid experience with cloud platforms (AWS, GCP, or Azure) and CI/CD for data pipelines Understanding of data privacy and healthcare compliance (GDPR, HIPAA More ❯

managing data lakes and optimizing data workflows. Preferred Qualifications Experience with Master Data Management (MDM) tools or Microsoft Fabric. Knowledge of additional data technologies such as SQL, NoSQL databases, Apache Spark, Kafka, ML with Python. Prior experience in regulated industries (e.g., finance, education, healthcare) with a focus on data compliance. Certifications in cloud architecture (e.g., AWS Certified Solutions More ❯

databases (SQL Server, MySQL) and NoSQL solutions (MongoDB, Cassandra) Hands-on knowledge of AWS S3 and associated big data services Extensive experience with big data technologies including Hadoop and Spark for large-scale dataset processing Deep understanding of data security frameworks, encryption protocols, access management and regulatory compliance Proven track record building automated, scalable ETL frameworks and data pipeline More ❯

Passion for data with extensive knowledge and experience in Machine Learning techniques. Expertise in key technologies related to Data Management. Proficiency in Python is required; knowledge of SQL and Spark is a plus. Experience with Cloud platforms, specifically Azure and Databricks. In-depth knowledge and experience in Data Analytics Architecture. Understanding of Data Governance processes and platforms. Experience with More ❯

a fast-paced, collaborative environment. Familiarity with NiFi, Airflow, or other data pipeline orchestration tools (not required, nice to have). Exposure to Kubernetes, IAM/security models, or Spark/Kafka environments (not required, nice to have). Working knowledge of Python or ability to collaborate on multi-language microservice systems (not required, nice to have). Prior More ❯

you require: Degree Qualification in Computer Science, Engineering or related field Proven experience as a Data Engineer Strong proficiency in Python and experience with data manipulation frameworks such as Apache Spark In-depth knowledge of relational and non-relational databases, data modelling, and SQL Experience with cloud platforms including Azure Synapse and Fabric Proficiency in designing and implementing More ❯

Big Data & Cloud Infrastructure Knowledge of building and deploying cloud based applications Hands-on experience with cloud data platforms (AWS/GCP/Azure) Proficiency with big data technologies (Spark, Kafka, or similar streaming platforms) Experience with data warehouses (Snowflake, BigQuery, Redshift) and data lakes Knowledge of containerization (Docker/Kubernetes) and infrastructure as code Preferred Experience Experience building More ❯

we’d love you to have... Understanding of cloud computing security concepts Experience in relational cloud based database technologies like Snowflake, BigQuery, Redshift Experience in open source technologies like Spark, Kafka, Beam Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in an agile environment Here’s a taste of More ❯

we’d love you to have... Understanding of cloud computing security concepts Experience in relational cloud based database technologies like Snowflake, BigQuery, Redshift Experience in open source technologies like Spark, Kafka, Beam Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in an agile environment Here’s a taste of More ❯

e.g., Azure Data Factory, Synapse, Databricks, Fabric) Data warehousing and lakehouse design ETL/ELT pipelines SQL, Python for data manipulation and machine learning Big Data frameworks (e.g., Hadoop, Spark) Data visualisation (e.g., Power BI) Understanding of statistical analysis and predictive modelling Experience: 5+ years working with Microsoft data platforms 5+ years in a customer-facing consulting or professional More ❯

e.g., Azure Data Factory, Synapse, Databricks, Fabric) Data warehousing and lakehouse design ETL/ELT pipelines SQL, Python for data manipulation and machine learning Big Data frameworks (e.g., Hadoop, Spark) Data visualisation (e.g., Power BI) Understanding of statistical analysis and predictive modelling Experience: 5+ years working with Microsoft data platforms 5+ years in a customer-facing consulting or professional More ❯

e.g., Azure Data Factory, Synapse, Databricks, Fabric) Data warehousing and lakehouse design ETL/ELT pipelines SQL, Python for data manipulation and machine learning Big Data frameworks (e.g., Hadoop, Spark) Data visualisation (e.g., Power BI) Understanding of statistical analysis and predictive modelling Experience: 5+ years working with Microsoft data platforms 5+ years in a customer-facing consulting or professional More ❯

exposure to cloud-native data infrastructures (Databricks, Snowflake) especially in AWS environments is a plus Experience in building and maintaining batch and streaming data pipelines using Kafka, Airflow, or Spark Familiarity with governance frameworks, access controls (RBAC), and implementation of pseudonymisation and retention policies Exposure to enabling GenAI and ML workloads by preparing model-ready and vector-optimised datasets More ❯

with excellent collaboration skills. Grit in the face of technical obstacles. Nice to have: Building SDKs or client libraries to support API consumption. Knowledge of distributed data processing frameworks (Spark, Dask). Understanding of GPU orchestration and optimization in Kubernetes. Familiarity with MLOps and ML model lifecycle pipelines. Experience with AI model training and fine-tuning. Familiarity with event More ❯

source systems into our reporting solutions. Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools like Apache Spark to ensure seamless data flow. Monitoring and Failure Recovery: Implement monitoring procedures to detect failures or unusual data profiles and establish recovery processes to maintain data integrity. … in Azure data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Lake Storage/One Lake, and Azure Data Factory. Understanding of data extraction from vendor REST APIs. Spark/Pyspark or Python skills a bonus or a willingness to develop these skills. Experience with monitoring and failure recovery in data pipelines. Excellent problem-solving skills and attention More ❯

best practices for data security and compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse and ETL/ELT development in More ❯

cloud technologies and modern engineering practices. ● Experience with the following technologies: o Cloud Provider: AWS o Languages: Python, PHP, Rust & SQL o Hosting: Kubernetes o Tooling & Analytics: Airflow, RabbitMQ, Apache Spark, PowerBI ● Proven ability to complete projects according to outlined scope, budget, and timeline ● Experience with industry standard tools such as Microsoft products, Jira, confluence, project management tools More ❯

cloud technologies and modern engineering practices. ● Experience with the following technologies: o Cloud Provider: AWS o Languages: Python, PHP, Rust & SQL o Hosting: Kubernetes o Tooling & Analytics: Airflow, RabbitMQ, Apache Spark, PowerBI ● Proven ability to complete projects according to outlined scope, budget, and timeline ● Experience with industry standard tools such as Microsoft products, Jira, confluence, project management tools More ❯

data infrastructure or data platforms, with proven ability to solve complex distributed systems challenges independently Expertise in large-scale data processing pipelines (batch and streaming) using technologies such as Spark, Kafka, Flink, or Beam Experience designing and implementing large-scale data storage systems (feature stores, timeseries databases, warehouses, or object stores) Strong distributed systems and infrastructure skills (Kubernetes, Terraform More ❯

data infrastructure or data platforms, with proven ability to solve complex distributed systems challenges independently Expertise in large-scale data processing pipelines (batch and streaming) using technologies such as Spark, Kafka, Flink, or Beam Experience designing and implementing large-scale data storage systems (feature stores, timeseries databases, warehouses, or object stores) Strong distributed systems and infrastructure skills (Kubernetes, Terraform More ❯

data infrastructure or data platforms, with proven ability to solve complex distributed systems challenges independently Expertise in large-scale data processing pipelines (batch and streaming) using technologies such as Spark, Kafka, Flink, or Beam Experience designing and implementing large-scale data storage systems (feature stores, timeseries databases, warehouses, or object stores) Strong distributed systems and infrastructure skills (Kubernetes, Terraform More ❯