1 to 25 of 241 Remote Apache Spark Jobs

Highly competitive Status: Inside IR35 Working pattern: Full time We are seeking an experienced Technical Delivery Lead with strong expertise in Python, Databricks, and Apache Spark to lead the successful delivery of enterprise-scale data engineering initiatives.. This is a hands-on leadership role combining technical delivery, solution … Technical Delivery Lead position. Strong hands-on expertise with Python. Extensive experience with Databricks, including Workflows, Notebooks, and Delta Lake. Strong experience with Apache Spark/PySpark. Proven track record building and optimising large-scale ETL/ELT pipelines. Strong understanding of modern data architecture and Lakehouse principles. ...

designing and delivering modern data platforms, working closely with clients and technical teams to build scalable, secure and high-performing solutions using Databricks, Apache Spark and cloud technologies. Key Responsibilities Design and architect Databricks-based data platforms and solutions. Build scalable data lakehouse and data warehouse architectures. Design … Databricks and modern data platform delivery. Skills & Experience Required 7+ years' experience in Data Architecture or Solution Architecture. Strong commercial experience with Databricks and Apache Spark . Experience delivering enterprise data platforms on AWS, Azure or GCP. Strong understanding of data warehousing, data lakes and lakehouse architectures. Experience ...

designing and delivering modern data platforms, working closely with clients and technical teams to build scalable, secure and high-performing solutions using Databricks, Apache Spark and cloud technologies. You'll also play a key role in pre-sales activities, solution design, and helping to drive strategic Databricks initiatives … Databricks and modern data platform delivery. Skills & Experience Required 7+ years' experience in Data Architecture or Solution Architecture. Strong commercial experience with Databricks and Apache Spark . Experience delivering enterprise data platforms on AWS, Azure or GCP. Strong understanding of data warehousing, data lakes and lakehouse architectures. Experience ...

data solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Python and Databricks Build and optimise ETL/ELT workflows leveraging Databricks (Spark, Delta Lake) Develop robust data models and data architectures on Lakehouse platforms Implement and manage data workflows within the Databricks ecosystem Ensure high data … Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/ ...

Mentor data engineers and other team members who want to grow towards architecture roles. Required technical skills & knowledge: Azure databricks, Microsoft Fabric, Power BI, Apache Spark, Azure Synapse, Azure Data Lake, Azure Data Factory, Python, PySpark, SQL Your profile You don't need to tick every ...

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines and lakehouse solutions … time. Your Role: Design, develop, and maintain scalable data pipelines using Databricks on AWS. Build and optimize data processing frameworks using PySpark, Spark SQL, Python, Delta Lake, and Databricks notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such ...

optimising ETL/ELT pipelines. Strong understanding of Lakehouse, Data Warehouse, and Data Lake architectures. Experience with Azure Data Factory, Azure Synapse, and Apache Spark. Knowledge of data modelling, semantic models, and enterprise reporting solutions. Experience implementing data governance, security, and quality frameworks. Strong understanding of modern data engineering ...

data models and ETL processes, to support the construction of scalable and efficient data systems. Familiarity with big data tools like DuckDB, Polars or Apache Spark for processing out-of-memory datasets" . Proven experience collaborating with first-team stakeholders within a professional sports environment, demonstrating strong interpersonal ...

concept to production. Strong knowledge of Python and core ML libraries (e.g., spaCy, NumPy, SciPy, Transformers). Understanding of big data and cloud technologies (Spark, Hadoop, Azure, AWS, or GCP). Deep expertise in MLOps principles, scalable model deployment, and model governance. Excellent communication and stakeholder management skills with ...

prem environments. Hard Skills Experience in data engineering, ideally aerospace/defence or industrial sectors. ETL/ELT tools (Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). Strong Python and SQL; C++, C#, or MATLAB a plus. Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg, TimescaleDB. Data Platform Capabilities ...

What the role entails: Some of the main duties of the Data Engineer will include: Develop high-performance data pipelines using Python (Pandas) or Spark within Palantir Foundry, ensuring seamless data transformation and integration to support analytics, reporting, and machine learning use cases across the enterprise. Design and implement ...

Oracle). Comprehensive knowledge of European and international gas and power markets, covering physical and financial trading, forward/futures/options, basis risk, spark and dark spreads, gas balancing, power scheduling, nominations, LNG and cross-commodity optimisation. Proven track record of defining and delivering enterprise-level architecture ...

Oracle). Comprehensive knowledge of European and international gas and power markets, covering physical and financial trading, forward/futures/options, basis risk, spark and dark spreads, gas balancing, power scheduling, nominations, LNG and cross-commodity optimisation. Proven track record of defining and delivering enterprise-level architecture ...

with Azure Cosmos DB, including query optimization, throughput management, and performance tuning. Thorough understanding of Cosmos DB Change Data Feed (CDF) and integration with Spark-based solutions. Experience working with Cosmos DB SDKs in technologies such as .NET, Java, Python, or Node.js. Strong understanding of distributed computing concepts … Apache Spark architecture. Hands-on experience developing data pipelines using PySpark. Experience with Pytest and automated testing frameworks. Strong understanding of partitioning, indexing strategies, and consistency models within Cosmos DB. Experience building scalable applications and working with concurrency patterns. Experience with Git, version control systems, and CI/ ...

sophisticated proofs‐of‐concept and public cloud platform(s). Coding expertise in a core programming language (Python, Java, Scala). Foundational understanding of Apache Spark architecture is preferable; hands‐on skills will be advantageous. Key Notes for the role: Location for the role needs to be within ...

concept and public cloud platform(s). Coding expertise in a core programming language (i.e., Python, Java, Scala). A foundational understanding of Apache Spark architecture is preferable; hands‐on skills will be advantageous for the role. We will be testing your ability to utilise AI‐assisted tools ...

Experience with Agentic and Gen AI projects. Experience with energy system modelling, grid analytics, demand forecasting or energy market data. Familiarity with distributed computing, Spark, PySpark or cloud-native data platforms. Experience developing models for planning, scenario analysis or operational decision support. Knowledge of data governance, model risk management ...

based applications. Expert level knowledge in python is desired . Solid experience with SQL-based database is required. Two year experience with Hadoop/Spark in either an administrative, development or support role is required. Hands-on experience with AWS or Azure. Candidate must possess excellent English written … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter ,LinkedIn and Facebook . Our Commitment to Diversity and Inclusion ...

modern data engineering best practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence ...

experience with the SQL Server platform and T-SQL querying. Familiarity with cloud data platforms (e.g., AWS, Azure) and big data technologies (e.g., Kafka, Spark, Databricks, Snowflake). Expert-level knowledge of building ETL processes in Python. Expert-level knowledge of performance tuning and optimization in a Python/ ...

working with databases or data warehouses Experience building and maintaining data pipelines for large datasets Hands-on experience with big data tools such as Spark, Hadoop, or Hive Experience working with cloud platforms such as AWS, Azure, or Databricks Exposure to machine learning frameworks and deploying or optimising models ...

working with databases or data warehouses Experience building and maintaining data pipelines for large datasets Hands-on experience with big data tools such as Spark, Hadoop, or Hive Experience working with cloud platforms such as AWS, Azure, or Databricks Exposure to machine learning frameworks and deploying or optimising models ...

working with databases or data warehouses Experience building and maintaining data pipelines for large datasets Hands-on experience with big data tools such as Spark, Hadoop, or Hive Experience working with cloud platforms such as AWS, Azure, or Databricks Exposure to machine learning frameworks and deploying or optimising models ...

working with databases or data warehouses Experience building and maintaining data pipelines for large datasets Hands-on experience with big data tools such as Spark, Hadoop, or Hive Experience working with cloud platforms such as AWS, Azure, or Databricks Exposure to machine learning frameworks and deploying or optimising models ...

working with databases or data warehouses Experience building and maintaining data pipelines for large datasets Hands-on experience with big data tools such as Spark, Hadoop, or Hive Experience working with cloud platforms such as AWS, Azure, or Databricks Exposure to machine learning frameworks and deploying or optimising models ...