76 to 100 of 249 Apache Spark Jobs in London

hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested ...

mesh concepts and their practical implementation within large organisations Experience integrating Databricks with AWS services including S3, IAM, VPC, and AWS Glue Familiarity with Apache Spark, dbt, data pipelines, and modern analytics architectures Understanding of access management, audit frameworks, and enterprise governance controls Delivery Experience Experience leading data ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

security Practical cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial ...

years of strong experience in Python development Experience with system design and leading multi-person development efforts Experience with distributed data processing frameworks like Spark Experience with building scalable APIs, and software applications w/focus on backend infrastructure Experience with cloud platforms and services for data storage ...

with experience in frameworks such as PyTorch, TensorFlow and Hugging Face Experience using data platforms and big data frameworks like Databricks and Spark Excellent communication skills for engaging C-level stakeholders and aligning technical outcomes with business value Nice to have Experience applying AI in drug discovery, safety monitoring ...

platforms. One or more Oracle certifications across OCI, Data Management, AI Services or Autonomous Database. Experience working with complementary technologies such as Python, Spark, Databricks, Azure, AWS, OpenShift, Kubernetes or MLOps platforms. Demonstrated ability to structure client problems, shape solutions and identify opportunities for further value creation. Ability ...

including data quality, data lineage, model testing, prompt testing, access controls, monitoring, explainability, human oversight and governance. Strong data analytics capability, including Python, SQL, Spark, Power BI, Tableau or equivalent tools. Ability to translate complex technical concepts into practical business risks, control implications and clear audit conclusions. Internal Audit ...

production. Experience working with large, complex and varied datasets. Experience with modern cloud and Data Science technologies such as AWS, Azure, Databricks, Snowflake, Spark or similar. Strong stakeholder management skills, with the ability to explain complex technical concepts clearly and influence decision-making. Strong commercial awareness and the ability ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … distributed data processing and performance tuning Hands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … distributed data processing and performance tuning Hands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … distributed data processing and performance tuning Hands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake and engines such as Spark, Flink, or Trino and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

Engineering Strong experience working with AWS in data engineering environments. Hands‐on experience with AWS Glue for data integration and processing. Strong experience using Apache Airflow for workflow orchestration and pipeline management. Advanced Python development skills. Experience with Apache Spark or similar distributed data processing technologies would ...

e.g., Java, TypeScript, SQL) Experience building and consuming APIs and event‐driven services in a cloud environment Experience with data processing frameworks such as Apache Spark and working with large structured datasets Working knowledge of LLM‐based application development, including prompting, tool calling, retrieval, and evaluation Experience using … agent frameworks (e.g., Google ADK, LangGraph) and agent protocols such as MCP, A2A, or AG-UI Experience with Databricks, MLflow, and Delta Lake or Apache Iceberg Exposure to model serving on Kubernetes and to model monitoring (drift, latency, accuracy) Familiarity with CRM platforms such as Salesforce and their APIs ...

Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform ...

frontier AI models. Ideal candidates are hands-on Data Engineers with strong SQL and Python, ETL/ELT, orchestration, streaming, and warehousing experience across Spark, Kafka, Airflow, dbt, Snowflake, BigQuery or Databricks on AWS, GCP or Azure. #J-18808-Ljbffr ...

tools, and emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated ...

/ML offerings. Strong programming skills (Python, Java, C++) and experience with DevOps practices (CI/CD). Knowledge of big data technologies (Hadoop, Spark, Flink). Knowledge of NoSQL databases (MongoDB, Cassandra, DynamoDB), message queueing systems (Kafka, RabbitMQ), and version control systems (Git). Preferred Skills: Hugging Face ...

security Practical cloud‐native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands‐on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial ...

auction systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

Company description Publicis Media harnesses the power of modern media through leading agency brands Dysrupt, Infinite Roar, Publicis Collective, Publicis Health Media, Spark Foundry, Starcom and Zenith, as well as global accelerator PMX; immersive experience group PMCI and access to integrated platform-based technologies and offerings from CJ, Epsilon ...

work. Technologies and Tools Python Machine learning frameworks (e.g. PyTorch, TensorFlow, scikit-learn) ML/MLOps tooling (e.g. SageMaker, MLflow, TFServing) Spark and Databricks AWS services (e.g. IAM, S3, Redis, ECS) CI/CD tooling and best practices Streaming and batch data systems (e.g. Kafka, Airflow, RabbitMQ) Additional Information ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything- spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

/GraphQL) Machine Learning: Foundations of MLflow, Weights & Biases, LoRA, PEFT, SageMaker, Vertex AI, AzureML, Kubeflow, Airflow, Prefect, TorchServe, Triton, vLLM, SHAP, LIME, pandas, Spark, SQL LLM tooling: Familiarity with LangChain, Semantic Kernel, prompt chaining, vector search, and context management Performance & observability: Skilled in debugging distributed systems, tuning ...

processes, and the adoption of new tools and frameworks. What you need to succeed: Python, SQL and a distributed framework like Spark; we use Databricks. Experience with marketing or attribution data, and a working understanding of how bidding platforms consume the signal you send them. You don't need ...