401 to 425 of 569 Apache Spark Jobs in the UK

/column-level security, lineage and audit Build reusable Terraform modules for Databricks and automate platform changes through CI/CD pipelines Tune Spark, Delta and Photon workloads to improve performance and reduce DBU and cloud spend Manage identity, SSO/SCIM, service principals, secret scopes and network isolation … experience as a Data Platform Engineer, Cloud Engineeror Data Engineer, with hands-on Databricks production experience Strong knowledge of the Databricks Lakehouse, Delta Lake, ApacheSpark and Unity Catalog Production experience administering Databricks on at least onemajor cloud (AWS, Azure or GCP), including networking, IAM and storage Proficiency with Terraform (ideally ...

designing and building robust, scalable, distributed data systems and pipelines, using open source and public cloud technologies. Strong experience with data orchestration tools: e.g. Apache Airflow, Dagster. Experience with big data storage and processing technologies: e.g. DBT, Spark, SQL, Athena/Trino, Redshift, Snowflake, RDBMSs (PostgreSQL/MySQL … . Knowledge of event‐driven architectures and streaming technologies: e.g. Apache Kafka, Kafka Streams, Apache Flink. Experience with public cloud environments: e.g. AWS, GCP, Azure, Terraform. Strong knowledge of software engineering practices: e.g. testing, CI/CD (Jenkins, Github Actions), agile development, git/version control, containers etc. ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits We strive to provide comprehensive benefits and perks that meet the needs of all our employees. ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive ...

Barlowe LLP is seeking an engineer to build and operate distributed analytics platforms in London. The role involves using technologies such as Spark, Kafka, and AWS to power research and trading teams. The ideal candidate will have strong experience with data analytics systems, automation skills in Python ...

lead embedded data science teams, partnering effectively with senior stakeholders across multiple domains Hands‐on technical expertise with tools such as Databricks, Python, Spark, and GCP/BigQuery Engineering mindset, with experience moving teams toward machine learning engineering best practice Credibility to lead long‐tenured individual contributors while providing ...

doing something you are passionate about. My client is based in LondonFollow me on Twitter for more roles and news @SebFintechSytems ENgineer – Security, Hadoop, SPark, debian, SELinux, Debian, puppet, ruby , python, kerberos ...

programming, hardware-aware optimization, or FPGA-adjacent systems. Experience with ML/data tooling such as PyTorch, JAX, CUDA, GPU clusters, Ray, Kafka, Kubernetes, Spark, or comparable technologies. Understanding of market microstructure, backtesting pitfalls, transaction costs, and the operational needs of live quantitative strategies. Experience in environments operating real ...

role you should have the following skills: Experience implementingdata processes for Data Warehouse and production systems. Strong experience withBig Data technologies(e.g.,Hadoop/Spark). Programming in modern languages (e.g.,ScalaandPython). Strong delivery engineering tooling:Git/GitHub,Jenkins,Ansible, andJIRAfor requirement/backlog management. Strong understanding ...

with stakeholders. Preferred qualifications, capabilities, and skills Experience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray) Experience of big data technologies (e.g. Spark, Hadoop) Have constructed batch and streaming microservices exposed as REST/gRPC endpoints Familiarity with GraphQL We recognize that our people are our strength ...

programming, hardware-aware optimization, or FPGA-adjacent systems. Experience with ML/data tooling such as PyTorch, JAX, CUDA, GPU clusters, Ray, Kafka, Kubernetes, Spark, or comparable technologies. Understanding of market microstructure, backtesting pitfalls, transaction costs, and the operational needs of live quantitative strategies. Experience in environments operating real ...

trust with stakeholders. Preferred qualifications, capabilities, and skillsExperience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray)Experience of big data technologies (e.g. Spark, Hadoop)Have constructed batch and streaming microservices exposed as REST/gRPC endpointsFamiliarity with GraphQL#CIBAppliedAIJ.P. Morgan is a global leader in financial services, providing ...

with stakeholders. Preferred qualifications, capabilities, and skills Experience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray) Experience of big data technologies (e.g. Spark, Hadoop) Have constructed batch and streaming microservices exposed as REST/gRPC endpoints Familiarity with GraphQL #CIBAppliedAI ABOUT US J.P. Morgan is a global ...

programming, hardware-aware optimization, or FPGA-adjacent systems. Experience with ML/data tooling such as PyTorch, JAX, CUDA, GPU clusters, Ray, Kafka, Kubernetes, Spark, or comparable technologies. Understanding of market microstructure, backtesting pitfalls, transaction costs, and the operational needs of live quantitative strategies. Experience in environments operating real ...

leadership teams. Minimum Qualifications Masters in Statistics, Mathematics, Data Science, Machine Learning, Physics, Engineering, Computer Science or equivalent Proficiency in data querying using SQL, Spark, or equivalent technologies Proficiency in scripting languages for data processing and analysis (ex: Python, R, or Scala) Excellent communication and multi-functional collaboration skills ...

roles. Prior experience using causal inference/econometric techniques and bayesian methodologies for hypothesis testing. Prior experience using large datasets with distributed computing (e.g. spark, hadoop or other map-reduce tech)Experience working in a fast-moving tech company or startup. This role is a full-time role based ...

roles. Prior experience using causal inference/econometric techniques and bayesian methodologies for hypothesis testing. Prior experience using large datasets with distributed computing (e.g. spark, hadoop or other map-reduce tech)Experience working in a fast-moving tech company or startup. This role is a full-time role based ...

insights that drive the AI product roadmap forward. Tech Stack Includes: AWS: EC2, Athena, S3, Kinesis, Lambda, Step Functions, RDS, QuickSight Languages & Frameworks: Python, Spark, TensorFlow, LLMs & LLM coding tools Databases & Search: MySQL, PostgreSQL, SOLR Experience: Minimum 3 years of commercial experience in a data science or machine learning ...

systems like Cassandra, Elasticsearch, Kafka, and more across different environmentsTechnologies We UseDifferent backend languages, including Java, Rust, and GoOpen-source technologies like Cassandra, ElasticSearch, Spark, Kafka, Kubernetes, FlinkIndustry-standard build tooling, including Gradle and GitHubWhat We ValueDemonstrated ability to collaborate and empathise with a variety of individuals. Able ...

PyTorch or TensorFlow) Demonstrated experience designing and analyzing controlled experiments (A/B testing, causal inference, hypothesis testing) Experience working with distributed data systems (Spark, Ray, Dask, or similar) Proficiency in SQL and working with large-scale structured datasets Experience building, deploying, and maintaining predictive models in production environments ...

bachelor's degree in a relevant field or equivalent practical experience.Useful, but not required• Event streaming or real-time processing with Kafka, Kinesis, Flink, Spark Structured Streaming, or similar technologies.• Distributed processing with Spark and familiarity with Parquet, Avro, JSON, and open table formats.• Search or retrieval systems ...

have else we will be enable you with the required trainingAWS Glue and Data Catalog, EMR, Athena, S3, Redshift/RDS, Kafka/Spark, AWS Security, IAMOR Azure Data factory, Azure Data bricks, Azure Data Lake, Azure SQL DW & Azure SQL, Azure Storage, HDInsight (Spark/Kafka),Azure … Event Hub, Azure AD/IAM and Data CatalogORCloud DataFlow, DataPrep, CloudSQL, BigTable, BigQuery, Cloud Spanner, Kafka/Spark, Google Cloud IAMWhy should you apply? For top talent we offer a highly competitive salary and a number of other benefits: Bonus programmeMedical and dental careLife InsuranceChildcare VouchersPension schemeFriendly work ...

safe scaling patterns and reuse. Preferred qualifications, capabilities, and skillsExperience with modern data platforms such as Databricks or SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks ...

driven technologies such as Kafka, Flink, Kinesis, and Pub/Sub Orchestrate workflows using Airflow, Dataflow, and Glue Process and transform large datasets using Spark and Flink Design production-ready systems Work across relational, NoSQL, and analytical data stores Optimise storage formats and access patterns Implement secure, compliant data … code, providing technical direction and mentorship Improve data quality, testing, observability, and operational excellence Требования Strong Python and SQL skills Deep experience with Spark and modern data platforms such as Databricks and Snowflake Solid understanding of cloud data services in AWS or GCP Demonstrated ownership of large-scale data ...

build and enhance our Databricks platform, ensuring it remains secure, scalable and reliable. Develop and maintain batch and real-time data pipelines using Databricks, Spark and Delta Lake. Contribute to platform architecture, engineering standards and best practices across the data estate. Collaborate with Product, Analytics and Data Science teams … improvement across the engineering team. Our ideal candidate Proven experience building and supporting scalable, production-grade data platforms. Strong hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL. Good understanding of cloud-based data engineering, ideally within Azure environments. Experience working with Infrastructure as Code tools such ...