51 to 75 of 374 PySpark Jobs in England

storage Proficiency with Terraform (ideally the Databricks provider) and Gitbased CI/CD workflows Solid programming skills in Python and SQL; comfortable with PySpark and shell scripting Experience tuning Spark jobs and Delta tables (partitioning, Working knowledge of SSO/SCIM, identity providers and secretsmanagement Understanding of data security ...

best practices, including DevOps, DataOps, and MLOps, will be advantageousWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such ...

hand over to Evoke teams Collaborate daily with Evoke ML engineering, the CF team, and AWS ProServe counterparts Skills & experience Must-have Python/PySpark - Expert. Production data pipeline development, not analysis-only AWS data/ML stack - Advanced. S3, Glue and/or EMR, IAM basics ...

hand over to Evoke teams Collaborate daily with Evoke ML engineering, the CF team, and AWS ProServe counterparts Skills & experience Must-have Python/PySpark - Expert. Production data pipeline development, not analysis-only AWS data/ML stack - Advanced. S3, Glue and/or EMR, IAM basics ...

hand over to Evoke teams Collaborate daily with Evoke ML engineering, the CF team, and AWS ProServe counterparts Skills & experience Must-have Python/PySpark - Expert. Production data pipeline development, not analysis-only AWS data/ML stack - Advanced. S3, Glue and/or EMR, IAM basics ...

with both technical and non-technical audiences Nice to have: Production experience specifically in Microsoft Fabric Power BI, particularly Direct Lake mode Python/PySpark Data governance or security experience Experience enabling AI/ML workloads from a data platform, not just BI The details: Salary ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...

Need to Be Successful Deep, hands‐on experience building and running production data platforms and products A good level of experience in Python (particularly pyspark) for building data pipelines Strong AWS and Terraform experience, with infrastructure‐as‐code as your default way of working rather than something ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...

Star Schema concepts (Fact and Dimension tables), from the ground up. Git: You're fluent in Git workflows — branching, troubleshooting, code reviews. Python/PySpark/SQL: You write clean, efficient code across all three, and you understand how it performs at scale. Database management: Experience with SQL/ ...

Informatica with thorough understanding of data transformation and orchestration processes Extensive Experience with AWS services including IAM Lambda EKS S3 Data Lake EMR PySpark MWAA and Lakehouse technologies, Apache Iceberg Proven expertise with Snowflake including performance tuning and efficient query design Experience with batch pipelines eg StreamSets and streaming ...

Strong hands‐on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud experience across ...

particularly interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. Experience working with large ...

particularly interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. Experience working with large ...

data feed pipeline migrations repointing from an existing Data Warehouse to a new AWS Data Lakehouse architecture, underpinned with AI ETL accelerators, Python/PySpark, SQL and YAML configs.Test initial bulk then deltas from legacy to new.Execute automated test suites as part of sprint-based delivery and regression cyclesDefect ...

Experience of implementing and working with an Azure analytical platform including Azure Databricks, Azure Data Factory, Azure Data Lake Storage, Delta Lake, Unity Catalog, PySpark, SQL, Python, and Azure DevOps. Strong foundation in data modelling and data warehousing concepts (medallion architecture, facts and dimensions). Experience using DevOps practices ...

clear technical tasks. Mentorship: A proactive mindset, eager to provide guidance to less experienced developers and support collaborative pair-programming. Desirable Skills Experience with PySpark or PySQL. Experience in Dimensional Modelling and data warehousing principles. Experience with SQL Server SSAS Tabular and DAX. Exposure to SQL Server RDBMS ...

enterprise-scale data products, defining technical strategy, architectural patterns and engineering standards while providing technical leadership across teams. Expert programming experience with Python/PySpark and advanced proficiency in SQL for large-scale data transformation, optimisation and analytics workloads. Extensive experience designing and implementing scalable data solutions on cloud ...

depth knowledge of the financial services industry and their IT systems Practical cloud native experience Preferred qualifications, capabilities, and skills Advanced Apache Spark experience(PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse and data platform ...

Resiliency, and SecurityIn-depth knowledge of the financial services industry and their IT systemsPractical cloud native experiencePreferred qualifications, capabilities, and skillsAdvanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns.Deep lakehouse and data platform expertise ...

designs & builds data solutions and mentor junior engineers. We’re looking for someone who can offer the following skills & experience. Strong experience in Python, PySpark/Spark, Databricks, Azure Data Factory, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest. Experience designing, building and supporting scalable data ...

/reporting outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Exposure to data governance concepts: critical data elements (CDEs), lineage, data quality dimensions, audit frameworks. ...

experience, including design and delivery of complex pipelines and data models in a cloud environment Strong proficiency in SQL; solid Python and/or PySpark capabilityDemonstrable experience with Databricks, Apache Spark, and Delta Lake architectureProven ability to design and own enterprise-grade canonical data models across multiple upstream source ...

testing, environment management Solid statistical foundations Insurance experience, ideally Lloyd's or London Market, is a big plus. So is deploying LLM based solutions, PySpark or Power BI. Permanent, up to £90,000 plus 20% bonus, three days a week in the office in the City. #J-18808-Ljbffr ...