51 to 75 of 119 Permanent PySpark Jobs

Data Manager/Lead Location : London (Hybrid) Comp : £115,000 P/A (FTC) Duration: 12 Months Skills: Python (Programming Language), SQL, Apache Spark, PySpark, Azure Databricks, Data Lakes, Data Warehouse Architecture, Leadership, Software Development, Management The role We're partnering with a large, data-driven organisation seeking ...

excellence. Champion AI and data architecture as strategic enablers of business transformation—linking technology decisions directly to measurable client outcomes. Technologies & Concepts Programming & APIs: PySpark , Python, FastAPI , FastMCP , SQL, OAuth2 AI Architectures: Retrieval-Augmented Generation, In-Context Learning, Agent Design, Function Calling, LLM Fine-Tuning & Adaptation Strategies Data Analytics ...

passion for solving complex data challengesWhile we advocate using the right tech for the right task, we often leverage the following technologies: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such ...

machine learning operations. Excellent communication skills with the ability to influence cross-functional teams and stakeholders. Experience with Python, SQL, Java, HQL, Hive, Pyspark, or similar programming languages preferred. Well-versed in design thinking principles. Great problem-solving and analytical skills. The total cash range for this position ...

DESIRABLE Understanding of SQL and relational datasets Experience building dashboards in SaaS (software as a service) products Knowledge of DAX, Power Query Knowledge of Pyspark Experience supporting live dashboards and reporting environments Experience automating reporting solutions Experience of other reporting and dashboard tools (Databricks, Qlik, Tableau, Looker, Amazon ...

implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models (bronze/silver/gold), data quality checks … Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT patterns, schema evolution, partitioning, Z-ORDER, OPTIMIZE/VACUUM ...

implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models (bronze/silver/gold), data quality checks … Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT patterns, schema evolution, partitioning, Z-ORDER, OPTIMIZE/VACUUM ...

data platforms. Responsibilities Design, develop and maintain scalable data pipelines using Microsoft Fabric, Azure Databricks and Azure Synapse. Develop ETL/ELT solutions using PySpark and SQL. Build and optimise data integration solutions using Azure Data Factory and Microsoft Fabric. Design scalable lakehouse and data warehouse solutions. Optimise data … engineering standards. Troubleshoot production issues and continuously improve platform performance. Requirements Strong commercial experience as a Data Engineer (5+ years). Strong experience with PySpark . Advanced SQL skills. Experience with Microsoft Fabric , Azure Databricks and/or Azure Synapse Analytics . Experience building data pipelines using Azure Data ...

teams to deliver trusted datasets, performant ETL/ELT pipelines, and well-governed data products. The role emphasizes batch/streaming pipelines, Spark/PySpark, Delta Lake, governance and security, with CI/CD and collaboration across environments. #J-18808-Ljbffr ...

junior engineers while translating Vision and Data Strategy into IT solutions and driving delivery across programs. The role requires hands-on work with Databricks, PySpark, Python and SQL, with a focus on data modeling, integration and cloud data services. #J-18808-Ljbffr ...

promote best practice across the analytics function. Independently own projects from discovery through to delivery. Skills & Experience Strong SQL skills. Python and/or PySpark experience. Experience within Product Analytics or Digital Analytics. Strong understanding of experimentation and A/B testing. Good grounding in statistics and analytical methodologies. ...

harness data.What we expect from you Degree in Statistics, Mathematics, Physics, Economics, or a related quantitative fieldStrong programming skills — Python and SQL are essential; PySpark experience is highly advantageousExperience with version control tools (e.g. GIT)Solid understanding of analytical technologies, tools, and techniquesExcellent logical thinking and problem‐solving abilitiesStrong ...

What we expect from you Degree in Statistics, Mathematics, Physics, Economics, or a related quantitative field Strong programming skills - Python and SQL are essential; PySpark experience is highly advantageous Experience with version control tools (e.g. GIT) Solid understanding of analytical technologies, tools, and techniques Excellent logical thinking and problem ...

unlock business value. You will collaborate with data scientists, analysts and software engineers, implement governance and security, and contribute to engineering standards. Strong PySpark and SQL skills, plus experience with cloud platforms are essential for success. #J-18808-Ljbffr ...

engineering is done across the organisation. Skills & Experience Strong hands‐on experience with Azure data platforms Advanced SQL skills Commercial experience using Databricks and PySpark Proven background building and maintaining scalable data pipelines #J-18808-Ljbffr ...

understand the 15/60 PM counter granularity and able to write codes for aggregation Build and optimize distributed data processing jobs using PySpark on AWS EMR with deployment of these jobs on Data warehouse platforms like Amazon Redshift & ClickHouse Work with Amazon MSK (Kafka) for real‐time data ...

Cloud Data Engineer About the Job you are considering: We are seeking an experienced Cloud Data Engineer with expertise in AWS, Snowflake, dbt, PySpark, and SQL to design, build, and maintain scalable cloud-based data platforms and analytics solutions. The ideal candidate should have strong experience in cloud data … data ingestion, transformation, validation, and reporting. Develop and maintain dbt models, tests, snapshots, macros, and documentation. Create high-performance data transformation solutions using PySpark and SQL. Design and optimize Snowflake data warehouse solutions. Integrate data from multiple sources including databases, APIs, cloud storage, and third‐party systems. Implement data ...

code backing our platform Qualifications: BS in Computer Science/Engineering (or related career field) preferred Strong Data Engineering background Experience with Spark, Pyspark, and/or Pandas Extensive experience coding/programming in: Python SQL, R or Similar Strong understanding of algorithms and data structures as they pertain ...

experience with joining, analysing, manipulating commercial and regulatory data for visualisation (Tableau) and apps Demonstrable experience in utilising data analytics tools and libraries (Pandas, PySpark, etc) Understanding of financial markets Experience in applying machine learning techniques to market data Experienced in developing user-friendly analytical products, e.g self-service ...

solutions Technical modelling: Experience building models on both static and time series data, including propensity and forecasting models Coding skills: Advanced SQL, Python and PySpark skills, built on solid software development principles Stakeholder management: Ability to work with both technical and non-technical stakeholders, with strong communication skills ...

data platforms, analytics engineering, and modern Lakehouse solutions. The client is looking for a hands‐on engineering professional with strong Microsoft Fabric, Spark, PySpark and SparkSQL expertise, capable of designing, building and supporting enterprise‐scale data platforms. Start Date: ASAP Duration: 3 Months Initially Pay Rate: Competitive IR35 Status … Outside IR35 Location: London (1 day per week onsite, remainder remote) The Role: The core requirement is strong Lakehouse expertise, particularly Spark, PySpark, and SparkSQL. Hands‐on experience in low‐code is essential. Data & Analytics Expertise: Data architecture (Lakehouse, Medallion, Kimball, 3NF). Data ingestion and processing. Data quality ...

with architects, engineers, and analysts to deliver secure, scalable, and high-performing migration solutions, leveraging technologies such as AWS Glue, Apache Iceberg, Python/PySpark, SQL, and YAML-based configurations.You will thrive in a collaborative, client-facing environment, with a passion for solving complex data challenges, ensuring delivery excellence … including AWS Glue and Apache IcebergValidate data feed pipeline migrations transitioning to new AWS Data Lakehouse architecturesTest pipelines underpinned by:AI ETL acceleratorsPython/PySpark transformationsSQL-based logicYAML configuration-driven orchestrationExecute and validate:Initial bulk data loadsSubsequent incremental/delta migrationsDevelop and run SQL-based validation and reconciliation queries ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

London, United Kingdom | Posted on 23/07/2026 Contract Role: Senior AI/ML Research Engineer Contract Location: London Contract Duration: 6 months initially (with potential extension) Contract Start Date: Immediate Contract Classification ...