126 to 150 of 201 PySpark Jobs

understand the 15/60 PM counter granularity and able to write codes for aggregation Build and optimize distributed data processing jobs using PySpark on AWS EMR with deployment of these jobs on Data warehouse platforms like Amazon Redshift & ClickHouse Work with Amazon MSK (Kafka) for real‐time data ...

clientsFull SDLC process experienceExcellent communications skills, both written and verbalBonus Points:Experience with the Go Lang eco-system tools and librariesExperience with Pandas and PySpark libraries would be a plusTechnical experience with Databricks or any similar Delta Lake platforms.Experience working with large datasetsExperience with AWS cloud deployments and APIExperience ...

Cloud Data Engineer About the Job you are considering: We are seeking an experienced Cloud Data Engineer with expertise in AWS, Snowflake, dbt, PySpark, and SQL to design, build, and maintain scalable cloud-based data platforms and analytics solutions. The ideal candidate should have strong experience in cloud data … data ingestion, transformation, validation, and reporting. Develop and maintain dbt models, tests, snapshots, macros, and documentation. Create high-performance data transformation solutions using PySpark and SQL. Design and optimize Snowflake data warehouse solutions. Integrate data from multiple sources including databases, APIs, cloud storage, and third‐party systems. Implement data ...

troubleshooting, with the ability to think beyond routine approaches to solve technical problems.Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, and Databricks on AWS, processing and transforming large-scale financial datasets for analytics and reporting.Develop fact and dimension data models for reporting … Code).Required qualifications, capabilities, and skillsProven experience in data management, ETL/ELT pipeline development, and large-scale data processing.Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning.Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar).Experience with AWS cloud services ...

code backing our platform Qualifications: BS in Computer Science/Engineering (or related career field) preferred Strong Data Engineering background Experience with Spark, Pyspark, and/or Pandas Extensive experience coding/programming in: Python SQL, R or Similar Strong understanding of algorithms and data structures as they pertain ...

experience with joining, analysing, manipulating commercial and regulatory data for visualisation (Tableau) and apps Demonstrable experience in utilising data analytics tools and libraries (Pandas, PySpark, etc) Understanding of financial markets Experience in applying machine learning techniques to market data Experienced in developing user-friendly analytical products, e.g self-service ...

solutions Technical modelling: Experience building models on both static and time series data, including propensity and forecasting models Coding skills: Advanced SQL, Python and PySpark skills, built on solid software development principles Stakeholder management: Ability to work with both technical and non-technical stakeholders, with strong communication skills ...

e.g. Mathematics, Statistics, Computer Science). Nice to Have: Familiarity with production environments, MLOps, or data pipelines. Experience working with large-scale data using PySpark or equivalent distributed data processing tools. Experience in gaming, e-commerce, or subscription-based products.Benefits: Discretionary bonus opportunityPrivate Medical InsuranceDental Scheme25 days holiday ...

business acts on.Fluency with large, complex behavioural datasets and the craft to work with them at scale: Python, SQL, and distributed processing (e.g. PySpark), plus standard ML tooling as the work requires.Excellent stakeholder communication: you can make a rigorous causal argument land with a non-technical commercial audience ...

data platforms, analytics engineering, and modern Lakehouse solutions. The client is looking for a hands‐on engineering professional with strong Microsoft Fabric, Spark, PySpark and SparkSQL expertise, capable of designing, building and supporting enterprise‐scale data platforms. Start Date: ASAP Duration: 3 Months Initially Pay Rate: Competitive IR35 Status … Outside IR35 Location: London (1 day per week onsite, remainder remote) The Role: The core requirement is strong Lakehouse expertise, particularly Spark, PySpark, and SparkSQL. Hands‐on experience in low‐code is essential. Data & Analytics Expertise: Data architecture (Lakehouse, Medallion, Kimball, 3NF). Data ingestion and processing. Data quality ...

with architects, engineers, and analysts to deliver secure, scalable, and high-performing migration solutions, leveraging technologies such as AWS Glue, Apache Iceberg, Python/PySpark, SQL, and YAML-based configurations.You will thrive in a collaborative, client-facing environment, with a passion for solving complex data challenges, ensuring delivery excellence … including AWS Glue and Apache IcebergValidate data feed pipeline migrations transitioning to new AWS Data Lakehouse architecturesTest pipelines underpinned by:AI ETL acceleratorsPython/PySpark transformationsSQL-based logicYAML configuration-driven orchestrationExecute and validate:Initial bulk data loadsSubsequent incremental/delta migrationsDevelop and run SQL-based validation and reconciliation queries ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

London, United Kingdom | Posted on 23/07/2026 Contract Role: Senior AI/ML Research Engineer Contract Location: London Contract Duration: 6 months initially (with potential extension) Contract Start Date: Immediate Contract Classification ...

London-based programme focused on data platforms, analytics engineering and Lakehouse solutions. You will design, build and support enterprise-scale data platforms with Spark, PySpark, SparkSQL and Microsoft Fabric. The role requires hands-on coding, data governance, ingestion and API integration capabilities, plus a passion for modern analytics ...

qualifications include proficiency in site reliability culture, incident management, and tools like AWS and Databricks. Candidates should also demonstrate strong skills in Python or PySpark, as well as AI-assisted software development. #J-18808-Ljbffr ...

visible role where you'll combine strong technical skills with commercial thinking to drive meaningful product improvements. Your Skills & Experience Strong SQL skills Python (PySpark experience is beneficial) Experience in Product Analytics Strong knowledge of A/B testing and experimentation Excellent statistical analysis skills, with exposure to modelling ...

visible role where you'll combine strong technical skills with commercial thinking to drive meaningful product improvements. Your Skills & Experience Strong SQL skills Python (PySpark experience is beneficial) Experience in Product Analytics Strong knowledge of A/B testing and experimentation Excellent statistical analysis skills, with exposure to modelling ...

outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices. Proficiency in Python or PySpark for AI/ML modeling. Must be able to reduce toil by building new tools to automate repeated tasks. Hands‐on experience in system ...

analytics capabilities. We welcome candidates from all backgrounds to apply, and we are committed to creating a diverse workforce.Key Responsibilities and Skills: - Proficiency in PySpark for effective data manipulation and transformation. - Experience with Microsoft B.I and DataBricks for advanced analytics solutions. - Strong problem-solving abilities to address complex challenges. ...

global organization through resourcing/location strategy. Experience and Qualifications Proven track record delivering solutions in complex organizations. Experience with AWS, Databricks, DBT, PySpark, React, JavaScript, Terraform, etc. Fast learner, thrives in ambiguity, translates vision into product scope. Passion for automation and user experience innovation. Builds high‐performing teams ...

location strategy.“I am the person Capital Group is looking for.” Proven track record delivering solutions in complex organizations. Experience with AWS, Databricks, DBT, PySpark, React, JavaScript Terraform, etc. Fast learner, thrives in ambiguity, translates vision into product scope. Passion for automation and user experience innovation. Builds high-performing ...

structured and unstructured datasets Validate large-scale data ingestion, transformation, and processing workflows Ensure data integrity, transformation accuracy, reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis … DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark/PySpark, Terraform, Docker Key Skills Strong proficiency in Python, SQL, Java, and Scripting languages such as Bash Hands-on experience testing Big Data platforms and Data Lake solutions Strong experience with ...

play a key role in advancing our data capabilities and driving innovation. Job Responsibilities: Develop workflows and ELT data pipelines using Python, Spark/PySpark, and Databricks Onboard enterprise datasets into Palmos, including ingestion, transformation, and validation of data assets Build, test, and maintain scalable data pipelines and data … coding standards; contribute learnings and reusable patterns to improve broader team effectiveness Required Qualifications, Capabilities, and Skills: Hands-on experience with Databricks, Spark/PySpark, Python, and SQL Experience developing and maintaining data pipelines and data processing systems Understanding of the data lifecycle, including ingestion, transformation, storage, and consumption ...

DescriptionRole –Senior Databricks ArchitectTechnology – Databricks (Spark, PySpark/SQL) and data lake architecturesLocation – London, UKCompensation – Competitive (including bonus)Job DescriptionWe are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role focuses on integrating legacy … systems with cloud-native platforms to support analytics, reporting, and domain-owned data products.Primary Skills• Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures.• Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka.• Experience integrating legacy data sources such as DB2 (mainframe ...