76 to 100 of 222 PySpark Jobs

improvement within your teamCollaborate with your peers on technical direction within your teamRequired Skills and ExperienceStrong analytic skills related to working with unstructured datasetsPython (PySpark, Pandas, PyArrow)Distributed data processing (Apache Spark)Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi)Cloud services (AWS, Azure or GCP)Messaging/ ...

improvement within your teamCollaborate with your peers on technical direction within your teamRequired Skills and ExperienceStrong analytic skills related to working with unstructured datasetsPython (PySpark, Pandas, PyArrow)Distributed data processing (Apache Spark)Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi)Cloud services (AWS, Azure or GCP)Messaging/ ...

fields.· Proven track record of designing, developing, and deploying AI systems in production environments.· Proficient in Python and key ML libraries (e.g. PyTorch, PySpark, scikit-learn, Hugging Face Transformers).· Hands-on experience with modern data platforms and AI tooling such as Azure ML, Databricks, MLflow, LangChain, LangGraph.· Proven ...

Logical, Physical and Dimensional) Extensive Azure Databricks experience within large-scale enterprise environments Databricks Lakehouse Architecture and Medallion Design Patterns Databricks Data Engineering using PySpark and Spark SQL Delta Live Tables (DLT) Unity Catalog implementation and governance Databricks Workflows Databricks Asset Bundles (DABs) Databricks Solution Accelerators Metadata-Driven Ingestion … Power BI semantic models and enterprise reporting environments Technical Environment Azure Databricks Delta Live Tables (DLT) Unity Catalog Databricks Asset Bundles (DABs) Databricks Workflows PySpark Spark SQL MLflow Mosaic AI Azure Data Factory Azure Synapse Azure Data Lake Storage Dynamics 365 Finance & Operations (D365 F&O) Azure DevOps Power ...

cloud‐native experience Preferred Qualifications, Capabilities, and Skills Experience with modern data technologies such as Databricks or Snowflake Hands‐on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

cloud‐native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands‐on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

securityPractical cloud-native experiencePreferred Qualifications, Capabilities, and Skills:Experience with modern data technologies such as Databricks or SnowflakeHands-on experience with Spark/PySpark and other big data processing technologiesDemonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine learning ...

/Azure/GCPExposure to LLM models like ChatGPT, Google Bard etc.Exposure to Langchain, RAG and Fine Tuning of the LLM Models.Exposure to PySpark and Databricks for Data pipeline creation. Familiarity with modern front-end technologiesExposure to cloud technologiesExperience with ReactJS front-end development is a plusJ.P. Morgan ...

Hybrid in The United Kingdom: LondonData Software EngineeringapplyWe're looking for a Senior Data Engineer – Tech Lead (Databricks, PySpark) to join our team in London, UK, in a hybrid working mode.In this role, you will lead the design, development and optimization of scalable cloud-native data architectures, focusing … Azure Databricks, PySpark and Lakehouse principles. You will work hands-on to deliver performant data solutions for high-volume workloads, ensuring governance, reliability and best practices for enterprise-grade platforms.As a technical leader, you will define data strategies, drive modernization initiatives and mentor engineers, fostering excellence and innovation throughout ...

MLflow, Airflow, model monitoring, API monitoring, data validation, data drift detection, autoscaling and access management. Strong Python experience, with helpful knowledge of Spark or PySpark for working with large‐scale data and machine learning systems. An understanding of feature stores and related data technologies used to support operational machine ...

MLflow, Airflow, model monitoring, API monitoring, data validation, data drift detection, autoscaling and access management.Strong Python experience, with helpful knowledge of Spark or PySpark for working with large-scale data and machine learning systems.An understanding of feature stores and related data technologies used to support operational machine learning products.Clear ...

safe scaling patterns and reuse.Preferred qualifications, capabilities, and skillsExperience with modern data platforms such as Databricks or SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model ...

Agentic AI, applied within enterprise environments.Experience with energy system modelling, grid analytics, demand forecasting or energy market data.Familiarity with distributed computing, Spark, PySpark or cloud-native data platforms.Experience developing models for planning, scenario analysis, operational decision support, model governance or regulatory reporting.Why Join CapcoDeliver high-impact technology and analytics ...

also a key driver in translating Vision and Data Strategy into IT solutions and delivering on them. Strong hands-on expertise in Databricks, PySpark, Python and SQL is central to this role. Key Skills: Azure Stack ELT Experience Data Modeling Data Integration & Ingestion Data Manipulation and Processing GitHub, GitHub ...

methodology. Platforms & Tools Experience with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large‐scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

apply for travel visas as required by travel needsPreferred qualifications • Additional data and analytics experience in Hadoop framework and coding using Impala, Hive, or PySpark • Experience generating new knowledge or creating innovative solutions for a firm • Relevant industry expertise • Working knowledge of Cloud platforms (AWS; Azure; Google Coud ...

that meet Barclays standards and drive decision-making.Some other highly valued skills include:Proficiency in SQL and experience working with relational databases.Knowledge of Python, PySpark, and data analytics techniques.Experience with SAP BusinessObjects (SAP BO).Strong communication and stakeholder management skills, with the ability to engage effectively across technical ...

that meet Barclays standards and drive decision-making.Some other highly valued skills include:Proficiency in SQL and experience working with relational databases.Knowledge of Python, PySpark, and data analytics techniques.Experience with SAP BusinessObjects (SAP BO).Strong communication and stakeholder management skills, with the ability to engage effectively across technical ...

delivery · Prepare and structure data so that it can be used effectively by AI-enabled analytics and data platform capabilities. · Work with SQL, Python, PySpark, notebooks or platform-native tooling to transform, test and expose data for downstream AI and analytics use cases. · Collaborate with data engineers and architects … understanding of data engineering or analytics engineering concepts, including data modelling, pipelines, ELT/ETL, data quality and analytical data products. · Experience using Python, PySpark, notebooks or equivalent platform tooling for data preparation, analysis, testing or automation. · Understanding of semantic models, business definitions, metrics, metadata and how they improve ...

cloud‐native experience Preferred Qualifications, Capabilities, and Skills Experience with modern data technologies such as Databricks or Snowflake Hands‐on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

securityPractical cloud-native experiencePreferred Qualifications, Capabilities, and Skills:Experience with modern data technologies such as Databricks or SnowflakeHands-on experience with Spark/PySpark and other big data processing technologiesDemonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine learning ...

datasets & commercial forecasting processes.Familiarity with web-analytics tools & optimizing user interfaces.Experience with survey exchange platforms or online market research products.Experience with Databricks, Spark, or PySpark for scalable data processing.Additional Information#LI-Remote#LI-JC1Our ValuesCollaboration is our superpowerWe uncover rich perspectives across the worldSuccess happens togetherWe deliver across borders.Innovation ...

LLMs for open-end generation.Experience with probabilistic modeling, or advanced profiling techniques.Familiarity with online market research or survey exchange platforms.Experience using Databricks, Spark, or PySpark for large-scale workflows.Additional Information#LI-RemoteOur ValuesCollaboration is our superpowerWe uncover rich perspectives across the worldSuccess happens togetherWe deliver across borders.Innovation ...

complex data led organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design ...