151 to 175 of 394 PySpark Jobs

experience with Databricks — running queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large‐scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

coding languages eg. Python, R, Scala, etc.; (Python preferred) Proficiency in database technologies eg. SQL, ETL, No‐SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques ...

Strong foundation in Python and SQL Experience building models for real commercial use cases (e.g. propensity, clustering, regression) Comfortable working with tools like PySpark, Databricks, Tableau or Power BI (nice to have, not essential, since the group uses a mix across its businesses) A track record of translating technical ...

Knowledge & Skills • Analytics: exploratory analysis, root-cause analysis, and KPI design • SQL: complex joins, CTEs, window functions, query performance optimization • Python (and/or PySpark) for analysis, data validation, and automation. Familiar with VSCode and Databricks Ideally • Ability to interpret telemetry and pipeline outputs in business context, including communicating ...

interrogate data andverify correctness Hands-on experience with Databricks — running queries, navigatingUnity Catalog, reading Spark job outputs and understanding what they Working knowledge of PySpark or Spark SQL — enough to read pipelinecode, understand transformations, and trace where data Understanding of Lakehouse/medallion architecture (bronze-silvergold) and how data ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

language models Experience with experiment design and conducting A/B tests Experience building shared or platform-style ML systems Experience with Databricks and PySpark Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with ...

economists and data scientists. Based in London with a hybrid model (3 days on site), the role requires 8+ years in data engineering, strong PySpark/SQL, and active SC clearance. You will implement governance, Delta Lake, Unity Catalog, and CI/CD practices to ensure reliable #J ...

covering statistical analysis, A/B testing, time-series, regression, classification and optimisation problems. Proficient in SQL & Python. Familiarity with data manipulation libraries (e.g. PySpark, Pandas, Numpy) and machine learning frameworks (e.g. scikit-learn, TensorFlow or PyTorch). Experience with MLOps tools (e.g., MLflow, Databricks Model Registry, Unity Catalog ...

Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the entire value chain of organisations. We are proud ...

cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

/GCP Exposure to LLM models like ChatGPT, Google Bard etc. Exposure to Langchain, RAG and Fine Tuning of the LLM Models. Exposure to PySpark and Databricks for Data pipeline creation. Familiarity with modern front-end technologies Exposure to cloud technologies Experience with ReactJS front-end development ...

Microsoft Azure. You will build production-grade data pipelines with Delta Lake, Unity Catalog, and Databricks tools. The role requires hands-on expertise in PySpark, Python, SQL, and Spark architecture, plus experience with GitLab-based CI/CD, Databricks Asset Bundles, and Azure Data Lake Storage. #J-18808-Ljbffr ...

months Location: Northampton, UK --------------------------------------- JOB DETAILS Role Title: Sr Data Engineer • Minimum 6-8 years of experience working as Sr Data Engineer • Bigdata Technologies – PySpark, Spark • Coding Languages – Python, SQL • AWS services – Glue, Cloud formation, Lake Formation, IAM, Lambda, Redshift • Exposure to cloud technologies – Snowflake, Redshift, Apache Flink ...

through production deployment and monitoring — and know where each stage tends to go wrong.You're fluent in Python and SQL, and you've used PySpark or Spark to process data at scale.You've built agentic AI workflows — not just used off-the-shelf tools, but designed the automation yourself.You ...

troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental data loadingCDC (Change Data Capture) implementationData quality controls, schema evolution, reconciliation, and error ...

successful candidate will design and implement modern data lakehouse architectures, ensuring high performance and scalability. Key responsibilities include developing data pipelines using Python and PySpark, working with AWS services, and implementing CI/CD pipelines. Strong experience in AWS Data Engineering and knowledge of data quality frameworks are essential. ...

work with data engineers and product teams, and support CI/CD practices in a DataOps context. The role requires strong SQL, Python/PySpark skills and experience with Azure Databricks, ADF and ADLS Gen2, in a hybrid setup with several Leeds-area locations. #J-18808-Ljbffr ...

environment. What You'll Need: Strong experience as a Senior Azure Data Engineer An expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity Catalog Knowledge of Kimball data modelling, such as star schema, snowflake, etc. A flexible approach to getting things done ...

Требования Strong SQL skills, including window functions, CTEs, and aggregations; Hands-on experience with Databricks, Unity Catalog, and Spark job outputs; Working knowledge of PySpark or Spark SQL; Understanding of Lakehouse and medallion architecture; Familiarity with YAML-based configuration and structured test definitions; Comfortable with Git and basic engineering ...

Databricks/Delta Lake builds - ACID transactions, time travel, schema evolution, and performance tuning (OPTIMIZE, Z-ordering, VACUUM) Strong command of medallion architecture and PySpark, lazy evaluation, partitioning, broadcast joins, and diagnosing partition skew Solid Azure data platform experience; familiarity with Azure Analysis Services or a comparable semantic layer ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

interest in developing into an architectural role Experience working closely with Data Engineers, Analytics teams, and business stakeholders Exposure to Microsoft Fabric, Databricks, Python, PySpark, or dbt would be beneficial Experience with data modelling tools such as Sparx EA, Sparx Enterprise Architect, Enterprise Architect, ER/Studio, ERwin, Erwin ...

experience with Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added ...